QA/Test Automation/2011-01-12: Difference between revisions

← Older edit

QA/Test Automation/2011-01-12 (view source)

Revision as of 22:43, 12 January 2011

4,981 bytes added , 12 January 2011

→‎Goals Overview

Mevans

1,285

edits

@@ Line 1: / Line 1: @@
-<small>[[QA/Test_Automation/2010-10-20|previous meeting]] | [[QA/Test_Automation|Meetings]] | [[QA/Test_Automation/2010-12-01|next meeting &raquo;]]</small>
+<small>[[QA/Test_Automation/2010-12-15|previous meeting]] | [[QA/Test_Automation|Meetings]] | [[QA/Test_Automation/2011-01-26|next meeting &raquo;]]</small>
 =Dial in=
@@ Line 7: / Line 7: @@
 =Attendees=
-*
+* Geo, Matt, Al, Henrik, Aaron, Anthony, Adrian
 = Last weeks action Items =
 * Geo: Update documentation for handling of failing Mozmill tests
-* Anthony: Update coding styles and keep discussion on-going to get remaining important rules defined
+** Did not update this, but the handling is somewhat up in the air right now. After the roundtable from this meeting should know better how to proceed.
+* {{done|}} Anthony: Update coding styles and keep discussion on-going to get remaining important rules defined
 = Goals Overview =
 * [https://intranet.mozilla.org/QA/Q42010_QAgoals#Automation_QA_Goals Q4 goals]
+** Letter Grade '''B''' for the quarter
 * [https://intranet.mozilla.org/QA/Q12011_QAgoals#Automation_QA_Goals Q1 goals] planning
+* Proposals
+** Triggered Update Tests
+*** Integrate triggered Mozmill update tests into nightly automation process
+**** One goal, but lower priority because it's an incremental change
+*** what about release candidate builds? For Q2?
+**** Release candidate tests are the higher priority for Q1. They will be on-demand, rather than triggered
+**** Separate goal
+**** Measurement will be "Matt can run them."
+** Endurance Tests
+*** Create suite of Mozmill tests to stress-test Firefox in different areas
+*** Performance tests
+** Module Refactoring
+*** Finalize the project by updating modules and tests
+**** Only for mostly used modules and tests for the beginning (focus areas)
+**** Updates per component
+** Collaborate with development teams to get XXX BFT tests automated
+*** We never found a good way to specify a realistic goal, whether via number of tests nor per component. Do we have other options?
+*** Given the Firefox 4 priority and other goals, how much time we would have this quarter?
+**** Note: let's carry over panorama test goal for this.
+*** Should we better move our efforts to keep the testsuite green?
+**** Not a Collaboration goal, can be a process goal
+** Testing Server
+*** Setting up a testing server which allows kill our dependencies on external websites.
+**** iterative effort
+**** need to create a feature milestone implementation plan to define what will be delivered in chronological order
 = Risky Goals =
@@ Line 23: / Line 50: @@
 ==Panorama Tests (Anthony/Al)==
-* Short updates
+* 4 / 15 doable tests currently in development/review
+* 6 tests currently blocked by drag & drop ({{bug|515776}})
+* Will be looking for team assistance once 4.0b9 work dies down (early next week)
+* Proposed Team Goal: All non-blocking tests automated
-==Triggered software updates (Geo/Henrik)==
+==Software updates (Geo/Henrik)==
-* Short updates
+* Finished PoC for triggered nightly updates.
+** Takes Pulse message
+** Downloads appropriate build
+** Runs update tests, including checking that end-build == expected build ID.
+* Emphasis on updates for next few weeks will be streamlining on-demand release updates, since that's expected to be the immediate process pain point
 ==Mozmill Crowd Extension (Henrik/Aaron)==
-* Short updates
+* Version 0.1 is live on [https://addons.mozilla.org/en-US/firefox/addon/270896/ AMO]
+* We have to inform localizers ASAP so they can run our tests against their locales and fix remaining issues before RC1
+* Version 0.2 planned for end of this month (non-blocking process)
+* Submitted proposal for my talk at FOSDEM 2011
 ==Endurance Tests (Dave/Aaron)==
-* Short updates
+* Update project page with notes from this week's meeting, roadmap redefined
+** Project page: https://wiki.mozilla.org/QA/Mozmill_Test_Automation/Endurance_Tests
+** Meeting notes: http://etherpad.mozilla.com:9000/endurance-tests-10-01-11
 ==Redesign Shared Modules (Geo/Henrik) ==
-* Short updates
+* PoC stage reached in December, running "toy" tests.
+* Currently porting real tests over to shake out early-stage issues.
+* Determining a milestone-based plan including pilot periods where the team as a whole can help contribute to porting (and also test out how easy the framework is to develop against)
 ==Mozmill Result Dashboard (Henrik/Anthony)==
-* Short updates
+* Update tests now have their own [http://mozmill-release.brasstacks.mozilla.com/#/update views]
+* A couple of minor issues identified which have to be fixed
 ==Build bot - local data (Anthony/Aaron)==
-* Short updates
+* No status
+* Most likely will not see progress this quarter due to Firefox 4 work and Automation priorities
 ==Others==
 ;DOM Walker + L10n API (Adrian)
-* Short updates
+* Slower progress but working on the last major items to get fixed by Adrian
+** For the most important bug left ({{bug|614579}}), we need input from people knowing about theming stuff, as it seems to be a Firefox bug...
+* Can be executed by the Crowd extension
 ;Tests / Shared Modules
 * General (Henrik)
-** Short updates
+** Nothing major to report
 * Test Refactoring (Anthony)
-** Short updates
+** '''Following is moot until Shared Modules are refactored'''
+** project in place to begin once Firefox 4b9 work dies down
+** will need team assistance to complete everything this quarter (please let me know if you are willing to help)
+** we still have a few [https://wiki.mozilla.org/QA/Mozmill_Test_Automation/Test_Modules_Refactor#Outstanding_Items outstanding items] for discussion
+*** if possible, I'd like to discuss these during the team meetings (resolve 1 per meeting)
+*** agreement would be integrated into refactoring, not styleguide
 * Broken Tests
+** General
+*** We have started combined efforts to get some of the failing tests fixed
+*** We have now [https://wiki.mozilla.org/QA/Mozmill_Test_Automation/Test_Writing/Owners owners] for the different sections which have to take care of broken tests
 ** Firefox 4.0 (Geo)
-*** Short updates
+*** No particular branch-based news, see General above and Round Table below.
 ** Firefox 3.5/3.6 (Anthony)
-*** Short updates
+*** Focusing on 4.0 failures for b9 release
+*** Will focus on branch failures once b9 is out
 ;Automation Scripts / Infrastructure
-** Short updates
+** Download script for builds from FTP is in progress. It already works for release and candidate builds
 = Personal Status =
@@ Line 70: / Line 124: @@
 =Roundtable=
 ;General
+* <strike>Styleguide & Refactoring</strike> - on hold until Shared Module refactoring complete
+** we should fork into two separate projects
+** freeze the styleguide
+** new additions go into refactoring work
+** changes/additions to styleguide should reflect "style" not logical/design changes
+* Failures
+** adopt a module-ownership model to deal with all failures (4.0, 3.6, and 3.5)
+** once we are green, go back to a branch-based model or stay with module-ownership?
 ;Issues