canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,747
edits
QA/Test Automation/2010-09-22: Difference between revisions
QA/Test Automation/2010-09-22 (view source)
Revision as of 16:22, 27 September 2010
, 27 September 2010→Action Items
m (→Personal) |
|||
| (26 intermediate revisions by 6 users not shown) | |||
| Line 7: | Line 7: | ||
=Attendees= | =Attendees= | ||
* Adrian, Henrik, Anthony, Geo, Matt, Al, Aaron | |||
= Last weeks action Items = | = Last weeks action Items = | ||
* Geo: Send out list of broken tests w/ Fx4 and MM1.5 | * Geo: {{miss|}} Send out list of broken tests w/ Fx4 and MM1.5 | ||
* Matt: Check what limits test harnesses on buildbot to use other local webservers (connect with John O'Duin) | * Matt: {{done|}} Check what limits test harnesses on buildbot to use other local webservers (connect with John O'Duin) | ||
* Matt: Action items from release tasks meeting? Need to notify release leads that they're responsible for coordinating resources. | * Matt: {{miss|}} Action items from release tasks meeting? Need to notify release leads that they're responsible for coordinating resources. | ||
* Geo: For Auto-tools meeting next wk, how to incorporate train model going forward | * Geo: {{done|}} For Auto-tools meeting next wk, how to incorporate train model going forward | ||
* Henrik & Matt: How to streamline patch checkin process | * Henrik & Matt: {{miss|}} How to streamline patch checkin process | ||
* Geo: Generate wiki page to sum up tab candy objective | * Geo: {{miss|}} Generate wiki page to sum up tab candy objective | ||
=Goals Overview= | =Goals Overview= | ||
* Current status for [https://intranet.mozilla.org/QA/Q32010_QAgoals#Automation_QA_Goals Q3 goals] | * Current status for [https://intranet.mozilla.org/QA/Q32010_QAgoals#Automation_QA_Goals Q3 goals] | ||
* Planning for Q4 | ** All: Set of tests for a main feature of Firefox 4 | ||
*** Can we make it an accomplished goal? | |||
*** What takes lesser resources: Addons Manager vs. Panorama | |||
** Henrik: Dropping Mozmill Crowd extension? | |||
** Henrik: Web dashboard will be usable for the general test-run | |||
** Geo: Model for re-designing the shared modules | |||
** Adrian: DOM walker and set of l10n tests | |||
* Planning for Q4 (proposals) | |||
** [https://intranet.mozilla.org/2010Q4Goals#QA QA Q4 goals] | |||
** Henrik: First public version of the Mozmill Crowd extension | |||
** Henrik: Dashboard for software updates and addons tests | |||
** Henrik: Continue preparation of tests to get run on buildbot (incl. local data and stablization) | |||
** Henrik: Continue creation of tests for Panorama (performance?) | |||
** Henrik: Refactoring the shared module system | |||
** Henrik: Enhancements to the on demand testing infrastructure | |||
=Projects= | =Projects= | ||
;Mozmill (Henrik) | ;Mozmill (Henrik) | ||
* Buildbot integration with 10 tests will happen today or tomorrow | |||
* Common JS module for a more streamlined module inclusion will be available with Mozmill 1.5.1 (smooth transition for Mozmill 2.0) | * Common JS module for a more streamlined module inclusion will be available with Mozmill 1.5.1 (smooth transition for Mozmill 2.0) | ||
| Line 28: | Line 44: | ||
** Creation of a PoC for Panorama tests during the QA workweek + Demo | ** Creation of a PoC for Panorama tests during the QA workweek + Demo | ||
* DOM Walker + L10n API (Adrian) | * DOM Walker + L10n API (Adrian) | ||
** see my personal notes | |||
* Redesign Shared Modules (Geo) | * Redesign Shared Modules (Geo) | ||
** Got first "refactored" test working | |||
;Local Test Data (Anthony) | ;Local Test Data (Anthony) | ||
* 4 r?, 11 checkin-needed, 10 need patches | |||
;Broken Tests | ;Broken Tests | ||
* Firefox 4.0 (Geo) | * Firefox 4.0 (Geo) | ||
** Fixed {{bug|575250}} for BuildBot set | |||
* Firefox 3.5/3.6 (Anthony) | * Firefox 3.5/3.6 (Anthony) | ||
** No update | |||
;Update Tests (Henrik) | ;Update Tests (Henrik) | ||
| Line 61: | Line 82: | ||
* Last two weeks | * Last two weeks | ||
** {{done|}} Attended CouchCamp | ** {{done|}} Attended CouchCamp | ||
** {{done|}} QA workweek - Creation of a PoC for Panorama | ** {{done|}} QA workweek - Creation of a PoC for Panorama + work on presentation | ||
** {{miss|}} Finish work on the dashboard (general testrun) | ** {{miss|}} Finish work on the dashboard (general testrun) | ||
** Fixes for TabbedBrowsingAPI and SoftwareUpdateAPI | ** Fixes for TabbedBrowsingAPI and SoftwareUpdateAPI | ||
| Line 72: | Line 93: | ||
;Anthony | ;Anthony | ||
* Last two weeks | * Last two weeks | ||
** | ** {{done|}} Patch review queue to 0 | ||
** | ** {{done|}} Patch checkins to 0 | ||
* Next two weeks | * Next two weeks | ||
** Review new patches to 0 (3 remain) | |||
** Commit new check-ins to 0 (11 remain) | |||
** Ensure 10 remaining local-data tests complete | |||
** <strike>Contribute 1 Panorama test</strike> | |||
** {{risk|}} QA results dashboard (if/when ready) | |||
** {{risk|}} QA crowd extension (if/when ready) | |||
;Al | ;Al | ||
| Line 82: | Line 108: | ||
** Finish carryover goals. | ** Finish carryover goals. | ||
* Next two weeks | * Next two weeks | ||
** Going to Bali. | |||
;Geo | ;Geo | ||
* Last two weeks: | * Last two weeks: | ||
** [http://hg.mozilla.org/users/gmealer_mozilla.com/mozmill-refactor/file/682657b71106/firefox/testAwesomeBar/testAccessLocationBar.js refactored testAccessLocationBar.js], functional | ** [http://hg.mozilla.org/users/gmealer_mozilla.com/mozmill-refactor/file/682657b71106/firefox/testAwesomeBar/testAccessLocationBar.js refactored testAccessLocationBar.js], functional | ||
*** Was functional as of last night, minorly broken again right now on a module chaining issue. Will have code pushed to personal hg as soon as that's fixed. | |||
** {{prev|}} collecting team feedback on prototype | ** {{prev|}} collecting team feedback on prototype | ||
** {{prev|}} Fixes for {{bug|573582}}, {{bug|573584}}, {{bug|575250}} | ** {{prev|}} Fixes for {{bug|573582}}, {{bug|573584}}, {{bug|575250}} | ||
*** {{bug|575250}} fixed | |||
** Presentation w/ Aaron | ** Presentation w/ Aaron | ||
*** Rough draft (info dump, really) created at [https://docs.google.com/present/edit?id=0AXEJ45MnstYjZGhkcGRkNHFfN2hyZzczaHN2&hl=en&authkey=CKzF3agM Google Docs] | |||
** QA work week | ** QA work week | ||
*** Finished first rev of performance checkpointing module | |||
*** Demoed for Tab Candy team | |||
*** Much hilarity had with all | |||
* Next two weeks: | * Next two weeks: | ||
** Two more tests for refactor project, one from AwesomeBar, one from elsewhere. | |||
** Bug fixes for broken 4.0 tests | |||
** Any other necessary work to fulfill team goals | |||
;Aaron | ;Aaron | ||
| Line 103: | Line 139: | ||
;Adrian | ;Adrian | ||
* Last two weeks: | * Last two weeks: | ||
** {{ | ** {{done|}} (mostly) finish Documentation + UML Diagrams for CommonA11yL10nAPI (renamed to: DOMUtilitiesAPI) and L10nAPI | ||
** {{ | ** {{done|}} (mostly) implementation of the CommonA11yL10nAPI (renamed to: DOMUtilitiesAPI) and L10nAPI: WIP patch attached to {{bug|562084}} - including two tests for the preferences window | ||
* Next two weeks: | * Next two weeks: | ||
** implement the L10n tests for the update window | |||
** bring the 2 shared-modules and 4 tests through the reviews | |||
=Contributor Status= | =Contributor Status= | ||
| Line 120: | Line 158: | ||
** Requested 32GB for both machines ({{bug|598305}}) - Matt has to approve | ** Requested 32GB for both machines ({{bug|598305}}) - Matt has to approve | ||
=Action Items= | =Meeting Notes= | ||
* Last action items | |||
** Limits of the test harness on buildbot to use other local webservers | |||
*** Pretty strict handling for accessing other servers | |||
*** It would be possible to have tests running on a Talos server; Therefore a test environment has to be setup | |||
* Goals Q3 | |||
** We drop the Mozmill crowd extension from our list | |||
** Dashboard will be ready for general test results by end of this week | |||
** Anthony and Aaron will finish the first part of the local test data project | |||
** Henrik updates the AddonsAPI and the existing tests until end of next week (Aaron will help out once the local data project is done) | |||
** Geo will have a solid plan for the refactoring work by end of this month | |||
** Adrian will work on the DOMWalker to have it ready by end of this month | |||
* Goals Q4 (Finalized by next week) | |||
** {{prev| Develop and deploy a Mozmill crowd-sourcing extension that will allow community users to execute and report results of Firefox 4 mozmill automated tests across the available platforms and configurations used by the Mozilla testing community. }} | |||
** {{new|Integrate local data-based Firefox 4 mozmill tests under Buildbot execution control and reporting}} | |||
** {{new|Collaborate with development team to expand Mozmill Panorama support to include an automated test suite for functional and performance testing of the Firefox 4 Panorama feature.}} | |||
** {{new|}} Implement Mozmill Dashboard views for software update and addon tests | |||
** {{new|}} Refactoring the shared module system based on the specification from Q3 | |||
* Others | |||
** Geo: More often conversations about goals and blockers except our 1-1's and the bi-monthly meeting | |||
*** We will setup our own IRC #mozmill channel and have an internal mailing list (qa-auto) | |||
** On demand test execution improvements | |||
*** Geo: Shouldn't be a team goal regarding time constraints | |||
*** Henrik: We can have at least the download script integration | |||
==Action Items== | |||
* Geo: {{prev|}} Send out list of broken tests w/ Fx4 and MM1.5 | |||
* Geo: schedule in-IRC meeting to do patch-handling proposal | |||
* Geo: {{prev|}} Generate wiki page to sum up tab candy objective | |||
* Henrik: Talk with John O'Duin about specifics of the test environment | |||
* Henrik: {{done|}} Setup IRC channel #mozmill + Mailing list (qa-auto) [{{bug|598883}}] | |||
* Henrik: {{done|}} Send out goals proposals for final rework | |||
* Matt: {{prev|}} Action items from release tasks meeting? Need to notify release leads that they're responsible for coordinating resources. | |||