canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,747
edits
QA/Test Automation/2010-11-17: Difference between revisions
QA/Test Automation/2010-11-17 (view source)
Revision as of 15:30, 18 November 2010
, 18 November 2010→Meeting Notes
| Line 85: | Line 85: | ||
= Meeting Notes = | = Meeting Notes = | ||
* Goals Review and Additions | |||
** {{prev|}} Mozmill Crowd extension | |||
*** Test environments for Windows and Linux have to be setup | |||
*** Need functionality to download and install the test environment | |||
*** Update the extension to use nsIProcess instead of IPC-PIPE component | |||
*** Demo of the extension during the all hands week | |||
== Action | ** {{ok|}} Mozmill Dashboard (update and add-on tests) | ||
*** Once the automation script sends all the data up to brasstacks (later this week) we have to think about the visualization of the results | |||
*** For add-on tests we will setup an addon testrun for Selenium IDE | |||
** {{ok|}} Buildbot execution of tests with local test data | |||
*** behind on creating new local test files - we can use the test day to get progress | |||
*** 35 tests moved to local test data last quarter | |||
*** 4 test files for this quarter / about 20 test files remaining | |||
*** we should make sure that tests with already local test data are stable and can go to buildbot | |||
*** what is the plan to get a specific set of tests to get run on buildbot | |||
**** tag tests with meta data in the file which can be used by a script to copy over | |||
** {{ok|}} Refactoring the shared module system | |||
*** not a lot of progress in the work week | |||
*** implementation of the initial classes | |||
*** we will have at least the majority pieces in place to implement most of our tests | |||
*** pick tests from nearly each of the existing test groups for a broad coverage | |||
** {{ok|}} Mozmill Panorama tests | |||
*** Lead needed for coordination -> Cameron? Who is 2nd? | |||
*** Geo will talk with Clint about drag and drop requirements | |||
*** Given by Anthony there should be 1/2 of the tests on Litmus be able to run without D&D | |||
*** Geo will come up with some performance tests | |||
*** Focus on already existent tests on Litmus | |||
**** Figuring out with QA leads and devs which of those tests are important to automate | |||
** {{new|}} Prototype for endurance tests | |||
*** For first thoughts see the [http://etherpad.mozilla.com:9000/mozauto-101119 etherpad] | |||
*** Have to find a lead and 2nd | |||
** {{new|}} Define trigger mechanism to start Mozmill test-runs | |||
*** whatever sort of notifications even manual ones to start test-runs on all the platforms | |||
*** Send out emails when failures happen | |||
*** Big impact of Mozmill tests to run RelEng update tests | |||
*** Talk to RelEng to define the type of notification (Lead: Geo, 2nd: Henrik) | |||
**** No ETA for RelEng inclusion | |||
**** We have requested a way to get notification from RelEng for our test-runs | |||
**** Make our update tests absolutely trustful before running them in the RelEng environment (i.e nightly-test channel) | |||
* Handling of manual testcases in Litmus for automated tests | |||
** Each automated test needs a manual test | |||
** Where they have to be located? | |||
*** Keep in mind that Litmus 2 is not that far away. | |||
*** jsdoc vs attachment on Bugzilla | |||
**** jsdoc can handle changes more gracefully - one step with script changes | |||
**** jsdoc comments can be automatically pulled | |||
*** Everything should be on one place, mark tests for automation only | |||
==Action items== | |||
Matt (Henrik): Goals review and addition of new goals to team goals | |||
Geo (Henrik): Schedule meeting about trigger notifications for test-runs | |||
Aaron (Geo): Tagging of tests and how to get them into buildbot | |||
Matt (Henrik): Finding a lead and 2nd lead for the Panorama goal | |||
? Al (Anthony)?: Check existing tests if they are already covered by mochitests and worth to transform local | |||
Henrik: Work with Waverly on updating the spreadsheet for Add-ons Manager tests - add labels for automated tests | |||