Auto-tools/Goals/2011Q4: Difference between revisions

* {{ok|Achieve and maintain a 5% or less intermittent orange rate for Android automation (talos and tests) to achieve parity with desktop intermittent orange rates.}}

** {{ok|Land modifications to make it easier for developers to run Android talos (to make it easier to debug failures)}}

** {{ok|Create and land specialpowers-core JS module to simplify test utilities and reduce bugs in "alive test", talos, reftest, and mochi* test families}}

* {{ok|Expand and enhance our test and performance automation to anticipate and address needs for Fennec and B2G.}}

** {{ok|Deliver video capture data from an alive test (Ts/Tp) an a "driver" test (Tuser) from eideticker to developers for analysis. Stretch: have system ready for duplication by end of Quarter}}

** {{ok|Deploy Talos RSS metrics in Android production}}

** {{ok|Land marionette with ability to "runscript" and enough action support to run Tuser eideticker test on both B2G and Fennec.}}

** {{ok|Land enhancements to special powers to entirely remove enablePrivilege from all testing frameworks}}

* {{ok|Deploy user responsiveness automation into production}}

** {{ok|Deploy Talos User Responsiveness test on desktop and android automation}}

** {{ok|Deploy Regression User Responsiveness (peptest) on desktop production automation}}

** {{ok|"Make Android Remote Talos" must be landed}}

** {{ok|Create specialpowers core to use in other tests, may not prove necessary}}

** {{ok|Deliver video capture data from an alive test (Ts/Tp) an a "driver" test (Tuser) from eideticker to developers for analysis.}}

** {{ok|Have system stable enough to be ready for duplication in MV by end of quarter (stretch)}}

** {{ok|Define two tests for completion at end of Quarter}}

** {{ok|Integrate with marionette JSON protocol to drive device under test}}

** {{ok|Potentially use JSON protocol and shim (supplied by selenium project) to drive non-gecko browsers on device (stretch/Q1)}}

** {{ok|Create receiver for JSON protocol}}

** {{ok|Implement enough action to support the Eideticker tests}}

** {{ok|Remove all enablePrivilege calls in all test frameworks and land patches in m-c}}

** {{ok|Deploy into production for desktop and android}}

** {{ok|Finalize equation for responsiveness metric}}

** {{ok|Determine if test can be deployed as addition to Tp5}}

** {{ok|UI Deployed, UI taken through find/fix cycle}}

** {{ok|Create repeatable, deployable build/flash system to get a functioning board running}}

** {{ok|Analyze results from unit and performance tests on board, but not in production this quarter}}

** {{ok|Work with Release Engineering MozHarness to deploy a MozHarness/MozBase Talos system}}

** {{ok|Attend meeting, ensure actions created from it are helpful to our efforts}}

** {{ok|Situate ourselves for greater involvement in Q1.}}

** {{ok|Adapt the harness to run in E10S firefox and fennec}}

** {{ok|All Mutt tests should pass in E10S mode}}

** {{ok|Demo at Mobile Summit for developers as a solution for new/hard to get hardware}}