This is the set of sprints and actions the Test Dev team is taking on for Q4. Due to the holidays and the All-Hands, we have some unusual dates for our sprints this Quarter.
- S1 - Sprint 1 is from Oct 5 - Oct 23 --> 3 weeks
- S2 - Sprint 2 is from Oct 26 - Nov 20 --> 4 weeks
- S3 - Sprint 3 is from Nov 23 - Dec 18 --> 3.5 weeks (contains half thanksgiving week and all-hands though)
Our plan is to do the smaller set of goals in S1, the bulk of the Quarter's work in S2 and spend S3 doing cleanup and polish due to the time constraints there.
The Plan
- OOP Plugins Testing completed
- npruntime tests
|
- Start looking at OOP content tests
- Run all test frameworks in OOP content mode, report issues
|
- Continue OOP Content testing
|
- Take demos, run them on emulator
- Implement interfaces to start process, kill process, query process
- Implement device heartbeat, ID
- Get prototype working for mochitest
- Test on WinMO/WinCE
|
- Implement remaining interfaces for device information and measurement
- Implement log and results handling
- Get mochitest and reftest running completely hands off
|
- Implement necessary portions for Talos
- Get Talos Running
|
- Job control subsystem implemented
- Inventory control subsystem
- Communication with devices
|
- Test Hosting framework per changeset
- Communication with buildbot for per changeset detection
- Queue control
- Hands off running mochitest and reftest
|
- Talos measurement
- Implement Talos pieces
|
- Finalized workflow between worker and server
- Exception handling
- Start using the system to compare against tomcat's "by hand" run
|
- Design system on how to use the results of automation
- Begin work on signature focused testing
|
- Implement signature focused tests
|
- Finish out all kinks in sprint 1 to get Firefox in the db
- Have Firefox and Fennec results in the database
|
- Finish requirements to make more useful to others (find/fix)
|
- Finish requirements to make more useful to others (find/fix)
|
|
|
- Use Shell reftests to add Microsofts suite to our tests
- Add Microsoft tests to the browser reftests
- Create any needed driver file or auxilary scripts and manifest files
|
- cleanup any loose ends, address review comments
|
- Checked in to central on next TM merge
- 1.9.2 tested and ready to go, 1.9.1 also ready
- Push to get RelEng to include the tests in buildbot automation
|
- Work with dmandelin to get Shell test completed and checked in
- Backport build integration work to 1.9.2 (easy) and 1.9.1 (hard, possibly)
|
|
- Provision machines
- Decide on types of machines, how to provision load
- Understand how to parse the results
- Get system running on chron job first (waiting for machines)
|
- Get system running on same onchange switch from the nightly builds
|
|
- Rationalize jetpack testing
- Tie into mozmill
- Write tests for modules
|
- Analyze security considerations with mrbkap
- write tests re security considerations
- Give to labs team
|
|
- Finish buildbot integration bugs
- Fix profile issues
- Small releases as needed
|
- Investigate templetized tree
- work with henrik on error templates
- integration with brasstacks and views on brasstacks for mozmill results
- Small releases as needed
|
- Address concerns and feedback from bugs and users
|
- Create sample testcases from assertions
- Create simple way to submit testcases
- Blog about the effort
- Get sicking, fantasi to review
|
|
|
- Attend meetings, push toward standardized test framework
|
- TBD, not expecting much action from group until Q1 2010
|
- TBD, not expecting much action from group until Q1 2010
|
|
|
|
- Prototype test runner
- Clean up views
- Get ready to separate from brasstacks
|
