QA/Mozmill Test Automation/Endurance Tests/Meetings/05-17-2011
Previous Action Items
- [DONE] Make all existing Endurance tests local. (Dave)
- [ON TRACK] Reach out to JS team about gathering additional metrics. (Dave)
- [ON TRACK] Reach out to JGriffin about collecting information from about:support. (Dave)
- [ON TRACK] Reach out to Metrics team about chart improvements & optimization. (Dave)
- [DONE] Determine phase II for endurance tests project and QA goals. (Dave)
- Fixes for running endurance tests on Nightly (bug 656869) - Patches in review.
- Restart between tests by default (bug 656393, bug 656388, bug 656404) - Patches in review.
- Pre-calculate statistics (bug 644654, bug 650382) - Patches in review.
- Show standard test results in report (bug 657347) - Patch in review
- [ASSIGNED] bug 635310 - Mozmill Endurance test for flash video performance - ashughes
- Currently reaching out to Benoit Girard and Chris Jones to get more detailed information about Flash rendering
- Prelim: video, ads, and games each use different rendering methods -- will likely mean 3 separate tests
- [ASSIGNED] Switching tabs in Panorama (bug 657368) - ashughes
Discussion Items / Updates
- Resources for new tests/failures/development
- More potential changes coming up in Firefox 6.0 (bug 657327)
- Identify desirable static and dynamic metrics
- Create actions for determining the feasibility of gathering the desired metrics
- Update documentation. (Dave)
- Land all patches currently in review. (Dave)
- Fix failing test. (Anthony)
- Investigate next steps for Flash content tests (bug 635310). (Anthony)
- Modify existing 'open new tab' tests to cleanup between iterations. (Dave)
- Implement a 'max memory' view to expose testruns consuming large amounts of memory. (Dave)
- Link from Mozmill documentation to Endurance Tests documentation. (Dave)
- Schedule endurance test involvement for Mozmill testday on June 24th. (Dave/Henrik)
- Add to backlog: Allow users to specify what metrics to gather
- Add to backlog: Gather all available memory metrics from about:memory
- Investigate if we can gather all details from about:support in the same way that we currently gather metrics from about:memory. (Dave)
- [UNASSIGNED] Investigate diagnostic interfaces to all systems.
- What can we gather?
- How can we sync this data up?
- Investigate extra resources for endurance tests. (Matt)
- Investigate caching results to disk during testrun to reduce any impact on memory metrics (bug 634580). (Dave)