QA/Mozmill Test Automation/Endurance Tests/Meetings/05-17-2011

From MozillaWiki
Jump to: navigation, search

« previous mtg Endurance Tests Home | next mtg »

Previous Action Items

  • [DONE] Make all existing Endurance tests local. (Dave)
  • [ON TRACK] Reach out to JS team about gathering additional metrics. (Dave)
  • [ON TRACK] Reach out to JGriffin about collecting information from about:support. (Dave)
  • [ON TRACK] Reach out to Metrics team about chart improvements & optimization. (Dave)
  • [DONE] Determine phase II for endurance tests project and QA goals. (Dave)

Current Work

  • Fixes for running endurance tests on Nightly (bug 656869) - Patches in review.
  • Restart between tests by default (bug 656393, bug 656388, bug 656404) - Patches in review.
  • Pre-calculate statistics (bug 644654, bug 650382) - Patches in review.
  • Show standard test results in report (bug 657347) - Patch in review
  • [ASSIGNED] bug 635310 - Mozmill Endurance test for flash video performance - ashughes
    • Currently reaching out to Benoit Girard and Chris Jones to get more detailed information about Flash rendering
    • Prelim: video, ads, and games each use different rendering methods -- will likely mean 3 separate tests

Failing Tests

  • [ASSIGNED] Switching tabs in Panorama (bug 657368) - ashughes

Discussion Items / Updates

  • Resources for new tests/failures/development
  • 'Micro-iterations'
  • More potential changes coming up in Firefox 6.0 (bug 657327)
  • Brainstorm
    • Identify desirable static and dynamic metrics
    • Create actions for determining the feasibility of gathering the desired metrics

Meeting Notes

Action Items

  • Update documentation. (Dave)
  • Land all patches currently in review. (Dave)
  • Fix failing test. (Anthony)
  • Investigate next steps for Flash content tests (bug 635310). (Anthony)
  • Modify existing 'open new tab' tests to cleanup between iterations. (Dave)
  • Implement a 'max memory' view to expose testruns consuming large amounts of memory. (Dave)
  • Link from Mozmill documentation to Endurance Tests documentation. (Dave)
  • Schedule endurance test involvement for Mozmill testday on June 24th. (Dave/Henrik)
  • Add to backlog: Allow users to specify what metrics to gather
  • Add to backlog: Gather all available memory metrics from about:memory
  • Investigate if we can gather all details from about:support in the same way that we currently gather metrics from about:memory. (Dave)
  • [UNASSIGNED] Investigate diagnostic interfaces to all systems.
    • What can we gather?
    • How can we sync this data up?
  • Investigate extra resources for endurance tests. (Matt)
  • Investigate caching results to disk during testrun to reduce any impact on memory metrics (bug 634580). (Dave)