QA/TDAI/2009 Q4 Sprints

From MozillaWiki
< QA‎ | TDAI
Jump to: navigation, search

We have three sprints in Q4 as you can easily see from https://wiki.mozilla.org/QA/TDAI/Goals/2009-Q4.

Sprint 1 Review: October 23

  • Mobile Test Framework
    • Quick Demo
    • Where we Are
      • Basic DeviceAgent with 90% functionality implemented
      • Working scripts for mochitest, xpcshell, and talos to utilize DeviceAgent
    • Next Steps
      • Heartbeat and DeviceInfo from Device->TestAgent
      • Integrate harness scripts into TestAgent, including download/install, verify, mochitest, xpcshell
      • Upload logs, status, and updates to TestServer
  • Valgrind Test Automation
    • (no demo -- too slow!)
    • Where we are
      • We have support for running valgrind with jsreftests, crashtests, reftests, xpcshell tests, and mochitests. All relevant patches have landed: bug 521130, bug 521457.
    • Next Steps
      • Collect valgrind exclusion list, for excluding certain uninteresting errors.
      • Investigate CPU/RAM requirements for VM's.
      • Investigate use of valgrind 3.2.1 (ships with CentOS 5) vs 3.5.
      • Investigate log parsing: XML output, error summaries.
  • Automatic Crash Reporting
    • Demo?
      • No demo. Looks like top site testing anyway.
    • Where we are
      • bug 411930 - crash reporter sends wrong url has some traction now. Hopefully we will be able to improve reproducibility when that lands.
      • Testing Sisyphus installation build infrastructure on qm-crash-xp01 today. Will also finish qm-crash-linux01 Sisyphus installation today (Oct 23).
    • Next Steps
      • How close are we to turning it on?
        • Should have qm-crash-xp01, qm-crash-linux01 running automated crash testing before Nov 9.
  • Electrolysis Testing
    • (no demo -- looks just like regular mochitest!)
    • Where we are
      • OOP Plugin Tests
        • Have a mochitest-ipcplugins make target for running OOP plugin tests: bug 519574. This is being tested for inclusion onto e10s tinderbox: bug 523712.
        • Nearly done with npruntime tests, still need tests for NPN_Construct and NPN_Evaluate - next week?
    • Next Steps
      • OOP Plugins Tests
        • Finish tests for NPN_Construct and NPN_Evaluate.
        • Dev moves OOP plugins onto moz-central, preffed off (bug 523094) after basic test framework changes are implemented (see below).
      • OOP Framework Changes
        • Mochitest and xpcshell test runners need to be made aware of multiple processes - in progress: bug 523208, bug 523211.
        • Need to figure out how to deal with hangs in child processes, maybe bug 501034 helps.
        • Crash detection probably works OK for multiple processes in the context of the python test runners, but need to test.
        • Need to figure out how automatic crash reporting is impacted by multiple processes (longer-term task).
        • Leak reporting in the context of the python test runners needs to be updated: bug 519570
      • OOP Talos - owner unclear
        • New plugin tests need to be implemented for plugin-related metrics: bug 519568.
        • Talos needs to be updated to expect/handle multiple processes.
      • OOP Content Tests
        • Need to attempt running all unittests with OOP content enabled, and identify problems - may be blocked pending additional necko and docshell work.
        • Fix mochitests which fail under OOP content because they use elevated privileges in the "wrong" place.
  • XBL 2
    • Demo
    • Where we are
    • Next Steps (beyond Q4)
  • Jetpack Test Harness
    • Demo
    • Where we are
    • Next Steps
  • Log Compare
    • (no demo)
    • Where we are
      • Fennec?
      • Firefox?
    • Next Steps
      • Fennec
      • Firefox
  • Adobe Test Harness Automation
    • (no demo)
    • Where we are
      • Waiting on Adobe for links/access
    • Next Steps
      • Jgriffin will own getting it running and automated if we ever hear back from Adobe
  • HTML 5 W3C Tests
    • (no demo)
    • Where We Are
      • Someone from Microsoft has been appointed to the position of Task Force Coordinator
      • Waiting on W3C to start doing stuff
    • Next Steps

Things Needing Attention

  • Gfx test harness
    • URGENT:: Need: Mechanism to measure "choppiness" of video for full screen tests
    • Need: Mechanism to test hardware accelerated 3d via extension
    • Need: Research into types of tests to write
    • Need: Research into WebGL implementation
  • E10S Talos integration
    • Can we get help from RelEng?
    • Does Jmaher have to own?
  • JS Ref Test
    • Browser Tests: Can we get this on Tinderbox quicker? bug 519213 - I believe after bug 487689 Split Mochitest lands we will be able to get more traction. Catlee has responded in the bug today (Oct 23) about which build types to run on. I believe this will be available on Tracemonkey soon.
    • Shell Tests: Can we get these completed quicker? bug 509629 Waiting on dmandelin to finish working on crashkill top crashers. Dmandelin responded in the bug yesterday (Oct 22) that he is actively working on this.

Sprint 2 Review: November 20

Sprint 3 Review: December 18

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/TDAI/2009_Q4_Sprints&oldid=177965"