QA/TDAI/WorkWeek Planning

HTML 5 Test Suite

• we have tests
• we have mochitest framework (created by dev for parser error)
  • can leverage that for more testing
• c++ & java DOM comparison testing (needed) make use of dev tool for that
• tokenizer, serializer, test development
• real-world site compare (mikeal)

By August 14

• c++ & java DOM comparison done
• tokenizer, serializer

By Sept 11

• real-world site compare (working with mikeal)
• DONE, maybe document.write stuff

Gfx Hardware Acceleration

By August 14

• harness around reftest ext
• get resutls, might need to get a program together that will grab all the information for inclusion when we send the data back up to the results server
• first pass canvas 2d

By Sept 11

• finish harness - work with jonathan
• more canvas 2d
• canvas 3d (bundle canvas 3d ext with it)
• punt on results reporting

Multi Process

By August 14

• have a build, keep an eye on the project

By Sept 11

• understand how to get crash reporting data
• begin understanding how to help test this, xpcshell multiprocess, etc
• how do we have to retrofit the crashtest so that it wills still detect crashes.
• see about qae tools (if they will be needed, what will be needed etc)
• new attend platform meeting next week see what we can do.

JS Reftests

By August 14

• Browser test completed - patch under review
  • DONE Reftests Done

By Sept 11

• Shell test completed
• Integrated with buildbot - all they need is the python line and they can integrate it into the buildbot slaves uch that they can integrate it into the parallel setup. (in other words, the build team will own this)
• This will be run on tracemonkey and m-c, then 1.9.2 and then 1.9.1 (in terms of decreasing priority

Fennec Test Results (Log Compare)

• Discuss data format for results

By August 14

• Scripts running and uploading to server
  • corner cases/fx & build to get buy in on log parsing
• DONE Version 1 of Web site to view the results will be ready
• Aki ready to plug them into his buildbot slaves

By Sept 11

• Scripts parsing correctly
• Scripts integrated into buildbot slaves
• Addressing Feedback, better views possibly or data mining

Mobile TUnit Integration

By August 14

• DONE Patches for test harnesses on WinMo for Reftest and XPCShell
• Beginning of patch for 1 by 1 Mochitest
• Manual testing as required
• pushed Settle on the requirements for beta and release builds

By Sept 11

• Plan of action to get faster cycling test boxes (possibly using fennec desktop), do we do tracemonkey, unit tests for mobile 1.9.2, m-c, how often?
• Results generated for end to end unit tests with patches for winmo and wince
• Maemo Testing (with qae as well) for 1.0 release

Automatic Crash Repro

By August 14

• Currently need help getting the crash URLs minimized/tested/bugged (weekly, ongoing)
  • folks will need builds, !exploitable, crashwrangler, lithium
  • can we get security folks to help us with it?
  • most important thing is to get the bug filed with a version of the saved page and then invite folks to minimize
  • Bob will email the crash URLs that are repro'd

• work out the procedure for dealing the URL duplicates
• web app interface for jobs
• pool mgmt on web app
• adding the proper interfaces to sisyphus for communication with the web app

By Sept 11

• getting good lists from socorro daily now 3.0, 3.5, 3.6
• what to do with ubuntu
• prioritize and implement the items from "phase 2" (see wiki), probably not all of them, but the highest priority ones first.
• clean up web app

Test Case Mgr

By August 14

• DONE Test case repository
  • First version of repo to store and retrieve testcases
  • Data model for the test cases
  • web interface for running them (maybe)
  • internationalization for test cases

By Sept 11

• • DONE simple test case writer UI
• tags, test runner, collections
• have discussion about porting litmus meta data over
• other workflow issues

XBL 2

By August 14

• Plan in place for changes to frameworks
• Plan for sample tests for development review
• Plan to address emulation mode (if one is implemented)

By Sept 11

• FOLLOW UP AT PLATFORM WORKWEEK
• Begin Implementing Plan (share into Q4)

Compositor

• In patches, not reviewed, passing existing tests
• May not need any framework help

By August 14

• plugin tests
• iframes
• scrolling tests

By Sept 11

Follow up with QAE about the tests that martijn and I found

JS Animation

By August 14

• DONE (for SMIL) use reftest to do the simple tests
• skipped use mochitest for frame by frame comparison
• pushed first pass of tests for css transitions and SMIL individually

By Sept 11

• SMIL tests, testing animating other properties than length
• Transition tests
• tests for CSS transitions and SMIL used together (for instance bug 504652)
• FOLLOW UP AT PLATFORM WORK WEEK JS Animation API

QAC

• Bugzilla tab is done
• design lunch next week
• Litmus tab this week
• Aaron's patch for login stuff
• end of the July --> DONE

By August 14

n/a

By Sept 11

• Stats tab, bug fixes
• Port to Fennec
  • Bug reporting
  • (Possibly) Take/Upload screen shots
  • (possibly) Collect meta data from the device (memory/uses etc)

Mozmill Integration

By August 14

• pushed Buildbot integration finished

By Sept 11

• Issues with windows
• Do mac and linux first with buildbot, do it soon
• Web interface for mozmill results to see the results up there.
• Adding interface to brasstacks for reporting information to hook to mozmill
• Adam will be helping with brasstacks and tcm

Intermittent Orange

By Sept 11

• jquery tests
• geolocation
• follow up with assignments made at the all hands
• results server stuff shoudl eb able to help track this data like ted's top fails report

Extension Performance Testing

By Sept 11

• Report for a set of extensions run through a fixed page set
• Give them a list of load times
• Reuse talos infrastructure locally if possible