From MozillaWiki
Jump to: navigation, search

Goals Progress

Below are our top level goals. Beneath each goal, note any progress made towards that goal this week.

  • Ensure B2G has all test automation available to prepare for launch
    • [ON TRACK] Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:
      • [DONE] Mochitest-plain (emulator first, panda second)
      • [SKIPPED] Mochitest-chome (emulator first, panda second)
      • [DONE] Reftest (emulator first, panda second
      • [ON TRACK] xpcshell (panda)
      • [DONE] webAPI (emulator only
      • [ON TRACK] webQA - gaia front end tests emulator and panda
      • [ON TRACK] gaia dev integration/unit tests emulator and panda
    • [ON TRACK] Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng
    • [DONE] Ensure all webAPIs testable through the emulator have comprehensive automated tests
    • [ON TRACK] Continue to aid QA in creating automated tests for B2G
  • Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.
    • [ON TRACK] Have (ARMv6)boards smoketest ready - rooted, working, plugged into Autophone
    • [ON TRACK] Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations
    • [DONE] Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing
    • [ON TRACK] Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.
  • Create and deploy performance automation for desktop, fennec, and B2G product benchmarks
    • [ON TRACK] Create Eideticker tests for B2G on panda boards and run them once a day
    • [ON TRACK] Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis
    • [ON TRACK] Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions
  • Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.
    • [DEFER] Deploy 4.2 version of Bugzilla
      • Deferred due to timing conflicts.
    • [DONE] Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.
    • [DONE] Complete the 'Sandstone' CSS update
      • Needs a few tweaks but it's usable as is.
    • [DEFER] Rework browserID <--> BMO integration
      • Deferred for B2G work.
    • [DONE] Deliver user and product dashboards
    • [DEFER] Deploy tracking flags extension database optimization
      • Deferred due to timing conflicts.
  • Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem
    • [DONE] Aid with B2G changes so that B2G oranges tracked in Orange factor, B2G tests show up in TBPL
      • [DONE] Add TBPL support for B2G emulator/Panda/Unagi/Otoro
      • [DONE] Make TBPL's log parser compatible with mozharness-style output
      • [DONE] File/track B2G intermittent failures, report issues with mozharness' error detection/reporting & unhide builds/testsuites on TBPL once they pass consistently
      • [DONE] Add the new B2G18 tree to OrangeFactor
    • [DONE] Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures
      • [DONE] Add support for more types of failures to TBPL's log parser
      • [DONE] Improve test harness & buildbot output for failures & make the output more compatible with TBPL's parser
      • [DONE] Improve TBPL's bug suggestions for failures
      • [DONE] Make TBPL's tree open/closed status update more frequently
      • [DONE] Resolve timeouts during TBPL log parsing/summary generation
      • [DONE] Simplify the 'TBPL job -> recent history for that machine' workflow
      • [DONE] Obtain BuildVPN access & investigate BuildVPN-only tools and their use to sheriffs
      • [DONE] Simplify the TBPL -> 'viewing job on the buildbot master' workflow
    • [DONE] Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.
    • [ON TRACK] Make orange factor update more than once a day to help make it easier to see changes over time
    • [AT RISK] Assist with getting Datazilla's UI integrated with TBPL, phase 1
    • [DONE] Reduce regression windows for tree bustage by switching off unused builds/tests to reduce coalescing
    • [DONE] Reduce TBPL & OrangeFactor load on BzAPI/b.m.o, by updating use of BzAPI & transitioning to keywords rather than whiteboard annotations
    • [DONE] Fix our test-harness detection of crashes on Android
    • [DONE] Make it easier to differentiate open/resolved/test-disabled bugs on OrangeFactor, to improve prioritisation as well as finding potential backport candidates
    • [DONE] Track down intermittent-failures missing from OrangeFactor
    • [DONE] Make the tree-closure hook fail closed if treestatus is down
    • [DONE] Increase the SnR of dev.tree-management regression emails by suppressing small percentage changes & cleaning up the subject lines
  • Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.
    • [ON TRACK] Re-organize and improve the automated testing documentation at MDN
    • [DONE] Develop first round of unit tests for WebRTC
    • [ON TRACK] Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage


  • [henrik] Mozmill ESX cluster in PHX data center is mostly ready to be finished off for Linux and Windows machines - still have to find a strategy in getting mac minis moved.

In-Depth Project Discussion

Show a few recently added features of MozTrap

The Overview

Progress, notes, problems, and other statuses about current projects. Add any new projects to this template!

AutoPhone/Noah's Ark


  • Converted sisyphus.bughunter from Fedora 16 to Fedora 17
    • Minor issue with building vmxnet3 in VMWare tools. Will investigate patches but this does not affect operations.
    • Minor issue with automatically starting bughunter process on reboot. Investigating.
  • Dumped and cleaned the all crash urls for retesting.
  • Truncated the database and logs to remove all old data.
  • Minor tweaks:
    • On Windows, use Windows process id instead of cygwin process id when killing processes. Improves ability to actually kill stuck processes.
    • Changed log directory layouts to use per minute directories rather than per hour to cut down on number of log files per directory.
    • Removed parenthesized argument declaration lists from crash signatures. This help consolidate the crash signatures over platforms which may or may not have incluced the argument declaration lists in the frame signature.
    • Removed branch specific prefix from ASSERTION location paths. This helps consolidate ASSERTIONs across branches making it clearer on which branches an ASSERTION occurs.
  • Began Retesting previous crash urls.


  • [glob] worked with IT to rename
  • [dkl+glob] lots of upsteam reviews
  • [dkl] significant work on bmo/4.2 dashboards
  • [glob] updates to the swag request form
  • [dkl] updates to the air-mozilla request form
  • [glob+dkl] lots of bugzilla administration work


  • [wlach] B2G integration tests now working on Panda!
  • [wlach] Working on dashboard/test manifest support for eideticker on b2g. Uploading soon

Firebug Testing



Mobile Automation

Mobile Evangelism Site-Scraping Tool





New Graphserver (Datazilla)

Open Web Apps




Signal From Noise




Sheriffing & Trees

  • [edmorley] PTO 20th Dec-2nd Jan.
  • [edmorley] Day-to-day sheriffing of the trees (yey pre-merge madness).
  • [edmorley] Patch for bug 737661.



War on Orange

  • [edmorley] More chasing up of toporanges.

Web QA

Automation Development

  • [rwood] Final two WebAPI tests are under review bug 813774 and bug 813777
  • [rwood] Finished post-review patch for maximum segment SMS test bug 806811
  • [rwood] Some WebTelephony tests are disabled on TBPL as they cause the following WebAPI tests to fail (battery tests), even though the WebTelephony tests were passing; I'll investigate using Cedar

Upcoming Events

Standing weekly meeting related to our projects should go here

Round Table

Put your questions and things to raise with the entire team here

Take notes here and copy and paste them to this section afterwards.


Holidays and Trips

Next week MC


I'm the computer man