Auto-tools/Meetings/2013-01-14: Difference between revisions

Jump to navigation Jump to search

Auto-tools/Meetings/2013-01-14 (view source)

Revision as of 17:59, 14 January 2013

5,262 bytes removed ,  14 January 2013
no edit summary
(→‎War on Orange: Add edmorley)
No edit summary
Line 1: Line 1:
== Goals Progress ==
== Goals Progress ==


Below are our [https://wiki.mozilla.org/Auto-tools/Goals/2012Q4 top level goals].  Beneath each goal, note any progress made towards that goal this week.
Below are our [https://wiki.mozilla.org/Auto-tools/Goals/2013Q1 top level goals].  Beneath each goal, note any progress made towards that goal this week.
* '''Ensure B2G has all test automation available to prepare for launch'''
 
** {{ok|Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:}}
* <insert goals here>
*** {{done|Mochitest-plain (emulator first, panda second)}}
*** {{skip|Mochitest-chome (emulator first, panda second)}}
*** {{done|Reftest (emulator first, panda second}}
*** {{ok|xpcshell (panda)}}
*** {{done|webAPI (emulator only}}
*** {{ok|webQA - gaia front end tests emulator and panda}}
*** {{ok|gaia dev integration/unit tests emulator and panda}}
** {{ok|Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng}}
** {{ok|Ensure all webAPIs testable through the emulator have comprehensive automated tests}}
** {{ok|Continue to aid QA in creating automated tests for B2G}}
* '''Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.'''
** {{ok|Have APC.io (ARMv6)boards smoketest ready - rooted, working, plugged into Autophone}}
** {{ok|Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations}}
** {{ok|Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing}}
** {{ok|Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.}}
* '''Create and deploy performance automation for desktop, fennec, and B2G product benchmarks'''
** {{ok|Create Eideticker tests for B2G on panda boards and run them once a day}}
** {{ok|Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis}}
** {{ok|Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions}}
* '''Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.'''
** {{ok|Deploy 4.2 version of Bugzilla}}
** {{ok|Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.}}
** {{ok|Complete the 'Sandstone' CSS update}}
** {{ok|Rework browserID <--> BMO integration}}
** {{ok|Deliver user and product dashboards}}
** {{ok|Deploy tracking flags extension database optimization}}
* '''Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem'''
** {{done|Aid with B2G changes so that B2G oranges tracked in Orange factor, B2G tests show up in TBPL}}
*** {{done|Add TBPL support for B2G emulator/Panda/Unagi/Otoro}}
*** {{done|Make TBPL's log parser compatible with mozharness-style output}}
*** {{done|File/track B2G intermittent failures, report issues with mozharness' error detection/reporting & unhide builds/testsuites on TBPL once they pass consistently}}
*** {{done|Add the new B2G18 tree to OrangeFactor}}
** {{done|Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures}}
*** {{done|Add support for more types of failures to TBPL's log parser}}
*** {{done|Improve test harness & buildbot output for failures & make the output more compatible with TBPL's parser}}
*** {{done|Improve TBPL's bug suggestions for failures}}
*** {{done|Make TBPL's tree open/closed status update more frequently}}
*** {{done|Resolve timeouts during TBPL log parsing/summary generation}}
*** {{done|Simplify the 'TBPL job -> recent history for that machine' workflow}}
*** {{done|Obtain BuildVPN access & investigate BuildVPN-only tools and their use to sheriffs}}
*** {{done|Simplify the TBPL -> 'viewing job on the buildbot master' workflow}}
** {{done|Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.}}
** {{ok|Make orange factor update more than once a day to help make it easier to see changes over time}}
*** Patch in progress, see {{bug|797811}}.
** {{risk|Assist with getting Datazilla's UI integrated with TBPL, phase 1}}
*** Awaiting finalised datazilla/talos output format, see: {{bug|816634}} & {{bug|794901}}.
** {{done|Reduce regression windows for tree bustage by switching off unused builds/tests to reduce coalescing}}
** {{done|Reduce TBPL & OrangeFactor load on BzAPI/b.m.o, by updating use of BzAPI & transitioning to keywords rather than whiteboard annotations}}
** {{done|Fix our test-harness detection of crashes on Android}}
** {{done|Make it easier to differentiate open/resolved/test-disabled bugs on OrangeFactor, to improve prioritisation as well as finding potential backport candidates}}
** {{done|Track down intermittent-failures missing from OrangeFactor}}
** {{done|Make the tree-closure hook fail closed if treestatus is down}}
** {{done|Increase the SnR of dev.tree-management regression emails by suppressing small percentage changes & cleaning up the subject lines}}
* '''Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.'''
** {{ok|Re-organize and improve the automated testing documentation at MDN}}
** {{ok|Develop first round of unit tests for WebRTC}}
** {{ok|Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage}}


== Highlights ==
== Highlights ==
Mcote
Confirmed users
1,927

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/499873"

Navigation menu