MozillaWiki

Auto-tools/Meetings/2013-01-14: Difference between revisions

Page Discussion

Auto-tools/Meetings/2013-01-14 (view source)

Revision as of 18:04, 21 January 2013

1,812 bytes removed ,  21 January 2013
→‎Highlights
(Created page with "== Goals Progress == Below are our [https://wiki.mozilla.org/Auto-tools/Goals/2012Q4 top level goals]. Beneath each goal, note any progress made towards that goal this week. * ...")
 
 
(19 intermediate revisions by 10 users not shown)
Line 1: Line 1:
== Goals Progress ==
== Goals Progress ==


Below are our [https://wiki.mozilla.org/Auto-tools/Goals/2012Q4 top level goals].  Beneath each goal, note any progress made towards that goal this week.
Below are our [https://wiki.mozilla.org/Auto-tools/Goals/2013Q1 top level goals].  Beneath each goal, note any progress made towards that goal this week.
* '''Ensure B2G has all test automation available to prepare for launch'''
 
** {{ok|Deploy B2G Test Automation into TBPL running on Emulator and Panda platforms. Specifically we are turning on:}}
* <insert goals here>
*** {{done|Mochitest-plain (emulator first, panda second)}}
*** {{skip|Mochitest-chome (emulator first, panda second)}}
*** {{done|Reftest (emulator first, panda second}}
*** {{ok|xpcshell (panda)}}
*** {{done|webAPI (emulator only}}
*** {{ok|webQA - gaia front end tests emulator and panda}}
*** {{ok|gaia dev integration/unit tests emulator and panda}}
** {{ok|Maintain B2G dog food update server as well as builds for development and QA while working to hand them off to releng}}
** {{ok|Ensure all webAPIs testable through the emulator have comprehensive automated tests}}
** {{ok|Continue to aid QA in creating automated tests for B2G}}
* '''Expand and stabilize Test Automation on new and existing device platforms for NEON and ARMv6 support.'''
** {{ok|Have APC.io (ARMv6)boards smoketest ready - rooted, working, plugged into Autophone}}
** {{ok|Monitor deployment of large scale android and B2G panda board testing, aid with bug investigations}}
** {{ok|Implement a new pool-based approach for device management in automation to increase stability on all mobile device testing}}
** {{ok|Deploy Autophone (Noah's Ark) mochitest and robocop test abilities.}}
* '''Create and deploy performance automation for desktop, fennec, and B2G product benchmarks'''
** {{ok|Create Eideticker tests for B2G on panda boards and run them once a day}}
** {{ok|Monitor Datazilla webservices and ensure our accuracy in detecting performance regressions on a per-push basis}}
** {{ok|Deploy Datazilla per-push UI to make it easier for developers to dig into the causes of performance regressions}}
* '''Update our bugzilla instance to 4.2 providing many benefits from UI to helping people keep track of bugs/projects from within Bugzilla.'''
** {{ok|Deploy 4.2 version of Bugzilla}}
** {{ok|Ensure that bug migration tools are in place for B2G so that we can share bugs with outside services and partners.}}
** {{ok|Complete the 'Sandstone' CSS update}}
** {{ok|Rework browserID <--> BMO integration}}
** {{ok|Deliver user and product dashboards}}
** {{ok|Deploy tracking flags extension database optimization}}
* '''Make the process of sheriffing easier by updating TBPL and helping make it easier to handle the intermittent test problem'''
** {{done|Aid with B2G changes so that B2G oranges tracked in Orange factor, B2G tests show up in TBPL}}
*** {{done|Add TBPL support for B2G emulator/Panda/Unagi/Otoro}}
*** {{done|Make TBPL's log parser compatible with mozharness-style output}}
*** {{done|File/track B2G intermittent failures, report issues with mozharness' error detection/reporting & unhide builds/testsuites on TBPL once they pass consistently}}
*** {{done|Add the new B2G18 tree to OrangeFactor}}
** {{done|Make it easier to sheriff and diagnose issues through better notification and automatic suggestions of issues for intermittent failures}}
*** {{done|Add support for more types of failures to TBPL's log parser}}
*** {{done|Improve test harness & buildbot output for failures & make the output more compatible with TBPL's parser}}
*** {{done|Improve TBPL's bug suggestions for failures}}
*** {{done|Make TBPL's tree open/closed status update more frequently}}
*** {{done|Resolve timeouts during TBPL log parsing/summary generation}}
*** {{done|Simplify the 'TBPL job -> recent history for that machine' workflow}}
*** {{done|Obtain BuildVPN access & investigate BuildVPN-only tools and their use to sheriffs}}
*** {{done|Simplify the TBPL -> 'viewing job on the buildbot master' workflow}}
** {{done|Split mochitest browser-chrome from mochitest-other to improve parallelism and overall turnaround time.}}
** {{ok|Make orange factor update more than once a day to help make it easier to see changes over time}}
*** Patch in progress, see {{bug|797811}}.
** {{risk|Assist with getting Datazilla's UI integrated with TBPL, phase 1}}
*** Awaiting finalised datazilla/talos output format, see: {{bug|816634}} & {{bug|794901}}.
** {{done|Reduce regression windows for tree bustage by switching off unused builds/tests to reduce coalescing}}
** {{done|Reduce TBPL & OrangeFactor load on BzAPI/b.m.o, by updating use of BzAPI & transitioning to keywords rather than whiteboard annotations}}
** {{done|Fix our test-harness detection of crashes on Android}}
** {{done|Make it easier to differentiate open/resolved/test-disabled bugs on OrangeFactor, to improve prioritisation as well as finding potential backport candidates}}
** {{done|Track down intermittent-failures missing from OrangeFactor}}
** {{done|Make the tree-closure hook fail closed if treestatus is down}}
** {{done|Increase the SnR of dev.tree-management regression emails by suppressing small percentage changes & cleaning up the subject lines}}
* '''Expand and Modernize the QA Automation framework as well as make it easier and more clear how to write automated tests for QA.'''
** {{ok|Re-organize and improve the automated testing documentation at MDN}}
** {{ok|Develop first round of unit tests for WebRTC}}
** {{ok|Ensure all B2G device WebAPIs testable through emulators have comprehensive low level test coverage}}


== Highlights ==
== Highlights ==


''This is a place for people to put announcements or to describe activities that they'd like to highlight.  It's not the responsibility of the MC to find things to put here.''
''This is a place for people to put announcements or to describe activities that they'd like to highlight.  It's not the responsibility of the MC to find things to put here.''
* Fennec x86
** android-x86
*** Installed android-x86 vms into ESXi, VMWare Fusion, VirtualBox. Android 2.2, 2.3 eeepc are supported. Later versions do not have ethernet support. Unfortunately can not run fennec x86 due to ABORT: OpenGL-accelerated layers are a hard requirement on this platform. Cannot continue without support for them. See: {{bug|771774}}, {{bug|778175}}. Are the performance and battery issues enough to continue to hard block non OpenGL devices? Being able to deploy android-x86 virtual machines for non-performance related testing may be a big enough win to allow non OpenGL devices.
** emulator-x86
*** Appear to be able to run fennec on emulator x86 on OSX with HAX installed. Still investigating.
* New intern has joined!
** please welcome Yiming to the team. She'll be with us for 4 months, working on Marionette
* [Henrik] We finally turned the knob and switched over to the new CI system for Windows and Linux (this includes a speed-up from 2h to 20min for ondemand update tests!!!)


== In-Depth Project Discussion ==
== In-Depth Project Discussion ==
Line 75: Line 28:


=== [[Auto-tools/Projects/AutoPhone|AutoPhone/Noah's Ark]] ===
=== [[Auto-tools/Projects/AutoPhone|AutoPhone/Noah's Ark]] ===
* Temporarily began running unit tests on phones in haxxor. Need to develop reduced test manifests.
* Currently rerunning s1s2 tests.


=== [[Auto-tools/Projects/Bughunter|Bughunter]] ===
=== [[Auto-tools/Projects/Bughunter|Bughunter]] ===
Line 86: Line 41:


=== [[Project_Eideticker|Eideticker]] ===
=== [[Project_Eideticker|Eideticker]] ===
* [wlach] - Landed faster/better dashboard, new test manifest support (in preparation for B2G stuff)
* [wlach] - B2G dashboard results *almost* there
* [wlach] - Much of the data generated in December / early Jan was wrong. Trying to regenerate it.


=== [https://github.com/mozilla/Firebug-Test-Runner Firebug Testing] ===
=== [https://github.com/mozilla/Firebug-Test-Runner Firebug Testing] ===
Line 92: Line 50:


=== [[Auto-tools/Projects/Marionette|Marionette]] ===
=== [[Auto-tools/Projects/Marionette|Marionette]] ===
* [mdas] had to change how we send touch events in the abstraction layer for b2g, due to https://bugzilla.mozilla.org/show_bug.cgi?id=823619. Dealing with fallout now.


=== Mobile Automation ===
=== Mobile Automation ===
* [ahal] Larger set of reftests now running on B2G emulator arm ({{bug|811779}})


=== [[Auto-tools/Projects/Mobile_Evangelism_Site_Scraper|Mobile Evangelism Site-Scraping Tool]] ===
=== [[Auto-tools/Projects/Mobile_Evangelism_Site_Scraper|Mobile Evangelism Site-Scraping Tool]] ===
Line 100: Line 60:


=== [[Auto-tools/Projects/MozBase|MozBase]] ===
=== [[Auto-tools/Projects/MozBase|MozBase]] ===
* [mcote] Various mozdevice fixes: made sut_tests pass again, improved reboot-callback handling
* [jhammel] Various minor cleanup fixes; looking into getting as much of m-c on mozbase as possible soon


=== [[Auto-tools/Projects/Mozmill|Mozmill]] ===
=== [[Auto-tools/Projects/Mozmill|Mozmill]] ===


=== [[Auto-tools/Projects/MozTrap|MozTrap]] ===
=== [[Auto-tools/Projects/MozTrap|MozTrap]] ===
* [camd] Wrapping up final changes for this dev push.
* [camd] I will be moving away from dedicated dev on MozTrap to work on other projects.  New requests will be triaged on a case by case basis after 1/17.
* [camd] wrapping up scalability fix for multiselect for tags, suites and runs.
* [camd] Will release 1.4.0 this week


=== [[Auto-tools/Projects/Datazilla|New Graphserver (Datazilla)]] ===
=== [[Auto-tools/Projects/Datazilla|New Graphserver (Datazilla)]] ===
* [jeads] Updated https://datazilla.readthedocs.org
* [jeads] Built the in application help
* [jeads] Fixed cross browser UI layout bug
* [jeads] Adding feature to display the comparison revision for a given push to help validate the data.


=== [[Auto-tools/Projects/Open_Web_Apps_Testing|Open Web Apps]] ===
=== [[Auto-tools/Projects/Open_Web_Apps_Testing|Open Web Apps]] ===
Line 124: Line 94:


=== Sheriffing & Trees ===
=== Sheriffing & Trees ===
* [edmorley] Day-to-day sheriffing of the trees.
* [edmorley] Notifying top users of Try about TryServer best practices ({{bug|829932}})
* [edmorley] TryChooser improvements in production: https://bugzil.la/823711,823907,823909,827886
* [edmorley] Reviewed {{bug|813132}}.
* [edmorley] Patches:
** Trychooser: https://bugzil.la/827886
** Mozcrash/mozlog: https://bugzil.la/828324,829332,829773,829941
** Talos: https://bugzil.la/829334,829367
** Misc: https://bugzil.la/828946,782633


=== [https://treestatus.mozilla.org/ Treestatus] ===
=== [https://treestatus.mozilla.org/ Treestatus] ===


=== [https://tbpl.mozilla.org/ Tinderboxpushlog] ===
=== [https://tbpl.mozilla.org/ Tinderboxpushlog] ===
* [edmorley] New things in production: https://bugzil.la/828272,829092,826671,826028
* [edmorley] Patches:
** Parser: https://bugzil.la/829092
** UI/UX: https://bugzil.la/828272,764883,829935,829950


=== [[Auto-tools/Projects/WarOnOrange|War on Orange]] ===
=== [[Auto-tools/Projects/WarOnOrange|War on Orange]] ===
* [edmorley] Patch to suppress the weekly War on Orange email to dev.tree-management if the OF is 0.0 ({{bug|827856}})
* [edmorley] Ongoing backouts of newly introduced flaky tests & disabling of chronic top-oranges.


=== Web QA ===
=== Web QA ===
Line 148: Line 133:


=== Holidays and Trips ===
=== Holidays and Trips ===
* mcote out from Jan 21 - Feb 4 inclusive.
* jmaher out the week of Jan 28 - Feb 4 (with some internets during the time, email only)


=== Next week MC ===
=== Next week MC ===
Whimboo
canmove, Confirmed users, Bureaucrats and Sysops emeriti
4,747

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/499806...503353"