Auto-tools/B2GAutomation/Meetings/2014-04-17

From MozillaWiki
Jump to: navigation, search

Round Table

  • Adding Mulet to TBPL - bug 997096. We'll file a bug for releng to stand up Mulet builds in TBPL, then work on getting tests running side-by-side with B2G desktop builds. We won't schedule mochitest-chrome or mochitest-browser-chrome. We should make this available on try at the same time as cedar/elm.
  • Enabling OOP in B2G desktop builds - bug 914584. Some tests fail when this is done; we'll run some suites side-by-side non-OOP runs
  • Ahal is making good progress on the MarionetteJS "common harness"; his ETA is approximately the end of May. This will allow us to run MarionetteJS tests on emulators and devices.
  • Gaia build-test-integration running on cedar but failing due to reporting problems - bug 997491 [owner: yuren]
  • Need to stand up Gaia linter tests on cedar as well
  • Ahal has landed better crash detection for mochitest on elm; this has exposed a frequent shutdown crash which Milan is looking into

Tarako automation support

Issues

  • make test-perf is not reporting data to datazilla: bug 987383 ; 'make' is crashing on the Jenkins slaves. [owner: hub] (update: testing a fix from hub now)
  • gaia-integration tests in TBPL don't conform to sheriff visibility guidelines: bug 986206 [owner: gaye] Re-opened, looks like Gareth is going to have to fork mocha to fix this.
  • Both regular and eng Tarako builds are actually userdebug; shall we turn one off? https://bugzilla.mozilla.org/show_bug.cgi?id=994985
    • I would say "yes," although shouldn't we be doing perf tests against a non-debug engineering build? - stephend (I commented in the bug)
  • Mochitests running on emulators on AWS VM's can become CPU-starved; this affects (at least) media and WebRTC tests. We plan to start running media mochitests as a separate chunk on hardware slaves instead of VMs: bug 994920

Completed

  • A new mailing list has been created, fxos-perf-alerts@mozilla.com, which is being used to deliver alerts for B2G performance testing: both "ingestion rate" alerts (alerts that are triggered when no data is added to Datazilla for a given device/branch over a certain period) and regression alerts. Please subscribe if you'd like to see these!
  • FxOS Branding Suite v 1.3.0.3 released: bug 970615
  • All testing has been moved off of the old r3 mac mini slaves, which allows them to be decommissioned (joint with releng): bug 818968

In progress

  • Get ssltunnel working with B2G mochitests: bug 987406
  • Create a common harness to run Python and NodeJS Marionette tests on devices and emulators: bug 990635
  • Add gaia-build tests to TBPL: bug 989126
  • Get Gaia try jobs to run with a pre-existing build (joint with releng): bug 986209
  • FirefoxOS Branding Certification Suite: bug 970615
  • B2G AreWeSlimYet tests: bug 899194 ; Waiting for AWSY dashboard to be updated for B2G data.
  • Adding gaia-ui-tests on emulators on TBPL: bug 916368
  • Fixing bugs related to B2G mochitest manifests, see bug 943326 and bug 983662
  • Schedule media mochitests on emulator-jb on trunk branches: bug 889772
  • More mach support for B2G, see bug 980952
  • Run gaia-integration tests on-device: bug 979077