B2G/QA/Meetings/Automation/2015-01-08

Overview

This is a weekly meetup to discuss all things regarding B2G automation and Quality. Please use wiki style.

• Meeting Name: QA Automation Roundtable
• Meeting Time: Thursday @ 9:30am - 10am PST
• Vidyo Room: QA Vidyo Room
• Video Link: https://v.mozilla.com/flex.html?roomdirect.html&key=nL4PESn1IaVh
• Dial in: 1-800-707-25331
• Meeting Notes: https://wiki.mozilla.org/B2G/QA/Meetings
• Previous Meeting:https://wiki.mozilla.org/B2G/QA/Meetings/Automation/2014-12-18

Q1 2015 Goals

2.2 - (OS QA (in Taiwan)

• • Support the stability of 2.2 branch with focused test runs, qawanted, bug triage, and daily smoke tests (manual and automation)
  • Support new feature testing
  • Manage the release out the door until CC date, and hand off to Devices QA team into Q2.

3.0 - (OS QA (in US/EU)

• • Automation goals:
    • Re-evaluate the smoke tests on v3 that will be needed for automation in javascript
    • Focus on getting existing Gij tests running against the lab of devices
    • Creating a set of best practices on creating JS gaia-UI automation tests
    • Focus on getting smoke tests running against the lab of devices (could be a combination of Gij, python, or JS tests).
  • Support the stability of master with focused test runs, qawanted, bug triage, and daily smoke tests (manual and automation)
  • Support lightweight dogfooding program off of nightly
  • Develop feature testplans when features are concrete

Status Highlights

• [Tony] PTO Jan 10+. Geo/Johan to cover
• [Viorela] PTO Jan 8
• [Robert] PTO Jan 15 - 16

Agenda

• [Bebe] Delete the flame-kk-319.mozilla-b2g34_v2_1.ui.functional from MV Jenkins as we are not using them any more. These jobs are ran by the TW team now and they look pretty stable.
• [Bebe] Bug 1117859 - Update base image for flame-kk builds to v18D ---- Should we update our build too? Yes. i'll share out the image today
  • v18D doesn't have ADB enabled by default.
• [jlorenzo] What's the best usage of PageRegions?
  • Everything started with the StatusBar in https://github.com/mozilla-b2g/gaia/pull/27155/files#diff-a766a1806cf0f6f0a4cec551d97e165cR10 (which was a regular App before this PR). To me, it's not a good practice to include the high level locator of a part of a page inside the class which represent it. It's preferable to put it in the main Page (like we do with calendar settings for instance https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/calendar/app.py#L281 )
  • But the problem is every part of the SystemApp is implemented as an App and not a PageRegion.
  • So when do you guys think we should use PageRegions?
    • If someone could explain the usage of this, that would be great
    • PageRegion is defined that way https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/base.py#L134
  • File new bugs on gaia-ui tests to write or add in-qa-testsuite? flag for regression bugs?
    • If the bug already exists (i.e. test for bug #######), you can add the above flag or a fxosqa-auto-backlog? qa whiteboard to get it triaged, and we can attach the test to that bug for review. If the conversation around it gets too complex, we can spin it out to a separate bug.
    • I'd only do this on bug-regression tests, though--new feature tests should probably start separate, not attach to a feature delivery bug.
  • STatus of Mulet builds? https://bugzilla.mozilla.org/show_bug.cgi?id=1110839
    • FYI: https://wiki.mozilla.org/Mulet
  • manifest.ini, I wonder how correct all the variables are (external = true by default, for example, I wonder if there are tests that are accidentally disabled on buildbot because of that), Would like to revisit all the skip-if = device=="desktop" rules
    • We can definitely review these.
  • Endurance tests can be deleted - rwood to review
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1117144
• [Tony] verbal update on Marionette JS work from James's team
  • Under active fix by James, Tony will update with bug numbers
  • However, team is shorthanded on this work, cycle may get extended
• [Tony] Discuss feedback on Test failure doc

This Week - Decisions & Action Items

• [Tony] request v18D-1 with adb enabled access from T2M
• [Tony] MarionetteJS bugs from James