This is a weekly meetup to discuss all things regarding B2G automation and Quality. Please use wiki style.
- Meeting Name: QA Automation Roundtable
- Meeting Time: Thursday @ 9:30am - 10am PST
- Vidyo Room: QA Vidyo Room
- Video Link: https://v.mozilla.com/flex.html?roomdirect.html&key=nL4PESn1IaVh
- Dial in: 1-800-707-25331
- Meeting Notes: https://wiki.mozilla.org/B2G/QA/Meetings
- Previous Meeting:https://wiki.mozilla.org/B2G/QA/Meetings/Automation/2014-12-18
Q1 2015 Goals
2.2 - (OS QA (in Taiwan)
- Support the stability of 2.2 branch with focused test runs, qawanted, bug triage, and daily smoke tests (manual and automation)
- Support new feature testing
- Manage the release out the door until CC date, and hand off to Devices QA team into Q2.
3.0 - (OS QA (in US/EU)
- Automation goals:
- Focus on getting existing Gij tests running against the lab of devices
- Creating a set of best practices on creating JS gaia-UI automation tests
- Focus on getting smoke tests running against the lab of devices (could be a combination of Gij, python, or JS tests).
- Support the stability of master with focused test runs, qawanted, bug triage, and daily smoke tests (manual and automation)
- Support lightweight dogfooding program off of nightly
- Develop feature testplans when features are concrete
- Automation goals:
Previous Action Items
- [Tony] PTO Jan 10+. Geo/Johan to cover
- [Viorela] PTO Jan 8
- [Robert] PTO Jan 15 - 16
- [Bebe] Delete the flame-kk-319.mozilla-b2g34_v2_1.ui.functional from MV Jenkins as we are not using them any more. These jobs are ran by the TW team now and they look pretty stable.
- [Bebe] Bug 1117859 - Update base image for flame-kk builds to v18D ---- Should we update our build too? Yes. i'll share out the image today
- v18D doesn't have ADB enabled by default.
- [jlorenzo] What's the best usage of PageRegions?
- Everything started with the StatusBar in https://github.com/mozilla-b2g/gaia/pull/27155/files#diff-a766a1806cf0f6f0a4cec551d97e165cR10 (which was a regular App before this PR). To me, it's not a good practice to include the high level locator of a part of a page inside the class which represent it. It's preferable to put it in the main Page (like we do with calendar settings for instance https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/calendar/app.py#L281 )
- But the problem is every part of the SystemApp is implemented as an App and not a PageRegion.
- So when do you guys think we should use PageRegions?
- If someone could explain the usage of this, that would be great
- PageRegion is defined that way https://github.com/mozilla-b2g/gaia/blob/master/tests/python/gaia-ui-tests/gaiatest/apps/base.py#L134
- File new bugs on gaia-ui tests to write or add in-qa-testsuite? flag for regression bugs?
- If the bug already exists (i.e. test for bug #######), you can add the above flag or a fxosqa-auto-backlog? qa whiteboard to get it triaged, and we can attach the test to that bug for review. If the conversation around it gets too complex, we can spin it out to a separate bug.
- I'd only do this on bug-regression tests, though--new feature tests should probably start separate, not attach to a feature delivery bug.
- STatus of Mulet builds? https://bugzilla.mozilla.org/show_bug.cgi?id=1110839
- manifest.ini, I wonder how correct all the variables are (external = true by default, for example, I wonder if there are tests that are accidentally disabled on buildbot because of that), Would like to revisit all the skip-if = device=="desktop" rules
- We can definitely review these.
- Endurance tests can be deleted - rwood to review
- [Tony] verbal update on Marionette JS work from James's team
- Under active fix by James, Tony will update with bug numbers
- However, team is shorthanded on this work, cycle may get extended
- [Tony] Discuss feedback on Test failure doc
This Week - Decisions & Action Items
- [Tony] request v18D-1 with adb enabled access from T2M
- [Tony] MarionetteJS bugs from James