This is a weekly meetup to discuss all things regarding B2G automation and Quality. Please use wiki style.
- Meeting Name: QA Automation Roundtable
- Meeting Time: Thursday @ 9:30am - 10am PST
- Vidyo Room: QA Vidyo Room
- Video Link: https://v.mozilla.com/flex.html?roomdirect.html&key=nL4PESn1IaVh
- Dial in: 1-800-707-25331
- Meeting Notes: https://wiki.mozilla.org/B2G/QA/Meetings
- Previous Meeting:https://wiki.mozilla.org/B2G/QA/Meetings/Automation/2014-10-23
Previous Action Items
- [Tony] look into access for SV into jenkins nodes
- [DONE] [Bebe] file a bug
[Tony] flash v188 on devices and enable adb [DONE] [Robert] send tony a list of forced nodes that will be used in automation tomorrow (send 10-15 devices)
- Please use v188-1 instead (userDebug image)
- [Tony] talk to releng and do some side testing on v188 with flashing devices
- Bugs found from v188 with full flashing master on it:
- [Bebe] Move some devices on V180 and test builds with V180 base build + full flash m-c + Branches Run tests manually too + add comment in the Bug^ to announce the way we are flashing and the results of V180 tests
[DONE] [Peter] follow up offline (w/ geo/bebe) on discussion about pointing testcases to the google doc [CARRY OVER][Johan] Follow up the plivo account with walter chen and paul and get some number. Follow up with bsilverberg too about if a minute can be split or not. Ask Bob about the test script that puts money into the account?
- Dates (do not share)
- Dev Start (2.2 sprint 1): 2014-11-24 (Gecko 37 will be on m-c and gaia master)
- Total 6 sprints: Last sprint is reserved for stability testing on master
- Feature Landing Readiness: 2015-02-09
- Feature Landing : 2015-02-23 (Gecko 38, base for FxOS 2.2 will be on mozilla-aurora and gaia v2.2 branch created)
- Feature Complete : 2015-04-06 (Gecko 38 moves to mozilla-beta and mozilla-b2g38_v2_2 is created)
- Code Complete : 2015-05-18
- Propose 11/30 to be the last week of supporting 2.1 smoketests (both manual and automation)
- Testcase update (Geo):
- Q4 Goals B2G (Draft)
- Complete 2.1 QA criteria: https://wiki.mozilla.org/Release_Management/FirefoxOS/Release_Milestones#Code_Complete_Criteria
- Identify 3-5 active contributors that has completed One and Done tasks
- Identify 2.2 features and have testplans drafted by end of sprint 2
- Automate 20 gaia UI tests from backlog. have them running on device [geo: add "if appropriate"?], and checked into the main repo. and reporting against Tree Herder
- [geo] Running on device (for MarionetteJS) and reporting to Treeherder are both harness features and belong to FxOS Auto, delivery of those is out of scope for QA. We should stick to "automate 20 gaia UI tests," implication being against whatever harness is currently supported and reasonably stable. I'd suggest leaving out "from backlog"--that's an implementation detail.
- Clean up 2.1 invalidated testcases and prepare for 2.2 testsuite
- performance goal for acceptance testing
- Q4 Goals B2G (Draft)
This Week - Decisions & Action Items
[Johan] Follow up the plivo account with walter chen and paul and get some number. Follow up with bsilverberg too about if a minute can be split or not. Ask Bob about the test script that puts money into the account?
- [jlorenzo] Email thread started. No incoming call during MTBF. They need a service like plivo. They have 30 devices max. They don't know how many tests they run per hour or day.
- [jlorenzo] I have access to the plivo account. 1 second is charged as 1 minute.
- [jlorenzo] Neither bsilverberg nor stephend know about script of that kind.
Next week agenda
[jlorenzo] any news from the project Mozpool? Bug 1020366 - Adapt Mozpool to B2G