B2G/QA/Meetings/2014-11-26
From MozillaWiki
Contents
- 1 Overview
- 2 Status Highlights
- 3 2.1 Overall blockers
- 4 2.2 Release Update
- 5 Agenda
- 6 This Week - Decisions & Action Items
- 7 Relevant sessions for FirefoxOS QA at Portland Work Week
Overview
- Meeting Name: B2G QA Roundtable
- Meeting Time: Wednesdays @ 5pm - 5:45pm PDT
- Vidyo Room: QA Vidyo Room
- Video Link: https://v.mozilla.com/flex.html?roomdirect.html&key=nL4PESn1IaVh
- Dial in: 1-800-707-25331
- Meeting Notes: https://wiki.mozilla.org/B2G/QA/Meetings
- Testrun Schedule: http://bit.ly/1jQzW79
- Roadmap for releases: https://wiki.mozilla.org/B2G/Roadmap
- B2G dashboard: http://charts.mozilla.org/
- QA Roundtable Template: https://etherpad.mozilla.org/b2g-qa-roundtable-template (Clone this)
Status Highlights
- Bug 1104338 - [b2g] we need root access on the b2g builds ( 2.2 )
- Q4 Goals B2G (Draft)
2.1 Overall blockers
noms = http://mzl.la/1ATgdGM blockers = http://mzl.la/1l1nmh4
2.2 Release Update
Features (Organized by Functional Team)
Overall Concerns
Remaining Tasks
Text Selection work
- Shing Lyu
- Health Status:
- Details: sent out guidelines to follow
Communications
- Contacts
- Tester: Johan Lorenzo
- Health Status: Green
- Details: 1 blocker (bug 1094966), the Facebook redirection changed on the Facebook side.
- Dialer
- Tester: Johan Lorenzo
- Health Status: Green
- Details: No blocker
- Cost control (SmartData)
- Tester: Johan Lorenzo
- Health Status: Green
- Details: No blocker
- SMS
- Tester: Eric
- Health Status: Green
- Details: No blocker
Media
- Tester:
- Health Status:
- Details:
- Non-Ringtone
- Tester: njpark
- Health Status: Green
- Details: 4 blockers. Among them, this one is sort of serious:
Multimedia Platform
- Tester: Edward
- Health Status:
- Details:No new features
Productivity
- Tester: Edward, Teri
- Health Status: Green
- Details:No blockers
Systems Front End
- Tester: marcia/pragmatic
- Health Status: Green
- Details:
- 2.1 Blocker:
- https://bugzil.la/1097941 B2G: Scrolling browser frame in landscape mode is broken
- Features
- https://bugzil.la/1067435 (app-grouping) [meta] Home-screen grouping - has landed, and App Grouping bugs can start being verified. Test cases are in progress.
- 2.1 Blocker:
Systems Platform
- Tester: Gerry
- Health Status:
- Details:
- 2.2 blockers: 2 deferred bugs of 2.1 (not new features)
- https://bugzilla.mozilla.org/show_bug.cgi?id=1098609 - Network selection not coming back up after failed manual selection
- https://bugzilla.mozilla.org/show_bug.cgi?id=1095802 - Piped STK messages results in no Terminal Responses
- 2.2 blockers: 2 deferred bugs of 2.1 (not new features)
Metrics Testing =
- Tester: Shing Lyu
- Health Status: Green
- Details: No blocker
Telephony/Networking
- Tester: eric
- Health Status: Yellow
- Details: unable to call out emergency from sim2 1089566
- NFC Tester: Alison
- Health Status: Green
- Details: No blocker
Devices
- Bluetooth Tester: Alison
- Health Status: Green
- Details: no blockers, need update on features
Graphics
- Tester: njpark
- Health Status: Yellow
- Details:
- https://bugzilla.mozilla.org/show_bug.cgi?id=1094028 (thumbnail)
- https://bugzilla.mozilla.org/show_bug.cgi?id=1068963 (video playback)
- https://bugzilla.mozilla.org/show_bug.cgi?id=1100549 (Rocketbar completely broken)
- 5 2.1 blockers open in graphics, mostly to do with scroll functionality
2.1 Smoketests
- Tester: Peter Bylenga
- Health Status: GREEN (5+ days)
- Details: 0 blockers
- Notes: We verified the full flash blockers
2.2 Smoketests
- Tester: Peter Bylenga
- Health Status: Yellow
- Details: 2 blockers
- https://bugzilla.mozilla.org/show_bug.cgi?id=1101707 (Guilty Commit Backed out)
- https://bugzilla.mozilla.org/show_bug.cgi?id=1103339 (Resolved Fixed)
- Notes: https://bugzilla.mozilla.org/show_bug.cgi?id=1083953 Privacy Panel app landed
RTL
- Tester: delphine
- Health Status:
- Details:
Localization
- Tester: delphine
- Health Status:
- Details:
Performance
- Tester:
- Health Status:
- Details:
MTBF Stability
- Tester:
- Health Status:
- Details:
Exploratory
- Tester: Derek H
- Health Status: Red
- Details: We have stopped exploratory work last Friday, and have been working on verifications for this week. 8 blockers for 2.1+, and 1 blocker for 2.2+
- 2.1+ Blockers
- https://bugzilla.mozilla.org/show_bug.cgi?id=1098654 - [Messages]
- https://bugzilla.mozilla.org/show_bug.cgi?id=1100714 - [Window Management]
- https://bugzilla.mozilla.org/show_bug.cgi?id=1095716 - [Loop]
- https://bugzilla.mozilla.org/show_bug.cgi?id=1097235 - [Loop]
- https://bugzilla.mozilla.org/show_bug.cgi?id=1099411 - [Calendar]
- https://bugzilla.mozilla.org/show_bug.cgi?id=1099445 - [Calendar]
- https://bugzilla.mozilla.org/show_bug.cgi?id=1096520 - [Video]
- https://bugzilla.mozilla.org/show_bug.cgi?id=1099419 - [Lockscreen/Camera]
- 2.2+ Blocker
- 2.1+ Blockers
Agenda
- Portland Work Week: FxOS CoreOS sessions https://docs.google.com/a/mozilla.com/spreadsheets/d/1eP0CXx1OM5TxjkpWGGOZzS6udXFILM1nlK4GihJpBY0/edit#gid=0
- see more relevant sessions at the bottom of this etherpad
- [Shing] For Text Selection, please read https://etherpad.mozilla.org/TextSelectionTest
- Bugzilla: QA whiteboard [textselection], MozTrap: textselection tag
- [jlorenzo] Portland: The Moztrap meeting and the RTL kickoff are at the same time (December 2nd, 3PM PST).
- [delphine] Candice organized that RTL Kickoff meeting
- [delphine] RTL bugs: QA whiteboard [rtl-impact], Moztrap tag: rtl
- Go through https://etherpad.mozilla.org/ArabicRTL-Testing if you still haven't received individual / specific test cases for your feature
- [Tchung] QA whiteboarding [2.2-features-qa+] for 2.2 features in this list: https://mozilla.aha.io/published/7e3fcb6703be666bf40f322ccafb7e30?page=2
- [eric] does the link contain all the features from draft spreadsheet 1? they have the same info?
- [marcia] Please make sure Window Management bugs are not tagged with [systemsfe] in the Status Whiteboard. They are now under System Platform according to Candice. Also please note: If they are sysfe, please make sure the whiteboard tag [systemsfe] is used. We track our blockers here: https://wiki.mozilla.org/FirefoxOS/systemsfe/releasetracking
- [Al Tsai, Shing] MozTrap Tool and Enhancements session at Portland Work Week
- Put your discussion points here:
- [Geo] Please nominate tests from your functional areas for Gaia regression testing automation.
- Picking appropriate tests:
- Prefer features that have settled and aren't changing currently, at least very quickly.
- Prefer tests that are very repeatable, and happen the same way every time.
- Prefer tests that have very clear and unambiguous success criteria.
- Prefer tests without many dependencies on complex test data or external servers
- Prefer tests that are already being run in our manual regression test runs, so we can lessen that effort.
- That said, if nothing you need fits those criteria, nominate what you do need most and we can consider.
- Filing an automation request:
- File a bug in Firefox OS | Gaia::UI Tests saying "Automate test for [...]"
- If test is in MozTrap, point at the MozTrap test case. Not much more is needed.
- If test isn't in MozTrap, you can describe what you want in the bug. You do not need to define the test in MozTrap at this time. The condition that makes the test successful should be spelled out explicitly, the rest can be fairly general as long as it's reasonably obvious.
- Ex: "Automate sanity test for status bar default icons," where description is "Immediately after a restart, the following icons should appear above the home screen...[list of icons]"
- Worst case, we'll needinfo you for clarification, so don't get hung up on the description.
- Tagging for nomination:
- Please add [fxosqa-auto-backlog?] to the QA Whiteboard to get it into our triage.
- Picking appropriate tests:
This Week - Decisions & Action Items
- [Peter, Kevin] make sure Qanalysts continues to own 2.1 regression , qawanted, smoketest work for next week
Relevant sessions for FirefoxOS QA at Portland Work Week
Location: Marriot Waterfront Ballroom D Tuesday, 12/2
- 2-3pm: QA: Strategy and Goals meeting
- Summary: This session will give an overview of the QA strategy for 2015. Please come to discuss and comment on what the roles and expectations will be for the OS and Devices QA team
- Audience: drivers, QA
- 3-4pm: QA: MozTrap Tool and Enhancements
- Summary: this discussion will involve the current usage of moztrap, what can be improved, and what best practices to use, and who can own the undocumented work.
- Audience: OS QA, Device QA, A-team members like Cameron, Jgriffin
- Put your discussion points here:
Wednesday, 12/3
- 11-12am: Gonk team: Power Tool and Auto Power Test by Jon Hylands/Danny Liang
- 1-2pm: QA: Device automation strategy
- Summary: This session will discuss the automation strategy in more details for FxOS testing going forward. It will also include framework and harness discussions, and how we can work with A-team to implement more test coverage.
- Audience: QA, A-team
- 3-4pm: QA: Reference phone Lab setup and testing (+A-team)
- Summary: discussion on how we manage our flames, what base images we tests beforehand, and how we are designing and setting it up.
- Audience: QA, A-team, Lab folks, Stephend
- 3-5pm: Arabic Bug Bash
- Summary: please bring your Flames and help with doing some BIDI (RTL) testing on FxOS
- Audience: QA, Bidi folks, UX
- 2-3pm: QA: MTBF improvements
- Summary: discuss the MTBF stability strategy, how we define it today, and what we need to define and scale tomorrow.
- Audience: MTBF QA, A-team, rwood, jlal
Thursday, 12/4
- 3-4pm: QA WW Session: Defining Bug Verifications strategy
- Summary: discussion on how we manage our flames, what base images we tests beforehand, and how we are designing and setting it up.
- Audience: QA, A-team, Lab folks, Stephend