Firefox OS/DeviceQA/2015-01-26Status
From MozillaWiki
< Firefox OS | DeviceQA
Contents
V2.2 Highlight & Risk
- Highlight
- Bug 1111890 - [Nexus 5] Create Lollipop based pvtbuilds for 2.2
- Bug list Issue
- Bug 1123205 - white screen issue, not in smoke test but a serious issue.
- Bug 1125769 - [Flame][Bluetooth]Music can't be played and stopped remotely via bluetooth headphone
V2.2 Next Milestone
- 2015/02/23 FL
V2.2 Testing Scope
- Focus on bugs in agreed list, prepare new test cases or regression, and perform test runs.
- B2G V2.2 Overview Page for more info.
V2.2 Testing by Type
Smoke Testing
RED
- New Bug Breaking the Smoketests: (Number of bugs: 1)
- Bug 1125769 - [Flame][Bluetooth]Music can't be played and stopped remotely via bluetooth headphone
- Existing Bugs Breaking the Smoketests: Updated (Number of bugs: 1)
- Resolved Fixed case, request to enable this case
- [Disabled] test_lockscreen_unlock_to_camera_with_passcode.py - Bug 1078270 - Camera app dies/crashes after unlocking from lockscreen with passcode, Bug 1122451 is resolved fixed in master, not yet in 2.2 branch.
- Existing Bugs Breaking the Smoketests: (Number of bugs: 4)
- [Disabled] test_setup_and_send_active_sync_email.py - Bug 1028192 - Problems with ActiveSync accounts
- [Disabled] test_receive_active_sync_email.py - Bug 1028192 - Problems with ActiveSync accounts
- [Disabled] test_cost_control_data_alert_mobile.py - Bug 1061698 - Need ability to work in the shadow root/shadow DOM
- [Disabled] test_call_log_all_calls.py - Bug 1096385 - Investigate failure in test_call_log_all_calls.py, call log is displayed erroneously
- Daily Reports & Historical data can be found here.
- Manual - are available on Google Drive & fxosqa-report-watchlist.
- Automation - are available on Github.IO & fxosqa-report-watchlist.
Functional Testing
On Track
- QA focus - 37 bugs found.
- feature-b2g=2.2+ 169 bugs found.
- blocking-b2g=2.2+ 37 bugs found.
- Privacy (Gerry)
- Health: Red
- Test case coverage:
- Details: 3 blockers
- No dedicated & specific owner for this feature.
- https://bugzilla.mozilla.org/show_bug.cgi?id=1111992
- https://bugzilla.mozilla.org/show_bug.cgi?id=1118155
- https://bugzilla.mozilla.org/show_bug.cgi?id=1121232
- Bhavana to help get assignee here from DT
- Cut, Copy, and Paste (Gerry, Edward, Teri, Eric)
- Health: Green
- Test case coverage: 80%
- Details: WIP, bug bash on copy/paste this Friday in Taipei
- NFC (Alison)
- Health: Green
- Test case coverage: 100%
- Details: WIP
- Rocketbar refactor & search suggestion based on provider (Hermes)
- Health: Green
- Test case coverage: 90%
- Details: WIP, (De-scoped: Bug 1053747 Expand/collapse Rocketbar on home screen)
- App-grouping (Hermes)
- Health: Green
- Test case coverage: 95%
- Details: WIP
- Task Manager (Hermes)
- Health: Green
- Test case coverage: 95%
- Details: WIP
- L-porting (Hermes)
- Health: RED
- Details:
- L build still failed, so use build with workaround (Bug 1119980)
- A initial test: Most of components work (e.g.: WIFI, RIL, Camera, Video, and BT) Will perform more tests.
- CAF bugs (Eric)
- Health: Yellow
- Test case coverage: 66%
- Details: bug 1020757 no FDN sim, checked with TEF, they don't have that type of SIM either, I will do regression for this bug; bug 1016807: stk icon, TEF implements this, but they suggest CAF maybe can help.. will check with [Mike lee]
- TCL bugs (Alison, Eric)
- Health: Green
- Test case coverage: 100%
- Details:
- [bugs related to feature <bug 1091544>]
- one blocker: https://bugzilla.mozilla.org/show_bug.cgi?id=1123567
- one nominate: https://bugzilla.mozilla.org/show_bug.cgi?id=1123602
- bug 1096815 - TEF will test this(FDN).
- Bluetooth (Eric, Teri)
- Health: Green
- Test case coverage: 100% Regression
- Details: Some problem with building, but manually change binary component is okay, will wait for stable & official build from pvt.
- Metrics (Shing)
- Health: Yellow
- Test case coverage:50%
- Details: 1 WIP
- follow up with Kevin Hu
- 256M RAM memory optimization for WFVGA, 5MP. (Teri)
- Health: Green
- Test case coverage: Working on it
- Details: Is this the same issue as the cgroup https://bugzilla.mozilla.org/show_bug.cgi?id=1082290 ?
- Mem Optimization for 2.1+ : https://bugzilla.mozilla.org/show_bug.cgi?id=1115631
- Multicore performance (Edward)
- Health: Yellow
- Test case coverage:
- Details:
- So far we have not a build support.
- Study and setup new performance testing environment. https://github.com/mozilla-b2g/raptor (https://www.npmjs.com/package/gaia-raptor)
- Better performance on Multicore (App startup time for core apps is comparable to Android
Exploratory Testing
On Track
- 176 bugs found - includes every bug in bugzilla affecting v2.2, not limited to the bugs after branch day.
- UNCONFIRMED, NEW, ASSIGNED, REOPENED - 121 bugs found.
- UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 17 bugs found.
- RESOLVED, VERIFIED, CLOSED - 69 bugs found.
- RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 2 bugs found.
Performance Testing
- Will get started asap
Stability Testing
- Current running device: Flame-KK
- Will will switch to Nexus-L & Flame-L when the build is available.
- Bug 1111890 - [Nexus 5] Create Lollipop based pvtbuilds for 2.2
- No statistics or report yet, we are trying to get familiar & stabilize the testing framework.
- Will try to run regularly after FL.
V2.2 Test Case Creation
Test Case Creation WIP
On Track
- [2.2-feature-qa+] & in-moztrap? (7 bugs) --> WORK IN PROCESS
Test Case Creation DONE
- [2.2-feature-qa+] & in-moztrap+ (15 bugs) --> WORK DONE