B2G/QA/2.2/2015-06-15Status
From MozillaWiki
Contents
V2.2 Highlight & Risk
- Highlight
- NEXUS-5-L 5.1 PVT build MTBF 104.81 hrs
- FLAME-KK 4.4 PVT build MTBF 21 hrs
- 4 bugs with blocking=2.2+ http://mzl.la/1CDEaqi
- 17 bugs with blocking=2.2? http://mzl.la/1CDEKV4
- 5 bugs with CAF-v2.2-metabug http://mzl.la/1ELeOXg
- FLAME-L 3.0 local build is ready, bugs here http://mzl.la/1KPpOoc.
V2.2 Milestone
- 2015/06/08 V2.2 CC
- 2015/04/29 V2.2 Updated FC
- 2015/02/23 V2.2 FL
V2.2 Testing Scope
- Focus on bugs in agreed list.
- B2G V2.2 Overview Page for more info.
- Status NEW: blocking-b2g=2.2+ 4 bugs
No results.
0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);
V2.2 Testing by Type
Smoke Testing
Nexus-5-L 5.1 Green
- Testing on PVT build from 2015/04/14
- Manual test results - are available on Google Drive
- Green for 3+ weeks
- Known issues
- No FM radio support, no DSDS support and no UMS support.
- OTA not working, but the update channel is fine for 2.2->2.2.
Flame-KK Green
- Manual test results - are available on Google Drive
- Automation reports - are available on Github.IO & fxosqa-report-watchlist.
- Green for 3+ weeks
- Known issues
Regression Testing
3rd Run (2015/05/19)
- Exploratory tests
2nd Run (2015/04/14)
- NEXUS-5-L 2nd Run (focus on essential functionality) Done
- Test cases http://mzl.la/1D76gXG
- Result
- 1119(total), 1090(passed), 8(failed), 21(invalid)
- 0.007% fail rate
- bugs
- Total 5
- http://mzl.la/1cYZZau (Bugs related to software home button, privacy or ui polish will be skipped for now)
- FLAME-KK 2nd Run (focus on dsds and gaia apps) Done
- Test cases http://mzl.la/1aKYIC8
- Result
- 342(total), 334(passed), 5(failed), 3(invalid)
- 0.01% fail rate
- bugs
- Total 5
- http://mzl.la/1GifEOz
1st Run (2015/03/03)
- RTL/V2.2 Flame-KK Done
- Moztrap result - 76 fail, 326 pass, check with Josh for detail.
- Full Regression/V2.2 Flame-KK Done
- Moztrap result - 55 fail, 3426 pass, 100% complete
- Partial Regression/V2.2 Nexus-5-L Done
- Moztrap result - 30 fail, 1216 pass, 100% complete
- Bug list
- Bluetooth PTS/V2.2 Flame-KK Done
- Bug Meta dependency tree - 0 open bugs
Functional Testing
- General
- Health Status: Yellow
- Details: 2.2+ http://mzl.la/1CDEaqi 13 bugs
- RTL (Josh)
- Health Status: Green
- Details: 0 blocker
- Cut, Copy, and Paste (Gerry, Edward, Teri)
- Health Status: Green
- Details: 0 blocker
- System Platform (Gerry)
- Health Status:
- Details: 1 blocker
- NFC (Alison)
- Health Status: Green
- Details: 0 blocker
- MTP/UMS (Alison)
- Health Status: Green
- Details: 0 blocker
- System Frontend bugs (Hermes)
- Health Status: Green
- Details: 0 blocker
- CAF bugs (Eric)
- Health Status: Yellow
- Details: http://mzl.la/1ELeOXg 5 Blockers
- COMMS, RIL bugs (Eric)
- Health Status: Yellow
- Details: http://mzl.la/1Jk9tuH 1 Blocker
- Bluetooth (Eric, Teri)
- Health Status: Green
- Details: 0 blocker
- Productivity (Teri)
- Health Status: Green
- Details: 0 blocker
- Metrics (Shing)
- Health Status: Green
- Details: 0 blocker
- Video (eric)
- Health Status: Green
- Details: 0 blocker
- Music (eric)
- Health Status: Green
- Details: 0 blocker
- Gallery (eric)
- Health Status: Green
- Details: 0 blocker
Exploratory Test
Flame-KK & NEXUS-5-L
- 321 bugs found
- UNCONFIRMED, NEW, ASSIGNED, REOPENED - 296 bugs found.
- UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 2 bugs found.
- RESOLVED, VERIFIED, CLOSED - 242 bugs found.
- RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 21 bugs found.
- Regression bugs
- UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 2 bugs found.
- RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 292 bugs found.
Stability
NEXUS-5-L 5.1
- Environment
- Display DIM
- Rest time with dummy case (30 sec)
- Running tests ** 14 case http://bit.ly/1aWkJyY
- NEXUS *5 with patch for [AudioSystem issue] https://bugzil.la/1155490
- Result
- NEXUS-5-L 5.1
- Current run - NEXUS-5-L 5.1 * 4: 104.81 hrs
V2.2 Test Case Creation
On Track
- Moztrap link of V2.2 New Features suite.
Test Case Creation WIP
- [2.2-feature-qa+] & in-moztrap? (0 bugs) --> WORK DONE
- feature-b2g=2.2+ & in-moztrap? (0 bugs) --> WORK DONE
Test Case Creation DONE
- [2.2-feature-qa+] & in-moztrap+ (19 bugs) --> WORK DONE
- feature-b2g=2.2+ & in-moztrap+ (78 bugs) --> WORK DONE
V2.2 Testing Verification Status
Overall Verification Summary
Need Verification
ID | Qa contact | Status | Resolution | Component | Assigned to | Summary |
---|---|---|---|---|---|---|
987532 | No-Jun Park [:njpark] | RESOLVED | WORKSFORME | Performance | Jerry Shih[:jerry] (UTC+8) (inactive) | [meta] Project Silk on B2G. |
1018320 | RESOLVED | FIXED | DOM: Core & HTML | Andrea Marchesini [:baku] | Implement RequestSync API for FirefoxOS | |
1020757 | RESOLVED | FIXED | Gaia::Settings | EJ Chen [:eragonj][:小龍哥][Contributor mode on](inactive) | [FDN] Enable/Disable FDN asks to enter PUK if the enable/disable FDN failed | |
1043558 | RESOLVED | FIXED | Graphics | Sotaro Ikeda [:sotaro] | Use gralloc buffers for WebRTC local camera preview | |
1061130 | RESOLVED | FIXED | Gaia::Dialer | Gabriele Svelto [:gsvelto] | [STK] 3GPP TS 22.030 6.6.4 An abbreviated dialling code shall be able to be read using the following procedure: N(N)(N)# | |
1068963 | RESOLVED | INCOMPLETE | Audio/Video | Blake Wu [:bwu][:blakewu] | Frame drops during end-to-end playback | |
1071484 | RESOLVED | FIXED | Audio/Video | Bobby Chien | [Meta][User story] Support *.mkv container format | |
1073419 | RESOLVED | WONTFIX | General | [ALA] All gecko code needed to support adjustable location accuracy | ||
1082268 | Edward Chen[:edchen] | RESOLVED | WORKSFORME | Gaia::Music | Bobby Chien | [meta] Music app launch latency is worse compared to Android on quad core 1 GB device |
1082290 | RESOLVED | FIXED | Performance | Dave Hylands [:dhylands] | Implement cgroup swappiness feature for low memory target (256 MB) | |
1083953 | RESOLVED | FIXED | Gaia | marta | [META] Privacy Panel move from dev_apps to apps | |
1086963 | Edward Chen[:edchen] | RESOLVED | WORKSFORME | Gaia::Gallery | Bobby Chien | [meta] Gallery app launch latency is worse than Android on quad core device |
1092427 | Eric Chang [:ericcc] [:echang] | RESOLVED | FIXED | Gaia::Dialer | Tamara Hills [:thills] | Dialer app should follow text selection pattern. |
1092437 | Eric Chang [:ericcc] [:echang] | RESOLVED | FIXED | Gaia::SMS | Messages app should follow text selection pattern | |
1092960 | RESOLVED | FIXED | Gaia::Camera | Wilson Page [:wilsonpage] | Camera app to follow text selection pattern | |
1092966 | RESOLVED | FIXED | Gaia::FMRadio | Justin D'Arcangelo [:justindarc] | FM Radio app to follow text selection pattern | |
1092967 | RESOLVED | FIXED | Gaia::Gallery | Punam Dahiya [:pdahiya] | Gallery app to follow text selection pattern | |
1092969 | Eric Chang [:ericcc] [:echang] | RESOLVED | FIXED | Gaia::Music | Dominic Kuo [:dkuo] | Music app to follow text selection pattern |
1092972 | RESOLVED | FIXED | Gaia::Video | Russ Nicoletti [:russn] | Video app to follow text selection pattern | |
1094121 | RESOLVED | FIXED | General | Steven Yang [:styang] | [meta] Android L 5.0 Porting for B2G | |
1097865 | Shing Lyu [:shinglyu] | RESOLVED | WONTFIX | Gaia::System | Marshall Culpepper [:marshall_law] | [Telemetry] Collect metrics for voice call info |
1098970 | RESOLVED | FIXED | GonkIntegration | Boris Chiou [:boris] | [lollipop] Porting NativeWindow from LL to gonk | |
1099076 | Shing Lyu [:shinglyu] | RESOLVED | WONTFIX | WebRTC | Screen sharing support on B2G | |
1099079 | RESOLVED | WONTFIX | Gaia | Implement permission dialog for screen sharing | ||
1107291 | RESOLVED | FIXED | MTP/UMS | viral [:viralwang] | SD/Storage (fuse) for L on Nexus 5 | |
1107296 | RESOLVED | WORKSFORME | General | Jamin Liu [:jaliu][:Jamin] | [meta] GPS Android L Porting | |
1107298 | RESOLVED | FIXED | General | viral [:viralwang] | [meta] Sensors Android L Porting | |
1107305 | RESOLVED | FIXED | General | Kai-Zhen Li [:seinlin][:kli] | [gonk-l] the error message: could not load library "libsigchain.so" from LD_PRELOAD | |
1109422 | Shing Lyu [:shinglyu] | RESOLVED | FIXED | General | Marshall Culpepper [:marshall_law] | AppUsage Metric collection enhancements for MonthlyActiveUsers computation |
1115356 | RESOLVED | INVALID | Gaia::Settings | Josh Cheng [:josh] | [meta]Privacy Bugs Meta |
30 Total; 0 Open (0%); 30 Resolved (100%); 0 Verified (0%);
Verification Done
ID | Qa contact | Status | Resolution | Component | Assigned to | Summary |
---|---|---|---|---|---|---|
1107304 | VERIFIED | DUPLICATE | General | [meta] BT Android L Porting |
1 Total; 0 Open (0%); 0 Resolved (0%); 1 Verified (100%);