B2G/QA/2.2/2015-06-01Status

V2.2 Highlight & Risk

Highlight
- NEXUS-5-L 5.1 PVT build MTBF 49.34 hrs (needs update)
- FLAME-KK 4.4 PVT build MTBF 19.16 hrs (needs update)
- 13 bugs with blocking=2.2+ http://mzl.la/1CDEaqi
- 21 bugs with blocking=2.2? http://mzl.la/1CDEKV4
- 10 bugs with CAF-v2.2-metabug http://mzl.la/1ELeOXg
- FLAME-L 3.0 local build is ready.

V2.2 Milestone

2015/06/08 V2.2 CC
2015/04/29 V2.2 Updated FC
2015/02/23 V2.2 FL

V2.2 Testing Scope

Focus on bugs in agreed list.
B2G V2.2 Overview Page for more info.

Status NEW: blocking-b2g=2.2+ 27 bugs

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 406 [1] => Not Acceptable ) ) 1

V2.2 Testing by Type

Smoke Testing

Nexus-5-L 5.1 Green

Testing on PVT build from 2015/04/14
Manual test results - are available on Google Drive
- Green for 2+ weeks
Known issues
- No FM radio support, no DSDS support and no UMS support.
- OTA not working, but the update channel is fine for 2.2->2.2.

Flame-KK Green

Manual test results - are available on Google Drive
Automation reports - are available on Github.IO & fxosqa-report-watchlist.
- Green for 2+ weeks
Known issues
- Bug 1165195 - [2.1][2.2][Flame][Settings][OTA] Can't detect update.

Regression Testing

3rd Run (2015/05/19)

Exploratory tests

2nd Run (2015/04/14)

NEXUS-5-L 2nd Run (focus on essential functionality) Done
- Test cases http://mzl.la/1D76gXG

Result
- 1119(total), 1090(passed), 8(failed), 21(invalid)
- 0.007% fail rate

bugs
- Total 5
- http://mzl.la/1cYZZau (Bugs related to software home button, privacy or ui polish will be skipped for now)

FLAME-KK 2nd Run (focus on dsds and gaia apps) Done
- Test cases http://mzl.la/1aKYIC8

Result
- 342(total), 334(passed), 5(failed), 3(invalid)
- 0.01% fail rate

bugs
- Total 5
- http://mzl.la/1GifEOz

1st Run (2015/03/03)

RTL/V2.2 Flame-KK Done
- Moztrap result - 76 fail, 326 pass, check with Josh for detail.

Full Regression/V2.2 Flame-KK Done
- Moztrap result - 55 fail, 3426 pass, 100% complete

Partial Regression/V2.2 Nexus-5-L Done
- Moztrap result - 30 fail, 1216 pass, 100% complete
- Bug list

Bluetooth PTS/V2.2 Flame-KK Done
- Bug Meta dependency tree - 0 open bugs

Functional Testing

General
- Flame-L 3.0 local build ready, will start checking.
- blocking-2.2+ http://mzl.la/1CDEaqi 13 in total
- blocking-2.2? http://mzl.la/1CDEKV4 21 in total
- CAF http://mzl.la/1IgvLNh 10 in total

RTL (Josh)
- Health Status: Green
- Details: 2 blockers
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1165824 (Dup 1154438, Patch ready waiting uplift for 2.2)
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1166203 (Assigned to Francisco)

Cut, Copy, and Paste (Gerry, Edward, Teri)
- Health Status: Green
- Details: 0 blockers

System Platform (Gerry)
- Health Status:
- Details: 2 blockers
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1158094, patch already has, but has some regression issue, they are fixing
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1161364 - review+

NFC (Alison)
- Health Status: Green
- Details:
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1163957 (patch reviewing)
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1165841 (RESOLVED FIXED, wait uplift to v2.2)

MTP/UMS (Alison)
- Health Status: Green
- Details: No blockers

System Frontend bugs (Hermes)
- Health Status: Yellow
- Details:
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1155424
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1164772
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1165849 (RESOLVED FIXED, wait uplift to v2.2)
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1157615 (RESOLVED FIXED, wait uplift to v2.2)

CAF bugs (Eric)
- Health Status: Yelllow
- Details: http://mzl.la/1ELeOXg 11 bugs, 5 without owner
  - Josh to identify owners for those 5 bugs
  - https://bugzilla.mozilla.org/show_bug.cgi?id=1155547
  - Cervantes created patch but TC fail to push.

COMMS, RIL bugs (Eric)
- Health Status: Yellow
- Details: 1 https://bugzil.la/1168515 reported by CAF, not sure this is dialer or RIL's issue. Also need to check 3.0's behavior

Bluetooth (Eric, Teri)
- Health Status: Green
- Details: no blockers

Productivity (Teri)
- Health Status: Green
- Details: No Blocking

Metrics (Shing)
- Health Status: Green
- Details: No blockers

Video (eric)
- Details: Yellow
- Trying to do a complete check on patch for https://bugzil.la/1162639 https://bugzil.la/1123950

Music (eric)
- 1 WIP https://bugzil.la/1168549

Gallery (eric)
- Health Status: Red
- Details https://bugzil.la/1161734 from CAF
  - Discuss today with Josh, Eric, Brian

Exploratory Test

Flame-KK & NEXUS-5-L

317 bugs found
- UNCONFIRMED, NEW, ASSIGNED, REOPENED - 297 bugs found.
- UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 4 bugs found.
- RESOLVED, VERIFIED, CLOSED - 232 bugs found.
- RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 22 bugs found.

Regression bugs
- UNCONFIRMED, NEW, ASSIGNED, REOPENED & 2.2?, 2.2+ - 7 bugs found.
- RESOLVED, VERIFIED, CLOSED & 2.2?, 2.2+ - 288 bugs found.

Stability

- no new data, showing data from 7 days ago - NEXUS-5-L 5.1

Environment
- Display DIM
- Rest time with dummy case (30 sec)
- Running tests ** 14 case http://bit.ly/1aWkJyY
- NEXUS *5 with patch for [AudioSystem issue] https://bugzil.la/1155490

- Result

NEXUS-5-L 5.1
- Current run - NEXUS-5-L 5.1 *5: 49.34 hrs

NEXUS-5-L 5.1 Rest time: 10%
- [AudioSystem issue] https://bugzil.la/1155490

FLAME-KK 4.4

Environment
- Display DIM
- Rest time with dummy case (30 sec)
- FLAME *9

- Result

FLAME-KK 4.4
- FLAME-KK 4.4 * 14: MTBF 19.16 hrs
  - https://bugzil.la/1141418

V2.2 Test Case Creation

On Track

Moztrap link of V2.2 New Features suite.

Test Case Creation WIP

[2.2-feature-qa+] & in-moztrap? (0 bugs) --> WORK DONE
feature-b2g=2.2+ & in-moztrap? (0 bugs) --> WORK DONE

Test Case Creation DONE

[2.2-feature-qa+] & in-moztrap+ (19 bugs) --> WORK DONE
feature-b2g=2.2+ & in-moztrap+ (78 bugs) --> WORK DONE

V2.2 Testing Verification Status

Overall Verification Summary

Need Verification

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 406 [1] => Not Acceptable ) ) 1

Verification Done

Bugzilla query error

Array ( [type] => error [message] => http-bad-status [params] => Array ( [0] => 406 [1] => Not Acceptable ) ) 1

B2G/QA/2.2/2015-06-01Status

Contents

V2.2 Highlight & Risk

V2.2 Milestone

V2.2 Testing Scope

Bugzilla query error

V2.2 Testing by Type

Smoke Testing

Regression Testing

3rd Run (2015/05/19)

2nd Run (2015/04/14)

1st Run (2015/03/03)

Functional Testing

Exploratory Test

Stability

V2.2 Test Case Creation

Test Case Creation WIP

Test Case Creation DONE

V2.2 Testing Verification Status

Overall Verification Summary

Bugzilla query error

Bugzilla query error

Navigation menu

B2G/QA/2.2/2015-06-01Status

V2.2 Highlight & Risk

V2.2 Milestone

V2.2 Testing Scope

Bugzilla query error

V2.2 Testing by Type

Smoke Testing

Regression Testing

3rd Run (2015/05/19)

2nd Run (2015/04/14)

1st Run (2015/03/03)

Functional Testing

Exploratory Test

Stability

V2.2 Test Case Creation

Test Case Creation WIP

Test Case Creation DONE

V2.2 Testing Verification Status

Overall Verification Summary

Bugzilla query error

Bugzilla query error

Navigation menu

Search