B2G/QA/2.2/2015-05-18Status

From MozillaWiki
< B2G‎ | QA‎ | 2.2
Jump to navigation Jump to search

V2.2 Highlight & Risk

  • Highlight
    • NEXUS-5-L 5.1 PVT build MTBF 74.063 hrs
    • FLAME-KK 4.4 PVT build MTBF 22.08 hrs
    • 26 bugs with blocking=2.2+ http://mzl.la/1CDEaqi
    • 10 bugs with blocking=2.2? http://mzl.la/1CDEKV4
    • 16 bugs with CAF-v2.2-metabug http://mzl.la/1ELeOXg
    • Regression check on resolved bugs before CC.
    • Availability of pvt FLAME-L is unknown, we will test dsds on local build first.

V2.2 Milestone

  • 2015/06/08 V2.2 CC
  • 2015/04/29 V2.2 Updated FC
  • 2015/04/06 V2.2 FC
  • 2015/02/23 V2.2 FL

V2.2 Testing Scope

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


V2.2 Testing by Type

Smoke Testing

Nexus-5-L 5.1 Yellow

  • Testing on PVT build from 2015/04/14
  • Cases blocking manual test
    • OTA not working, but the update channel is fine for 2.2->2.2.
  • Manual - are available on Google Drive
  • Known issues - no FM radio support, no DSDS support and no UMS support.

Flame-KK Yellow

  • Bugs breaking the Manual smoketests
  • 'Mozilla Mobile' temporarily turned back, but we still need to update our config for future testing.
  • Bugs breaking the automation smoketests, couple automation bugs need to investigate.
    • [Investigating] test_clock_create_new_alarm.py - Bug 1138278- [Clock] After running test_clock_create_new_alarm.py, it should set the alarm time to 1 min more than the current time
    • [Investigating] [Not reproduced manually] test_ftu_skip_tour.py - Bug 1134257 - test_ftu_skip_tour should reflect new Send Data initial state and separate SIM- and network-available conditions
    • [Investigating] [Not reproduced manually] gallery/test_gallery_flick.py - Bug 1161441 - test_gallery_flick.py failed
    • [Investigating] [Reproduced manually] test_marketplace_packaged_app.py - Bug 1158732 - Marketplace app always shows notification/message 'You have been signed out' on startup
    • [Not reproduced] test_sms_with_attachments.py - Bug 1151696 - [v2.2] Investigate test_sms_with_attachments.py failure . (Cannot reproduce locally)
    • [Not reproduced manually] test_music_artist_mp3.py - Bug 1149886 - using to replace at Music app results in unable to locate <span.list-song-index> error

Regression Testing

3rd Run (2015/05/19)

  • Planned on regression bugs

2nd Run (2015/04/14)

  • Result
    • 1119(total), 1090(passed), 8(failed), 21(invalid)
    • 0.007% fail rate
  • bugs
    • Total 5
    • http://mzl.la/1cYZZau (Bugs related to software home button, privacy or ui polish will be skipped for now)
  • Result
    • ​342(total), 334(passed), 5(failed), 3(invalid)
    • ​0.01% fail rate

1st Run (2015/03/03)

Functional Testing

Exploratory Test

Flame-KK & NEXUS-5-L

  • Regression bugs

Stability

NEXUS-5-L 5.1

- Result

  • NEXUS-5-L 5.1
    • Current run - NEXUS-5-L 5.1 *5: 74.063 hrs
      • test with local build which underclock cpu to single core.


FLAME-KK 4.4

  • Environment
    • Display DIM
    • Rest time with dummy case (30 sec)
    • FLAME *9

- Result

V2.2 Test Case Creation

On Track

Test Case Creation WIP

Test Case Creation DONE

  • [2.2-feature-qa+] & in-moztrap+ (19 bugs) --> WORK DONE
  • feature-b2g=2.2+ & in-moztrap+ (78 bugs) --> WORK DONE

V2.2 Testing Verification Status

Overall Verification Summary

Need Verification

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Verification Done

No results.

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);

Retrieved from "https://wiki.mozilla.org/index.php?title=B2G/QA/2.2/2015-05-18Status&oldid=1075012"