B2G/QA/Meetings/2015-03-11

// For 2015/03/11

Overview

• Meeting Name: B2G QA Roundtable
• Meeting Time: Wednesdays @ 5pm - 5:45pm PDT
• Vidyo Room: QA Vidyo Room
• Video Link: https://v.mozilla.com/flex.html?roomdirect.html&key=nL4PESn1IaVh
• Dial in: 1-800-707-2533, #92, #9245
• Meeting Notes: https://wiki.mozilla.org/B2G/QA/Meetings
• Testrun Schedule: http://bit.ly/1jQzW79
• Roadmap for releases: https://wiki.mozilla.org/B2G/Roadmap
• B2G dashboard: http://charts.mozilla.org/
• QA Roundtable Template: https://etherpad.mozilla.org/b2g-qa-roundtable-template (Clone this)

Vacation plan

2.2 Feature work

• General (eric)
  • Flame-L might be late for FC, roughly 6 weeks or more needed after SOW (SOW, legal review, porting, build time), PM will have an internal discussion to decide what to do.
  • feature-b2g|blocking-b2g == 2.2?|+ [not resolved] http://mzl.la/1C4X1KT Check if code uplifted
  • feature-b2g|blocking-b2g == 2.2?|+ [resolved] http://mzl.la/1E5mQqM Need verification
  • status-b2g-v2.2 affected not blocking any bugs http://mzl.la/1E5ArOQ
• RTL (Marigold)
  • Health: Red
  • Test case coverage: 100%
  • Details: 15 blockers
    • http://mzl.la/1b2ddSG

• Privacy (Gerry)
  • Health: Yellow
  • Test case coverage:
  • Details: 5 blockers
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1111992
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1093688
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1134711
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1136286
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1137083
    • http://mzl.la/1vvsAfY
• Josh to help talk with dave/eitan on the backout for the compile time config
• Cut, Copy, and Paste (Gerry, Edward, Teri, Eric)
  • Health: Yellow
  • Test case coverage:
  • Details: 2 blockers
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1021499
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1130982
• NFC (Alison)
  • Health: Green
  • Test case coverage:100%
  • Details:
    • No blockers
• short_name support in manifest.webapp (Hermes)
  • Health: Yellow
  • Test case coverage: 100%
  • Details: Not fully landed. 1 WIP blockers
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1137498
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1137501 (Resolved, but not uplift to 2.2)
• Search App (Hermes)
  • Health: Yellow
  • Test case coverage: 100%
  • Details: 5 WIP blockers
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1140325
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1141315
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1140525 (RTL)
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1139807 (RTL)
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1132262 (patch will be landed this week)
• App-grouping (Hermes)
  • Health: Green
  • Test case coverage: 100%
  • Details: No 2.2+ blockers for now
• Task Manager (Hermes)
  • Health: Green
  • Test case coverage: 100%
  • Details: 1 Blockers
    • https://bugzilla.mozilla.org/show_bug.cgi?id=1107139
• L-porting (Hermes)
  • Health: Green
  • Details:
    • Patch of camera recording is waiting for review. https://bugzilla.mozilla.org/show_bug.cgi?id=1120780
• CAF bugs (Eric)
  • Health: Green
  • Test case coverage: 100%
  • Details: Running regression test now.
• TCL bugs (Alison, Eric)
  • Health: Green
  • Test case coverage: 100%
  • Details: Running regression test now.
• Bluetooth (Eric, Teri)
  • Health: Yellow
  • Test case coverage: 100%
  • Details:
  • PTS certification tested on flame-kk this week. 5/6 profile test completed. Couple cases failed but we need to know they are due to steps error or they are actually bugs http://bit.ly/1FLiZl8 https://bugzil.la/1139349
  • Need to manually turn on bluetooth daemon. Currently the new code with daemon is off by default due to bluetooth function break after FOTA, http://bugzil.la/1134821 a patch is under review.
  • What is Bluetooth PTS? https://wiki.mozilla.org/B2G/QA/Bluetooth_PTS
• Metrics (Shing)
  • Health: Yellow
  • Test case coverage: 95%
  • Details:
    • 3 Blocker (2.2?) http://mzl.la/1EbkJ7r
• 256M RAM memory optimization for WFVGA, 5MP. (Teri)
  • Health: Green
  • Test case coverage: 100%
  • Details: Testing in progress, no blocker.
• Multicore performance (Edward)
  • Health:
  • Test case coverage:
  • Better performance on Multicore (App startup time for core apps is comparable to Android
  • Set up raptor for v2.2
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1132515
    • analyze and narrow down result from QA jenkins, http://bit.ly/1Glz30h. The problem should be between 4044 and 4045.

2.2 Smoketests

• Tester: Marigold
• Flame-KK Health Status: Red
  • No OTA or not working, need to investigate, mike was testing with v2.1 yesterday but I cannot find the bug they are working on https://bugzil.la/1141965
• Nexus-5-L Health Status: Red http://bit.ly/1B3Y90x
  • Details: Skipped (17 cases): DSDS, FM, Video recording, OTA

2.2 Full run

• Tester: Marigold
  • Details:
  • Flame-KK http://mzl.la/1wtR60Y 41% done 21 fail
  • RTL Flame-KK http://mzl.la/1EzHpRx 6% done, 23 fail
  • Nexus-5-L http://mzl.la/1EXfMQQ 51% done, 16 fail

2.2 Stability

• Tester: Paul, Alison
• Health Status: Red
• Details:
  • https://bugzil.la/1141418 Bug 1141418 - After 9 hours running, marionette server response incorrectly
  • https://bugzil.la/1141962 Bug 1141962 - [MTBF][Nexus5-L] MTBF testing aborted and found lots of "E MP-Decision: Error 1 setting online status to 1 forcpuX" error in logcat
  • Flame-kk: max: 23 hours, avg.: 15 hours
  • Nexus5-L: Less than 10 hours

Localization

• Tester: Delphine, localizers
• Health Status: Green
• Details: No late-l10n open blockers http://mzl.la/1FcrHLe

3.0 Smoketests

• Tester: Peter Bylenga
• Health Status: Red
• Details: 2 new blockers today, 1 existing qablocker this morning
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1142100 [Windows Management][Notifications] Tapping on a screenshot notification gives the error message "Image is invalid or corrupt and cannot be displayed"
    • Resolved Fixed, will verify
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1142132 BluetoothHandsfreeResultHandler, void>::~BluetoothHALInterfaceRunnable0() - Device crashes when connecting to a bluetooth headset or speaker device.
    • Resolved Fixed, will verify
  • https://bugzilla.mozilla.org/show_bug.cgi?id=1141633 - bug 1140437 broke flame kitkat nightlies
    • Verified Fixed

3.0 Exploratory

• Tester: Peter Bylenga
• Health Status: Red
• Details: 54 Issues written, 12 Blockers & Nominations, 9 RTL issues
  • http://mzl.la/1zaHjY4

Agenda

• [eric] Possible to move the meeting an hour or half hour late?
• [shing] MozTrap enhancement retrospect. (slides: https://dl.dropboxusercontent.com/u/7281903/slides-e148c1--moztrap-enhancement-/20150226pitch.md.html#/ )
• [shing] Wiki test case management system (slides: https://dl.dropboxusercontent.com/u/7281903/slides-1e3a0b--wiki-test-case-management/testmgmt.md.html#/ )
  • [pbylenga] could use https://wiki.mozilla.org/QA/B2G/How_To_Test_Firefox_OS
• [Bhavana] l10n testing update for 2.2. https://etherpad.mozilla.org/2-2TestCases
• [delphine] Privacy Panel: finding a way to expose feature to localizers since have to translate those whatever happens -- see with Gerry. Gerry + Josh confirmed that we can back-out that patch on 2.2 so localizers can see their strings and we can test that on 2.2 during l10n run (ETA = approx.before FC)
• [nhirata] 4:00 pm build for today was cancelled due to back out of https://bugzilla.mozilla.org/show_bug.cgi?id=1142132 confirmation.
• [delphine] suggestion: using pseudolocales (Mirrored English) during exploratory testing -- helps find hardcoded/l12y issues earlier in the cycle then waiting for a Loc Run. Just one tester doing this would probably be enough

This Week - Decisions & Action Items

• [Brian, Eric] Talk to Keven to inform QC about the concern of Flame-L delay. That may not meet the FC criteria.
• Josh Cheng to help on the privacy build situation :https://bugzilla.mozilla.org/show_bug.cgi?id=1132377