B2G/QA/2.5

From MozillaWiki
< B2G‎ | QA
Jump to: navigation, search

Team Breakdown

Testing

  • V2.5 Contact window
    • Overall : Naoki Hirata
    • MGSEI related: William Hsu
    • QAnalyst : Peter Bylenga / Kevin Tucker
    • Automation: Johan Lorenzo
  • Smoketest
    • Daily by QAnalyst + MGSEI
    • Make sure not breaking anything on flame-kk.
    • Make sure working okay on L device.
    • Daily report:
  • Smoketest Automation
    • Daily by
    • Daily by
    • Make sure not breaking anything on Flame-kk.
    • Make sure working okay on L device.
    • Daily report:
      • gaia ui test -
  • Automation new case creation & QA Ownership
    • Feature, bug, functional owners will help on this.
  • Full run
    • Starting from FL by MGSEI, targeting twice in 6 weeks
    • Also help to report invalid test cases.
    • Mozilla will fix invalid test cases after that.
  • Bugzilla related
    • Marigold & Qanalyst will help with qawanted, verifyme & other tags.
  • MTBF
    • MTBF: Paul Yang
  • Performance
    • Overall QA Team
    • gwagner is the dev manager
  • Exploratory
    • Will limit the scope of testing.
    • By MGSEI + QAnalyst
  • Usability testing

V2.5 QA Owners

Here is a table indicating QA contacts by major areas for Firefox OS V2.5

see https://docs.google.com/spreadsheets/d/1fmMXgcWakykU9s15FraJzxKFRykcu0c1WXfwM-mfiGA/edit#gid=1520387764

Feature by Components

Feature tracking by components

Feature Area Owners

Area QA Contacts
Management Al Tsai, Pallavi Yaramada
Release Lead Naoki Hirata
Productivity Teri Wen, John Dolrus
Media Alison Shiue, No Jun Park
System Front End Naoki Hirata, Alison Shiue
Communications Johan Lorenzo
Performance All QA Members for their areas : see [[1]]
Networking/Connectivity Alison Shiue
System Platform Gerry Chang, Naoki Hirata
Multimedia Platform No Jun Park
Devices Askeing Yen
Bluetooth Teri Wen
Metrics Shing Lyu, Gerry Chang
Stability Paul Yang, Walter Chan

Reports & Update

Code Complete

Release Signoff Status

Feature Complete

Release Signoff Status

Health:

Feature Complete Criteria

FC

  • All feature landing exceptions must be resolved
    • feature-b2g:
    • blocking-b2g:
  • Bugs with keyword late-l10n should be zero
    • Yes,
  • Chipset vendor requirements or bugs needed to meet their FC date must be resolved
    • CAF FC for 2.5
  • Completed one functional test run
    • Yes, 2 runs.
  • QA defines 2 areas of "functional areas" for exploratory testing and lead will send a "signoff" note when he feels its qualitatively ready -
    • Yes.
  • Smoketest Greenness - Latest length of time that we haven't had smoketest regressions needs to be greater than five days
    • No, OTA issue is not yet resolved.
  • MTBF Test Run lasts for 30+ hours:
    • Yes.
    • NEXUS-5-L 5.1 PVT build MTBF 38.5 hrs & counting.
    • FLAME-KK 4.4 PVT build MTBF 34.42 hrs.
  • blocking_b2g = 2.5+
ID Assigned to Status Resolution Component Summary
1166136 Seth Fowler [:seth] [:s2h] NEW Layout: Images Too many images are locked in the B2G Gallery app
1172163 [:fabrice] Fabrice Desré NEW DOM: Apps [Aries][Homescreen][OTA] Uninstalled apps are added back to the homescreen during an OTA System Update.
1184592 REOPENED Gaia::Clock Alarm didn't ring until I power up the screen
1190498 NEW Preinstalled B2G Apps [Gallery] Sharing photo from Gallery to Facebook does not set photo in post box.
1196268 Kevin Grandon :kgrandon REOPENED Gaia::Components The cancel button is missing the oval button in the Action Menu
1206646 NEW Gaia::Camera [Aries KK]Connect device with computer by USB(UMS), then record a video, tap the Stop button, it does not react.
1207371 NEW Gaia::Customizer [Aries] Customizer app is present in Spark Aries builds and cannot be disabled or deleted, despite being non-functional, and labeled as obsolete.
1209434 REOPENED Gaia::Music [Flame KK][Music NGA]Music can't resume play after user pauses it over 60s.
1211115 REOPENED Gaia::System::Accessibility When the screen reader is enabled, sometimes the focus is not correctly set after some operations.
1220032 NEW DOM: Device Interfaces SystemMessageInternal may fail to (re) launch an app if the app shuts down/unregisters around the time the message is sent
1222387 REOPENED Gaia::Music [music] Fast-seeking to the end of the song doesn't correctly start next song
1223224 NEW Gaia::System [Add-ons] Holding the power button during boot does not disable add-ons on Flame
1226612 NEW Gaia::Components [RTL] In Persian RTL the datepicker's year is composed of a number and a suffix, causing severe overlapping
1227011 NEW Gaia::System No prompt for creating/reading contact on Privileged App
1227475 NEW AudioChannel Busy tone is played through the speaker

15 Total; 15 Open (100%); 0 Resolved (0%); 0 Verified (0%);


  • blocking_b2g = 2.5?
ID Assigned to Status Resolution Component Summary
1196324 NEW Gaia::Feedback phone keeps turning on in my pocket
1209326 NEW General Use correct manifest properties when verifying addon update
1219449 NEW Gaia::Keyboard [Keyboard] The french word "pendait" is both suggested and corrected
1220812 NEW Gaia::System::Lockscreen After unlocking the device from the lockscreen there is a long pause until reaching the home screen
1221802 NEW Gaia::First Time Experience There is no visual indication in FTU that the device is trying to connect to a hidden network
1222539 NEW Gaia::Homescreen [Old Homescreen] The old homescreen should display the pinned sites, possibly in a degraded form
1223598 NEW Metrics [Aries] User can change metrics data setting in FTU before it is set as unselectable on Dogfood builds.
1224112 NEW JSMarionette Can't use the custom build to run the Marionette JS test
1224509 NEW Gaia::System::Lockscreen [Lock Screen] The lockscreen will turn to landscape mode automatically.
1226747 NEW Gaia::System::Music Control Music control does not appear in the notifications drawer after opening and closing camera app
1228513 NEW Wifi Turn on wifi and when it is searching aps, turn on "Airplane mode", the wifi will be toggled on/off alternately
1231736 NEW Gaia Display of Greek characters is not fully supported
1231833 NEW Gaia::System::Accessibility [2.5] Screen becomes completely unresponsive to touches after turning off screen reader

13 Total; 13 Open (100%); 0 Resolved (0%); 0 Verified (0%);


Feature Landing

Release Signoff Status

Functional Signoff Status

Smoketest Status

Nominations

ID Assigned to Status Resolution Component Summary

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Blockers

ID Assigned to Status Resolution Component Summary

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Gaia UI Test Regressions Status

Nominations

ID Assigned to Status Resolution Component Summary

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Blockers

ID Assigned to Status Resolution Component Summary

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Test Coverage Completed

Everything with [2.5-feature-qa+] in QA Whiteboard

ID Qa contact Status Resolution Component Assigned to Summary

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Verification Status

Overall Verification Summary

Verification Needed

ID Qa contact Status Resolution Component Assigned to Summary

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Verification Complete

ID Qa contact Status Resolution Component Assigned to Summary

0 Total; 0 Open (0%); 0 Resolved (0%); 0 Verified (0%);


Definitions

Health Notations

  • [ON TRACK] QA thinks the health is trending towards green, but signoff isn't complete yet
  • [AT RISK] QA thinks the health is trending towards yellow, but signoff isn't complete yet
  • [DONE] QA has signed off to indicate the health is green
  • [MISSED] QA has chosen to not sign off to indicate the health is red

FLR Functional Team Criteria

FL Functional Team Criteria

The functional team FL health was determined against the following criteria:

  • MozTrap test coverage done?
  • All user stories verified to spec?
  • Smoketests green around your area?

The lowest health score out of each of the criteria above is used as the team's overall health status.

FC Functional Team Criteria

The functional team FC health is determined against the following criteria:

  • Have all features gone through a functional test run?
  • Did any risky feature areas get exploratory testing?
  • Were smoketests around your area green for 5+ days?
  • Did each outstanding FC signoff blocker get fixed?

The lowest health score out of each of the criteria above is used as the team's overall health status.