Confirmed users
94
edits
Firefox OS/DeviceQA/DeviceQAWiki: Difference between revisions
Jump to navigation
Jump to search
Firefox OS/DeviceQA/DeviceQAWiki (view source)
Revision as of 10:04, 10 September 2014
, 10 September 2014no edit summary
No edit summary |
|||
| Line 1: | Line 1: | ||
== | == Note of handling device project == | ||
* Weekly report should be delivered to stakeholders. | * Weekly report should be delivered to stakeholders. | ||
* "[https://mail.mozilla.org/listinfo/fxosqa-report-watchlist fxosqa-report-watchlist]" is a public mailing list, please consider if report should be delivered to them. | * "[https://mail.mozilla.org/listinfo/fxosqa-report-watchlist fxosqa-report-watchlist]" is a public mailing list, please consider if report should be delivered to them. | ||
== | == Manual Test == | ||
* Sanity Test | * Sanity Test | ||
* Smoke Test (Manual/Auto) | * Smoke Test (Manual/Auto) | ||
| Line 16: | Line 14: | ||
* RAT (Release Acceptance Test) | * RAT (Release Acceptance Test) | ||
== | == Automation == | ||
When a project kick-off, Device QA team will start to arrange following environment for testing. Partners should aware it's not required. However, we still recommend partners setup the same environment for quick responding issues found at both sides. We also suggest to discuss about which testing you'd like to have and their pass criteria with TAM before a project kick-off. | When a project kick-off, Device QA team will start to arrange following environment for testing. Partners should aware it's not required. However, we still recommend partners setup the same environment for quick responding issues found at both sides. We also suggest to discuss about which testing you'd like to have and their pass criteria with TAM before a project kick-off. | ||
=== Functionality === | |||
* Gaia-UI-Test | * Gaia-UI-Test | ||
** gaia-ui-tests cover functionality testing by checking and interacting with HTML elements rendered on screen. Device QA team suggests to execute on real devices. | ** gaia-ui-tests cover functionality testing by checking and interacting with HTML elements rendered on screen. Device QA team suggests to execute on real devices. | ||
** Suggested Pass Criteria: All green for five days | ** Suggested Pass Criteria: All green for five days | ||
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/gaia-ui-tests MDN: Gaia-ui-test] | ** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/gaia-ui-tests MDN: Gaia-ui-test] | ||
=== Stability === | |||
* MTBF (Mean Time Between Failures) | * MTBF (Mean Time Between Failures) | ||
** MTBF is often used for measuring stability. Failures mentioned here include crashes and hang. | ** MTBF is often used for measuring stability. Failures mentioned here include crashes and hang. | ||
| Line 35: | Line 33: | ||
** Endurance won't be part of our regular tests since it takes more time. You can have a look of endurance testing result of our reference phone (Flame) from [https://datazilla.mozilla.org/ datazilla]. | ** Endurance won't be part of our regular tests since it takes more time. You can have a look of endurance testing result of our reference phone (Flame) from [https://datazilla.mozilla.org/ datazilla]. | ||
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Endurance MDN: Endurance] | ** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Endurance MDN: Endurance] | ||
=== Performance === | |||
* Performance (not maintained in Taipei) | * Performance (not maintained in Taipei) | ||
** You can have a look of performance result of our reference phone (Flame) from [https://datazilla.mozilla.org/ datazilla] | ** You can have a look of performance result of our reference phone (Flame) from [https://datazilla.mozilla.org/ datazilla] | ||
** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests MDN: Gaia UI Performance] | ** [https://developer.mozilla.org/en-US/Firefox_OS/Platform/Automated_testing/Gaia_performance_tests MDN: Gaia UI Performance] | ||
=== Certification === | |||
* MCTS (Mozilla Certificate Test Suite) | * MCTS (Mozilla Certificate Test Suite) | ||