Firefox OS/DeviceQA/DeviceQAWiki
From MozillaWiki
< Firefox OS | DeviceQA(Redirected from FirefoxOS/DeviceQA/DeviceQAWiki)
Contents
Note of handling device project
- Weekly report should be delivered to stakeholders.
- "fxosqa-report-watchlist" is a public mailing list, please consider if report should be delivered to them.
Manual Test
- Sanity Test
- Smoke Test (Manual/Auto)
- Acceptance Test
- CRT (Certification Requirement Test) [TBD]
- Functional Full Test
- Regression Test
- Exploratory Test
- Performance Test (Manual/Auto)
- RAT (Release Acceptance Test)
Automation
When a project kick-off, Device QA team will start to arrange following environment for testing. Partners should aware it's not required. However, we still recommend partners setup the same environment for quick responding issues found at both sides. We also suggest to discuss about which testing you'd like to have and their pass criteria with TAM before a project kick-off.
Functionality
- Gaia-UI-Test
- gaia-ui-tests cover functionality testing by checking and interacting with HTML elements rendered on screen. Device QA team suggests to execute on real devices.
- Suggested Pass Criteria: All green for five days
- MDN: Gaia-ui-test
Stability
- MTBF (Mean Time Between Failures)
- MTBF is often used for measuring stability. Failures mentioned here include crashes and hang.
- Suggested Pass Criteria: MTBF > 100 hours
- MDN: MTBF
- Wiki: MTBF
- Monkey Test
- Monkey Test of Firefox OS is based on Orangutan tool.
- Suggested Pass Criteria: No crashed > 100 hours
- Github: Monkey Test
- Endurance Test (not maintained in Taipei)
- Endurance won't be part of our regular tests since it takes more time. You can have a look of endurance testing result of our reference phone (Flame) from datazilla.
- MDN: Endurance
Performance
- Performance (not maintained in Taipei)
- You can have a look of performance result of our reference phone (Flame) from datazilla
- MDN: Gaia UI Performance
Certification
- MCTS (Mozilla Certificate Test Suite)
UX Reference
Firefox OS UX
. v2.1: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1759289051 . v2.0: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1732424767 . v1.4: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1226137087 . v1.3: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1147153091 . v1.2: https://mozilla.app.box.com/s/44utizl9oz4eupyu3fuu/1/1030015572
Productivity
Email (Juwei) : https://mozilla.box.com/s/ltg9nwbw134p8e0hzrnr Calendar (Harly) : https://mozilla.box.com/s/as74q673zgr7c7d28xeh Clock (Juwei) : https://mozilla.box.com/s/x1b7dsp7v21sorvp65gr
Comms
Dialer (Carrie) : https://mozilla.app.box.com/files/0/f/1803746785/Dialer Message (Jenny) : https://mozilla.app.box.com/files/0/f/1966102506/Messages Contact (Carrie) : https://mozilla.app.box.com/files/0/f/2107691536/Contact
System
Keyboard & Text Selection (Omega) : https://mozilla.box.com/s/6cu12ppxdr2m0czuurel Settings (Jenny) : https://mozilla.box.com/s/og17l1iy5so2akk5xeo0 NFC (Juwei) : https://mozilla.box.com/s/u1gxr4io329zn3m27kq1
