B2G/QA/Gaia Test Plan/Email Templates: Difference between revisions
< B2G | QA | Gaia Test Plan
Jump to navigation
Jump to search
| (12 intermediate revisions by 2 users not shown) | |||
| Line 3: | Line 3: | ||
=== Template Details === | === Template Details === | ||
Use this email template to send out a daily report to | Use this email template to send out a daily report to the Mozilla testing team for the smoke test results. | ||
Subject: <Date> Smoke Test Result - <number passed> | Subject: <Date> Smoke Test Result - <number passed>/<total tests ran> tests passed, <number of blocking issues> blockers, <number of partial fails> | ||
Hi Everyone, | Hi Everyone, | ||
| Line 11: | Line 11: | ||
<number passed> out of <total tests ran> tests passed for the <date> build. Here are the major issues that prevented the smoke tests from passing: | <number passed> out of <total tests ran> tests passed for the <date> build. Here are the major issues that prevented the smoke tests from passing: | ||
'''Bugs Breaking Smoke Tests''' | *Smoketest: <Link to Google spreadsheet or MozTrap run> | ||
Tests were performed with: (this can be found in the sources.xml file in the b2g directory) | |||
*Build ID <Insert Build ID here> | |||
*gecko: <Insert Gecko version here> | |||
*gaia: <Insert Gaia version here> | |||
*Base image <insert Base image here> | |||
*x reboots | |||
*x crashes (Indicate crash reports here, or existing bugs if already on file) | |||
'''New Bugs Breaking Smoke Tests''' | |||
* [NEW] <Summary of issue, hyperlink to github/bugzilla issue> | |||
'''Existing Bugs Breaking Smoke Tests''' | |||
* [EXISTING] <Summary of issue, hyperlink to github/bugzilla issue> | |||
* ... | * ... | ||
'''New | '''New Issues Not Breaking Smoke Tests''' | ||
* [NEW] <Summary of issue, hyperlink to github/bugzilla issue> - no assignee | * [NEW] <Summary of issue, hyperlink to github/bugzilla issue> - no assignee | ||
| Line 31: | Line 48: | ||
=== Example === | === Example === | ||
8/31/2012 Smoke Test Result - 10 | 8/31/2012 Smoke Test Result - 10/17 tests passed, 3 blockers | ||
Hi Everyone, | Hi Everyone, | ||
| Line 37: | Line 54: | ||
10 out of 17 passed for the 2012-08-31 build. Here are the major issues that prevented the smoke tests from passing: | 10 out of 17 passed for the 2012-08-31 build. Here are the major issues that prevented the smoke tests from passing: | ||
'''Bugs Breaking Smoke Tests''' | '''New Bugs Breaking Smoke Tests''' | ||
* [NEW] [https://github.com/mozilla-b2g/gaia/issues/4240 Launching multiple apps on the homescreen occasionally causes the phone to get stuck in a whitescreen of death] - no assignee | * [NEW] [https://github.com/mozilla-b2g/gaia/issues/4240 Launching multiple apps on the homescreen occasionally causes the phone to get stuck in a whitescreen of death] - no assignee | ||
* [NEW] [https://github.com/mozilla-b2g/gaia/issues/4239 Cannot usually login into marketplace - crashes upon launch, sometimes immediately during login] - no assignee | * [NEW] [https://github.com/mozilla-b2g/gaia/issues/4239 Cannot usually login into marketplace - crashes upon launch, sometimes immediately during login] - no assignee | ||
* [EXISTING] [https://github.com/mozilla-b2g/gaia/issues/3395 | |||
'''Existing Bugs Breaking Smoke Tests''' | |||
* [EXISTING] [https://github.com/mozilla-b2g/gaia/issues/3395 camera A dialog about "device-storage" appears when a photo is taken] - Dale Harvey | |||
'''New Regressions Not Breaking Smoke Tests''' | '''New Regressions Not Breaking Smoke Tests''' | ||
* [NEW] | * [NEW] [https://github.com/mozilla-b2g/gaia/issues/4244 When adding a contact, selecting a zipcode input field should open the number keyboard, not the alpha keyboard] - No assignee | ||
* [NEW] [https://github.com/mozilla-b2g/gaia/issues/4245 Selecting Auto-Configure in the APN settings, hitting cancel, and going back to APN settings shows the values were actually set] - No assignee | |||
Please address these issues as soon as possible to get these tests passing. | Please address these issues as soon as possible to get these tests passing. | ||
Sincerely, | Sincerely, | ||
Jason Smith | Jason Smith | ||
Latest revision as of 00:50, 2 November 2013
Smoke Test Email Template
Template Details
Use this email template to send out a daily report to the Mozilla testing team for the smoke test results.
Subject: <Date> Smoke Test Result - <number passed>/<total tests ran> tests passed, <number of blocking issues> blockers, <number of partial fails>
Hi Everyone,
<number passed> out of <total tests ran> tests passed for the <date> build. Here are the major issues that prevented the smoke tests from passing:
- Smoketest: <Link to Google spreadsheet or MozTrap run>
Tests were performed with: (this can be found in the sources.xml file in the b2g directory)
- Build ID <Insert Build ID here>
- gecko: <Insert Gecko version here>
- gaia: <Insert Gaia version here>
- Base image <insert Base image here>
- x reboots
- x crashes (Indicate crash reports here, or existing bugs if already on file)
New Bugs Breaking Smoke Tests
- [NEW] <Summary of issue, hyperlink to github/bugzilla issue>
Existing Bugs Breaking Smoke Tests
- [EXISTING] <Summary of issue, hyperlink to github/bugzilla issue>
- ...
New Issues Not Breaking Smoke Tests
- [NEW] <Summary of issue, hyperlink to github/bugzilla issue> - no assignee
- [NEW] <Summary of issue, hyperlink to github/bugzilla issue> - no assignee
- ...
Please address these issues as soon as possible to get these tests passing.
Sincerely,
<Your Name>
Example
8/31/2012 Smoke Test Result - 10/17 tests passed, 3 blockers
Hi Everyone,
10 out of 17 passed for the 2012-08-31 build. Here are the major issues that prevented the smoke tests from passing:
New Bugs Breaking Smoke Tests
- [NEW] Launching multiple apps on the homescreen occasionally causes the phone to get stuck in a whitescreen of death - no assignee
- [NEW] Cannot usually login into marketplace - crashes upon launch, sometimes immediately during login - no assignee
Existing Bugs Breaking Smoke Tests
- [EXISTING] camera A dialog about "device-storage" appears when a photo is taken - Dale Harvey
New Regressions Not Breaking Smoke Tests
- [NEW] When adding a contact, selecting a zipcode input field should open the number keyboard, not the alpha keyboard - No assignee
- [NEW] Selecting Auto-Configure in the APN settings, hitting cancel, and going back to APN settings shows the values were actually set - No assignee
Please address these issues as soon as possible to get these tests passing.
Sincerely,
Jason Smith