B2G/QA/Gaia Test Plan/Email Templates: Difference between revisions
| (One intermediate revision by the same user not shown) | |||
| Line 5: | Line 5: | ||
Use this email template to send out a daily report to the Mozilla testing team for the smoke test results. | Use this email template to send out a daily report to the Mozilla testing team for the smoke test results. | ||
Subject: <Date> Smoke Test Result - <number passed>/<total tests ran> tests passed, <number of blocking issues> blockers | Subject: <Date> Smoke Test Result - <number passed>/<total tests ran> tests passed, <number of blocking issues> blockers, <number of partial fails> | ||
Hi Everyone, | Hi Everyone, | ||
| Line 14: | Line 14: | ||
Tests were performed with: (this can be found in the sources.xml file in the b2g directory) | Tests were performed with: (this can be found in the sources.xml file in the b2g directory) | ||
*Build ID <Insert Build ID here> | |||
*gecko: <Insert Gecko version here> | *gecko: <Insert Gecko version here> | ||
*gaia: <Insert Gaia version here> | *gaia: <Insert Gaia version here> | ||
*Base image <insert Base image here> | |||
*x reboots | *x reboots | ||
*x crashes | *x crashes (Indicate crash reports here, or existing bugs if already on file) | ||
| Line 33: | Line 34: | ||
* ... | * ... | ||
'''New | '''New Issues Not Breaking Smoke Tests''' | ||
* [NEW] <Summary of issue, hyperlink to github/bugzilla issue> - no assignee | * [NEW] <Summary of issue, hyperlink to github/bugzilla issue> - no assignee | ||
Latest revision as of 00:50, 2 November 2013
Smoke Test Email Template
Template Details
Use this email template to send out a daily report to the Mozilla testing team for the smoke test results.
Subject: <Date> Smoke Test Result - <number passed>/<total tests ran> tests passed, <number of blocking issues> blockers, <number of partial fails>
Hi Everyone,
<number passed> out of <total tests ran> tests passed for the <date> build. Here are the major issues that prevented the smoke tests from passing:
- Smoketest: <Link to Google spreadsheet or MozTrap run>
Tests were performed with: (this can be found in the sources.xml file in the b2g directory)
- Build ID <Insert Build ID here>
- gecko: <Insert Gecko version here>
- gaia: <Insert Gaia version here>
- Base image <insert Base image here>
- x reboots
- x crashes (Indicate crash reports here, or existing bugs if already on file)
New Bugs Breaking Smoke Tests
- [NEW] <Summary of issue, hyperlink to github/bugzilla issue>
Existing Bugs Breaking Smoke Tests
- [EXISTING] <Summary of issue, hyperlink to github/bugzilla issue>
- ...
New Issues Not Breaking Smoke Tests
- [NEW] <Summary of issue, hyperlink to github/bugzilla issue> - no assignee
- [NEW] <Summary of issue, hyperlink to github/bugzilla issue> - no assignee
- ...
Please address these issues as soon as possible to get these tests passing.
Sincerely,
<Your Name>
Example
8/31/2012 Smoke Test Result - 10/17 tests passed, 3 blockers
Hi Everyone,
10 out of 17 passed for the 2012-08-31 build. Here are the major issues that prevented the smoke tests from passing:
New Bugs Breaking Smoke Tests
- [NEW] Launching multiple apps on the homescreen occasionally causes the phone to get stuck in a whitescreen of death - no assignee
- [NEW] Cannot usually login into marketplace - crashes upon launch, sometimes immediately during login - no assignee
Existing Bugs Breaking Smoke Tests
- [EXISTING] camera A dialog about "device-storage" appears when a photo is taken - Dale Harvey
New Regressions Not Breaking Smoke Tests
- [NEW] When adding a contact, selecting a zipcode input field should open the number keyboard, not the alpha keyboard - No assignee
- [NEW] Selecting Auto-Configure in the APN settings, hitting cancel, and going back to APN settings shows the values were actually set - No assignee
Please address these issues as soon as possible to get these tests passing.
Sincerely,
Jason Smith