Processes for Smoke and Dogfood

Smoke Test Process

Builds are daily tested by QAnalysts; nightly-latest and dogfood-latest builds are tested:
- Build location for Flame nightly: https://tools.taskcluster.net/index/artifacts/#gecko.v1.mozilla-central.latest.linux.flame-kk-ota/gecko.v1.mozilla-central.latest.linux.flame-kk-ota.debug
- Build location for Aries dogfood: https://tools.taskcluster.net/index/artifacts/#gecko.v1.mozilla-central.latest.linux.aries-dogfood/gecko.v1.mozilla-central.latest.linux.aries-dogfood.debug
- Test Suite in MozTrap: https://moztrap.mozilla.org/manage/cases/?filter-suite=241
- Initial Logging : https://docs.google.com/spreadsheets/d/13BjtbYDVaiC6TvOAd1x-SuoYNNM8aH5-xdJS7TU3o-U/edit#gid=328354895
If there is any failure, these will fail the build from being tested further for release for nightly/dogfood.
Depending on the failure, the build may or not be used QA for feature testing.
Daily automation results from jenkins running GIP tests are looked at, reviewed, and seeing if they manually reproduce.

Results are posted in 4 different places:
- Announced in QA Standup meeting daily : #QA-B2G in vidyo 10 am PDT Daily
- Posted in QAStandup etherpad daily : https://public.etherpad-mozilla.org/p/b2g-qa-daily-standup
- Posted in https://mail.mozilla.org/listinfo/fxosqa-report-watchlist daily
- Posted in QA Weekly round table weekly: https://public.etherpad-mozilla.org/p/b2g-qa-roundtable ; #QA-B2G in vidyo 5 pm PDT Wednesdays
Daily automation results are posted to "gaia-ui-automation@mozilla.org"

If smoke test passes :
- For nightly builds, nightly-latest is pushed to nightly on [1]. ( currently on hold until new base build/FOTA occurs)
  - https://wiki.mozilla.org/B2G/QA/Flame_OTA is updated when the change is made
- Dogfood Tests are executed for further testing:
  - Dogfood Test Suite : https://moztrap.mozilla.org/manage/cases/?filter-suite=846

Dogfood test results are posted in https://mail.mozilla.org/listinfo/fxosqa-report-watchlist
If passed then dogfood channel is updated on balrog with the build
- a quick sanity test to make sure people download the right build is done
- https://wiki.mozilla.org/B2G/QA/Spark is updated when the change is made

Bugs that are smoke test failures should be investigated for regression ranges
- If a regression window is found :
  - Dev is notified and discussion occurs
  - If a dev cannot be notified in time, process will occur
Dogfood failures may end up having the similar process. Smoke test failures is higher priority.
If a patch is made for an issue the following day and a new failure occurs, the tested build is patched with the new patch and a new build is kicked off and tested with the dogfood test suite.

backouts/patching needs to occur faster
- Gecko often may have a regression that cannot necessarily be backed out easily
  - This is the main reason why there are delays in a dogfood build beyond any new bug failures.
more often than not, there can be multiple dogfood and smoke test failures that occur.
Close monitoring of smoke test failures and pushing of backouts has not occurred since Bhavana (release manager) left