Release Management/Release Process Checklist Documentation

The goal of this page is to document the Release Process Checklist being used by Firefox Release Managers to track each release throughout the cycle. Any changes to this documentation or the checklist should be reflected in both documents.

Nightly Checklist

Given the nature of how nightly builds are created and shipped, the role of the release manager during this phase of the cycle skews much more heavily to the monitoring aspect rather than release mechanics.

Prior to the start of the Nightly cycle, the following tasks need to be performed before Merge Day

1. Create a Release Checklist before the cycle starts.
   • Use the template from Google Drive.
2. Create the Milestones Document for the Release Checklist.
   • Use the sheet linked in the Release Checklist.
3. Create a Firefox Retrospective Notes document.
   • Use the template from Google drive.
   • It is useful to document during the release cycle.
4. Create a folder in the Release Tracking Google Drive for the release.
   • Move the Release Checklist and Firefox Retrospective Notes to this folder.
5. Add a Release Tracking Sheet link to Release Owners wiki
   • Add the link to the date in the Release Date column.
6. Ensure that Bugzilla is bumped to the Nightly version
   • Note: This is performed on the Friday before Merge Day.
   1. Update the Release Milestones in Bugzilla
   2. Go to Bugzilla > Administration > New Firefox Release
      • The products and milesontes are automatically completed
   3. Click [ Submit ]
   4. Update the Release Tracking flags in Bugzilla
   5. Go to Bugzilla > Administration > Release Tracking Flags
   6. Add the cf_tracking_firefox and cf_status_firefox for the Nightly release
      • Example: If 97 is becoming the Nightly, then add cf_tracking_firefox97 and cf_status_firefox97.
   7. Deactivate the oldest cf_tracking_firefox and cf_status_firefox flag.
      • Example: If 95 is going to Release, 96 is going to Beta, and 97 is becoming the Nightly, then deactivate cf_tracking_firefox94 and cf_status_firefox94.
   8. Edit the current cf_tracking_firefox_relnote tracking flag to add the new release as a Value and deactivate the oldest release.
      • Example: If 95 is going to Release, 96 is going to Beta, and 97 is becoming the Nightly, then edit cf_tracking_firefox_relnote to add a Value 97+ set to mozilla-next-drivers and deactivate the Value 94+
   9. Edit the current cf_tracking_firefox_esr tracking flag to add the new release as a Value and deactivate the oldest release.
      • Example: If esr is based on 91, 95 is going to Release, 96 is going to Beta, and 97 is becoming the Nightly, then edit cf_tracking_firefox_esr91 to add a Value 97+ set to mozilla-next-drivers and deactivate the Value 94+
   10. Add the cf_tracking_thunderbird and cf_status_thunderbird for the Nightly release.
       • Example: If 97 is becoming the Nightly, then add cf_tracking_thunderbird_97 and cf_status_thunderbird_97
   11. Deactivate the oldest cf_tracking_thunderbird and cf_status_thunderbird flag.
       • Example: If 95 is going to Release, 96 is going to Beta, and 97 is becoming the Nightly, then deactivate cf_tracking_thunderbird_94 and cf_status_thunderbird_94.
   12. Edit the current cf_tracking_thunderbird_esr tracking flag to add the new release as a Value and deactivate the oldest release.
       • Example: If esr is based on 91, 95 is going to Release, 96 is going to Beta, and 97 is becoming the Nightly, then edit cf_tracking_firefox_esr91 to add a Value 97+ set to mozilla-next-drivers and deactivate the Value 94+.
   13. Edit the cf_tracking_fxios tracking flag to add the new release as a Value and deactivate the oldest release.
7. Create a release notes skeleton in Nucleus.
   1. Click Admin Interface
   2. Click the Releases.
   3. Select the previous Nightly release notes (enable the checkbox beside the release).
   4. At the top of the page, expand the Action dropdown
   5. Select Copy releases.
   6. Click Go.
   7. Select the copy.
   8. In the Version text box edit the version to the current Nightly version.
      • Example: 99.0a1
   9. Set the Release Date to match when Central was bumped to Nightly
   10. Keep the nightly-only notes and remove the release notes from the previous release.
   11. For additional information on the Release Notes process see The Firefox release notes process wiki page.
8. Update the bugs with nightly+ release note tracking.
   1. Remove nightly+ from any bugs older than 3 releases.
      • Add a comment as a reminder to add a release note request when the feature rides the train.
   2. Remove nightly+ from any bugs that are riding the train.
      • If the same bug is tracking the final release note then set the appropriate release in the relnote track
      • If a different bug is tracking the final release note then add a comment mentioning which bug is tracking the release note.
   3. See The Firefox release notes process wiki page for additional information.
9. Update the Google calendar Releases Scheduling.
   1. Open the following URL and change the version to the Nightly release.
      • https://whattrainisitnow.com/calendar/release/schedule/?version=118.
   2. Download the generated ical file.
   3. Review locally on a calendar app of your choice
   4. Open Google Calendar.
   5. Click + beside Other Calendars.
   6. Select Import.
   7. Select the ical file previously downloaded.
   8. Select the Releases Scheduling calendar.
   9. Click Import.
   10. Please Note: Importing the ical to Google Calendar is a one-shot operation. If you notice incorrect info or dates after importing, correct them directly in Google Calendar. Importing the ical file again will cause duplication.
10. Ensure the REO (Regression Engineering Owner) is listed on Release Owners.
    • If required check the Google Sheet REO Rotation by Director.
    • Update the wiki with information from the spreadsheet.
      • Ensure that the REO and RelEng owners are listed.
      • Ensure the dates are accurate.
    • If required reach out to the Director to find out who the REO is.
    • Ensure they are invited to the Weekly Regression Triage.
11. Ensure the Release Engineer on Release Duty is listed on Release Owners.
12. Ensure the Release Owners wiki is up to date
    • The REO and Release Engineer on Release Duty should be listed

At the start of the Nightly cycle, the following tasks need to be performed on Merge Day

1. Verify that Central was bumped to the Nightly version.
   • Check version_display.txt
   • Please Note: This is performed on Merge Day of the release going to beta, if it is incorrect talk to the RelMan on duty for the Beta cycle.
2. Verify that the firefox-android version was updated.
   • Chheck version.txt should be xxx.0a1, where xxx matches the nightly version of desktop.
   • If the version was not updated, then sync with release engineering.
3. Make the release notes public in Nucleus
4. Verify the application services version was updated
   • The version in version.txt should be xxx.0a1, where xxx matches the nightly version of desktop.
   • If the version was not updated then talk to the RelMan on duty for the Beta cycle.
5. Verify the firefox_versions.json is correct.
   • Please Note: Usually this json update is performed the day after merge day
   • Please Note: fx trains and various scripts in other tools use this data.
   • Verify the following are correct:
     • FIREFOX_NIGHTLY, NEXT_MERGE_DATE
   • If these are incorrect, contact the RelEng on duty via the #release-duty channel in Matrix. It is possible a PR was missed during the RelEng merge day steps.
   • Please Note: There are also date changes that are monitored by autonag. For example, if Template:NextReleaseDate was not updated an autonag email will be sent.
6. Verify the mobile_versions.json is correct
   • Verify the following are correct:
     • alpha_version, nightly_version
   • If these are incorrect, contact the RelEng on duty via the #release-duty channel in Matrix.

On a daily (or thereabouts) basis, the following items should be monitored during the Nightly cycle

1. Review Pending tracking-firefoxXX requests
   • Example Filter
   • Ensure that the tickets are progressing, follow up as required.
2. Review Open tracking-firefoxXX+ and blocking bugs
   • Example Filter
   • Ensure that the tickets are progressing, follow up as required.
3. Review incoming regression bugs
   • See the REO section in the BugDash
   • Ensure that new tickets are prioritized.
   • Ensure that carry-over tickets are still on the team’s radar.
   • Review with the REO in the weekly regression meeting.
4. Review stability rates and reported crash spikes
   • Leverage information from the automated emails, monitor top crashes in Crash Stats, and monitor Stability & Release Monitoring Looker Dashboard
   • Ensure that there are bugs tracking stability rates and crash spikes.
   • Ensure these tickets are tracked and prioritized.
   • Please Note: If ever there are stats/bugs that cannot be explained/identified, then reach out to the team for assistance on the #stability Matrix channel, it may be an issue with the tooling issue.
5. Review the Release Notes requests for Nightly Tickets.
   • See Release_Notes#Daily_during_the_Nightly_cycle for details.

On a weekly (or thereabouts) basis, the following items should be monitored during the Nightly cycle

1. Review the tracking Mana page for the release.
   • The tracking page can be found as a child of Firefox Release Tracking
   • Ensure that features targeting the release are on track, reach out as needed if any are slipping.
2. Update the Weekly Release Tracking Digest document, as requested on #fx-weekly-digest.
   • Update the Release Milestones (Details) section to add Status (On Track, Delayed), Ship Date, and additional details depending on where the nightly is in the cycle.
   • Review the dates in the Release Milestones section and update the status.

As available, the following items should be monitored during the Nightly cycle

1. Review and land mozilla-central repo-update HSTS HPKP remote-settings tld-suffixes patches.
   • Every Monday and Thursday revisions are automatically created for review in Phabricator.
   • Monitor email notifications sent by Phabricator: “No Bug, mozilla-central repo-update HSTS HPKP remote-settings tld-suffixes”
   • Review and land the patch to autoland.
2. Review and land android nightly application-services version update bumps to main
   • An automated nightly application-services build is triggered daily. The build will only run if commits merged to main since the previous nightly build.
   • update-bot creates a patch to bump the version for android in mozilla-central, runs a Try build, and sets the release-mangers group as reviewer.
   • Monitor email notifications sent by Phabricator: Update android nightly application-services version bump
   • Review and land the patch to autoland.
   • If the Try build failed:
     • Check the failure.
     • Check the possible commit in application-services main that corresponds to the failure.
     • Start a thread on #application-services-eng for investigation.
   • If the application-services bump no longer applies cleanly
     • The patch sometimes fails because application-services was bumped on autoland but not yet merged to main, so your new PR becomes stale once main catches up and your patch no longer applies cleanly.
     • You can re-run ./mach vendor .../moz.yaml to bump it to the latest. It regenerates dependent metadata, and commit a two-file change.
     • example: https://github.com/mozilla-firefox/firefox/commit/642afeeca47f
3. Review the Release Notes requests for Nightly tickets
   • These are nominated by setting the relnote flag to ? for a particular release
4. Review Test Plans for features targeting the release. QA shares test plans via email. For example:
   • Become familiar with how the feature works.
   • Understand how the team ensures it is of sufficient quality to ship in that release.
   • Provide feedback on the risk analysis and mitigations put in place.
5. Review mid-Nightly and pre-Beta testing reports QA shares mid-Nightly and pre-Beta testing reports via email. For example:
   • If there are any red or yellow reports review the recommendations/plan and the bugs mentioned in the report.
   • If QA flagged that the feature is behind a pref verify that this is expected.

At the end of the Nightly cycle, the following tasks need to be performed prior to and during Merge Day

1. Prepare the Beta release notes in Nucleus
   1. Click Admin Interface
   2. Click the Releases
   3. Select the Nightly release notes (enable the checkbox beside the release)
   4. At the top of the page, expand the Action dropdown
   5. Select Copy release
   6. Click Go
   7. In the Channel dropdown select Beta
   8. In the Version text box edit the version to the Beta version
      • Example: 96.0beta
   9. Replace the content in the Text box with content from a previous beta release
   10. Set the Release Date to the date when the first Beta is released
   11. Ensure to remove any notes that are only applicable to nightly
2. Sync with RelEng to confirm the RelEng on duty for the Central to Beta merge.
   • Sync on Friday before Merge Day
   • Use the #releaseduty channel on Matrix.
   • Align they are available for support if needed.
3. Create checklist(s) for the new ESR cycle
   • Create a checklist for the relevant ESR that corresponds to the Firefox mainline release
   • See ESR calendar
4. Verify the automated mozilla-central->mozilla-beta no-op trial run
   • The job automatically runs every Friday and the link is shared on the #release-duty channel in Matrix.
   • See #do-migration-no-op-trial-runs for documentation.
5. Create a Desktop Beta Checklist and Mobile Beta Checklist for beta 1
   • Use the b1 macro

Beta Checklist

Once a release moves to the Beta cycle, the daily tasks performed during the Nightly cycle will continue to be carried out as new bug reports come in from a wider audience and new features move through the QA cycle towards shipping. There is also an added triage step during Beta - monitoring the “missed uplifts” email or queries to find issues fixed in Nightly that still affect Beta.

The following tasks need to be performed on Merge Day at the start of the Beta cycle

1. Sync with the Sheriff team to confirm Central is in a good state so that the Merge Day activities can begin.
   • Sync on Merge Day morning
   • Use the #sheriffs channel on Matrix.
   • Check the Beta simulations document for the release, this document is emailed to Release Management at the start of the night cycle.
   • Ensure that central does not currently have any known severe regressions or performance issues that would be carried into beta.
2. Submit a Main -> Beta merge automation with Dry Run disabled in Ship-it
   • 
3. Start the Main -> Beta merge automation
4. Verify the changesets are visible on the beta branch and Treeherder
   • It may take up to 45m before the changeset are on Treeherder
   • beta should get a version bump and branding changes
   • There will be two new tags. One tag should be in the form FIREFOX_BETA_xxx_END - where xxx is the major Gecko version that Beta had prior to the merge. The other tag should be in the form FIREFOX_BETA_yyy_BASE - where yyy is the major Gecko version that Beta now has.
5. Submit a Bump main merge automation with Dry Run disabled in Ship-it
   • 
6. Start the Bump main merge automation
7. Verify the changesets are visible on the firefox-main branch and Treeherder
   • It may take up to 45m before the changeset are on Treeherder
   • firefox-main should get a version bump and a new tag in the form FIREFOX_NIGHTLY_xxx_END - where xxx is the major Gecko version that firefox-main had prior to the version bump.
8. Perform the version bump steps in the ESR checklists
9. Update the wiki versions
   • NEXT_VERSION, CENTRAL_VERSION, BETA_VERSION, RELEASE_VERSION, and Next release date
10. Send the merge complete email to release-signoff and release-drivers
    • Example email
11. application-services: Create an application-services release branch off of the main branch named release-vXXX
12. application-services: The following steps can be performed by running as-merge-day.py in your fork of the application-services repo:
    • application-services: Update version.txt and the CHANGELOG.md in the release-vXXX branch
      • In version.txt, update the version from XXX.0a1 to XXX.0.
      • In CHANGELOG.md, change In progress to _YYYY-MM-DD_ to match the Merge Day date and add a URL to the release version change log
      • Create a commit named 'Cut release vXXX.0`
      • Create a pull request and merge to the release branch, see example PR
    • application-services: Update the application-services version from XXX to YYY and CHANGELOG.md in the main branch
      • In version.txt, bump the version from XXX.0a1 to YYY.0a1
      • In CHANGELOG.md, bump the in progress version from XXX.0 to YYY.0, add a header for the previous release version, and add a URL to the previous release version change log
      • Create a commit named 'Start release vYYY.0`
      • Create a pull request and merge to the main branch, see example PR
13. application-services: Create a new Application Services release in Ship-It
14. application-services: Start the Application Services build in Ship-It via Promote
15. application-services: Ship the Application Services build in Ship-It
16. application-services: Tag the release in the Application Services repo
    1. See example tag
17. application-services: Notify the application-services dev team on #application-services-eng that the release branch is ready, the version bumped was merged, and the release is ready
18. Switch to Application Services in Firefox Android to Release
    • Change the Version and Channel in mobile/android/android-components/plugins/dependencies/src/main/java/ApplicationServices.kt
      • See example commit
    • Push directly mozilla-beta
19. Review any pending uplifts that may be required before proceeding with beta 1 build 1
    • This should not be a common occurrence, however, an example would be a crash fix that may have landed in central post-merge.
20. Create Desktop and DevEdition build 1 in Ship-It on the initial merge commit
    • This build will not be shipped and is only to test the build pipeline after the merge
    1. Connect to the VPN
    2. Log into Ship-It
    3. Click Releases dropdown
    4. Click Firefox dropdown
    5. Click New
       • 
    6. For Desktop complete as follows and click Create Release:
       • Please note: Revision should be the tip. The Partial updates field is automatic.
       • 
    7. For DevEdition complete as follows and click Create Release
       • Please note: Revision should be the tip. The Partial updates field is automatic.
       • 
21. Create Firefox Android (AC, Fenix, Focus) build 1 release in Ship-It
    • This build will not be shipped and is only to test the build pipeline after the merge
22. Start the Desktop and DevEdition in Ship-it via the Promote Task
    1. View the Pending Releases in Ship-It
    2. Click the Promote Task for Desktop and continue through the pop-ups
    3. Click the Promote Task for DevEditon and continue through the pop-ups
23. Start Firefox Android (AC, Fenix, Focus) release in Ship-It via Promote
    • Please note: you can only start a ship-it build on a revision that has a completed decision task
24. Confirm the Desktop and DevEdition builds have started
    • Taskcluster email: “[desktop] Build of firefox xxx.0b1 build 1”
    • Taskcluster email: “[desktop] Build of devedition xxx.0b1 build 1”
25. Confirm the Firefox Android (AC, Fenix, Focus) builds have started
26. Confirm notification sent when the Desktop and DevEdition builds finish
    • Taskcluster email: “firefox xxx.0b1 build1/mozilla-beta is in the candidates directory”
    • Taskcluster email: “devedition xxx.0b1 build1/mozilla-beta is in the candidates directory”
27. Confirm notification sent when the Firefox Android (AC, Fenix, Focus) builds finish
    • Taskcluster email: “firefox-android XXX.0b# build1/mozilla-beta is in the candidates directory"
28. Cancel Desktop and DevEdition build 1 in Ship-It
29. Cancel Firefox Android (AC, Fenix, Focus) build 1 in Ship-It

The following tasks need to be performed the day after Merge Day

1. Review tracking-firefoxXXX+ bugs and approval requests
2. Create Desktop and DevEdition build 2 release in Ship-It
3. Create Firefox Android (AC, Fenix, Focus) build 2 release in Ship-It
4. Start Desktop and DevEdition builds in Ship-It via Promote Task
5. Start Firefox Android (AC, Fenix, Focus) build in Ship-It via Promote
6. Confirm the Desktop and DevEdition builds have started
7. Confirm the Firefox Android (AC, Fenix, Focus) builds have started
8. Confirm notification sent when the Firefox Android (AC, Fenix, Focus) builds finish
9. Push Firefox Android (AC, Fenix, Focus) release in Ship-It via Push
10. Confirm notification sent when the Firefox Android (AC, Fenix, Focus) push finishes
    • Taskcluster email: “Focus/Fenix XXXb#-build2 has been pushed to the closed testing track on Google Play"
11. Ship Firefox Android (AC, Fenix, Focus) from Ship-It via Ship
12. Promote Fenix from Closed Testing to Production with a 99% rollout and submit for review
13. Promote Focus from Closed Testing to the Foxfooding track with a 99% rollout and submit for review
14. Confirm notification sent when the Desktop and DevEdition builds finish
15. Schedule push to Desktop and DevEdition CDN from Ship-It via Push
    1. Click Push
       • 
    2. Click Schedule
       • 
16. Confirm notification sent when the Desktop and DevEditon CDN push finishes
    • Taskcluster email: “firefox xxx.0b1 build1/mozilla-beta has been pushed to cdntest”
    • Taskcluster email: “devedition xxx.0b1 build1/mozilla-beta has been pushed to cdntest”
17. Confirm notification sent when the Desktop and DevEdition CDN push finishes
18. Ship Desktop and DevEdition to Beta from Ship-It via Ship
19. Activate automated Desktop, DevEdition, and Firefox Android betas in Ship-It
    1. At the bottom of the page, click the red x beside Firefox Desktop: Beta
       • 
    2. Click Enable
       • 
    3. Perform the same actions for Firefox Developer Edition: Beta and Firefox Android (Android-Components, Fenix, Focus): Beta
    • Please Note: Automated Betas are built at 13:00 UTC every Monday, Wednesday, and Friday as determined here.
20. Make release notes live in Nucleus
    • Enable the Is Public checkbox and save
      • 
21. Verify that the Desktop and DevEdition Balrog rule changes are live and correctly set
    • View the Balrog Rules for Firefox Beta
      • Verify that the Background rate for Firefox-xxx.0b1-build2 is 50
    • View the Balrog Rules for Aurora (DevEdition)
      • Verify that the Background rate for Devedition-xxx.0b1-build2 is 100
22. Verify that FIREFOX_DEVEDITION & LATEST_FIREFOX_DEVEL_VERSION in firefox_versions.json are correct
    • If incorrect, contact the RelEng on duty via the #release-duty channel in Matrix.
23. Verify the beta_version in mobile_versions.json is correct
    • If incorrect, contact the RelEng on duty via the #release-duty channel in Matrix.
24. Update https://whattrainisitnow.com/release-notes to point to the latest release notes doc
25. Share a link to the Beta release notes and the release notes draft document in #release-notes Slack channel
    • See example
26. Email release-notes with a link to the draft document and the submission deadline.
    • See example

Monitor for QA sign-offs on beta 1 build 2

QA sign-offs are usually provided the day after build 2 is ready

1. Monitor for QA update tests on Aurora & Beta
   • QA will post a message to the #qa-coordination channel in Slack when they complete update testing.
2. Monitor for QA manual testing signoff for Desktop and DevEdition
   • QA will post a message to the #qa-coordination channel in Slack when they complete functional testing.
3. Monitor for QA sign-off on Fenix/Focus build validation
   • Fenix build validation testing is tracked here
   • Focus build validation testing is tracked here
   • Focus and Fenix QA sign-off are sent via #qa-coordination Slack Channel
4. Bump the Beta update rate to 100% in Balrog
5. Update the Fenix Production track rollout to 100%
6. Update the Focus Foxfooding track rollout to 100%

The following tasks need to be performed daily during the Beta cycle

1. Monitor Uplift Requests
   • Example Filter and Phabricator Dashboard
   • Accept or reject the beta uplift requests.
   • See Uplift Rules for guidelines
   • Please Note: During the RC week, also monitor release uplift requests.
   • Please Note: When pushing uplifts be conscious of timing around the automated beta builds. Pushing an uplift will trigger CI, the automated beta build will not trigger until the CI completes.
   • Please Note: If an uplift contains string changes, then l10n approval is required and the commit message must end with l10n=approver_name. Add a ni on the bug for approval before taking the uplift.
2. Monitor Pending tracking-firefoxXX requests
   • Example Filter
   • Ensure that the tickets are progressing, and follow up as required.
3. Monitor Open tracking-firefoxXX+ and blocking bugs
   • Example Filter
   • Ensure that the tickets are progressing, and follow up as required.
4. Monitor Newly-filed regression bugs
   • New regressions bugs can be found here
   • Review with the REO - ensure that new tickets are prioritized and ensure that carry-over tickets are still on the team’s radar
5. Monitor stability rates and reported crash spikes
   • Leverage the automated emails, monitor top crashes in crash-stats, and monitor Stability & Release Monitoring
   • Ensure that there are tickets tracking issues with stability rates and crash spikes
6. Monitor the Release Notes requests for Beta Tickets.
   • Example Filter
   • Check through tickets in the release that are tagged for release notes. Review and add these notes to the Beta Release Notes in Nucleus.

The following tasks need to be performed for each beta build (Monday, Wednesday, Friday) during the Beta cycle

Please Note: The Beta build pipeline is also monitored by the Sheriffs, if there is an intermittent test failure they will rerun the failed job. If there is a permanent failure or an issue in the build infrastructure, then it may require canceling the build. Discuss in the #releaseduty Matrix channel. If the decision is made to cancel, then view the pending release in ship-it and click cancel. To create a new build, follow the same steps as outlined in One Time - Start of Beta Cycle to produce build 2 of the current beta release.

1. Create a tab in the release tracking sheet “xxx.0bx”
   • Use the relevant beta macro
   • Please note: After all the beta steps are complete, the tab can be hidden in order to minimize clutter.
2. Review tracking-firefoxXX+ bugs and approval requests.
   • This step is performed as part of the daily activities, tracked in the checklist for confirmation.
3. Confirm the Desktop and DevEdition builds have started.
   • Taskcluster email: “[desktop] Build of firefox xx.0bx build 1”
   • Taskcluster email: “[desktop] Build of devedition xx.0bx build 1”
   • Please Note: The automated beta builds schedule is determined here.
4. Confirm the Firefox Android (AC, Fenix, Focus) builds have started
   • Taskcluster email: “Build of firefox-android xx.0bx build 1"
5. Confirm notification sent when the Firefox Android (AC, Fenix, Focus) builds finish
   • Taskcluster email: “firefox-android xx.0bx build1/mozilla-beta is in the candidates directory"
6. Confirm notification sent when the Firefox Android (AC, Fenix, Focus) push finishes
   • Focus/Fenix XXXb#-build1 has been pushed to the closed testing track on Google Play
7. Promote Fenix from Closed Testing to Production with a 99% rollout and submit for review
8. Promote Focus from Closed Testing to the Foxfooding track with a 99% rollout and submit for review
9. Confirm notification sent when the Desktop and DevEdition builds finish.
   • Taskcluster email: “firefox xx.0bx build1/mozilla-beta is in the candidates directory”
   • Taskcluster email: “devedition xx.0bx build1/mozilla-beta is in the candidates directory”
10. Confirm notification sent when the Desktop and DevEditon CDN push finishes.
    • Taskcluster email: “firefox xx.0bx build1/mozilla-beta has been pushed to cdntest”
    • Taskcluster email: “devedition xx.0bx build1/mozilla-beta has been pushed to cdntest”
11. Verify that the Desktop and DevEdition Balrog rule changes are live and correctly set.
    • View the Balrog Rules for Firefox Beta
      • Verify that the Background rate for Firefox-xx.0bx-build1 is 100%
    • View the Balrog Rules for Aurora (DevEdition)
      • Verify that the Background rate for Devedition-xx.0b1-buildx is 100.
12. Monitor for QA sign-off on Desktop and DevEdition build validation for both update/functional testing.
    • QA will post a message to the #qa-coordination channel in Slack when they complete update testing and another message when they complete functional testing.
    • Build validation testing is tracked here
13. Monitor for QA sign-off on Fenix/Focus build validation before proceeding with bumping the Fenix/Focus rollout.
    • Focus and Fenix QA sign-off are sent via #qa-coordination Slack Channel.
    • Please Note: Build Validation sign-off is usually provided the day after Fenix/Focus builds are produced. If the sign-off is yellow/red, then follow-up on the #mobile-android-team Slack channel. Ensure a plan is in place or a decision is made to block/proceed with the release.
    • Once QA has signed off, roll out Fenix to the production track in Google Play to 100%.
      • Reply to the QA sign-off Slack message and include the rollout percentage.
    • Once QA has signed off, roll out Focus to the Closed testing - Foxfooding track in Google Play to 100%.
      • Reply to the QA sign-off Slack message and include the rollout percentage.

As available, the following items should be monitored during the Beta cycle

1. QA shares beta testing and pre-release testing reports via email. Review the test reports for the release. For example:
   • If there are any red or yellow reports, review the recommendations/plan and the bugs mentioned in the report. For example:
     • Track the bug reported as blockers.
     • If there is no clear plan to address the report, follow up to ensure that engineering is reviewing.
   • If QA mentions the feature is behind a pref verify that this is expected.
2. Phabricator patches for remote-settings updates are generated automatically. Typically this is every Monday and Thursday. These patches must be included in the beta uplifts.
   • Monitor email notifications sent by Phabricator: “No Bug, mozilla-beta repo-update HSTS HPKP remote-settings tld-suffixes”
   • Review the and land patch.
3. Use the Bugdash to review the Beta Quality Metrics.
   • The counts of these queries should decline as the cycle advances. (See Triage guidelines in the tool tip "?")
   • Take a look at the metric/charts over time to ensure the value is stable.
   • The data is presented at the Thursday channel meeting.
4. Update the Weekly Digest document, as requested on the #fx-weekly-digest Slack channel.
   • Update the Release Milestones (Details) section to add Status (On Track, Delayed), Ship Date, information on the current beta release for both Desktop/DeveEdition and Fenix/Focus, information on where we are in the Beta cycle for example when coming up to end of early beta.
   • Review the Release Milestones section dates and update the status.
   • Update the Quality Metrics section to add information on the number of outstanding regressions, if there are any release blockers, and an overview on the trend of the Beta Quality Metrics metrics.

The following needs to be performed at the End of Early Beta

Please Note: This is performed on the Friday at the midway point of the beta cycle. This is typically after beta 5 has been released. This should be done before pushing any other uplifts to beta after beta 5.

1. Submit a Early -> late beta merge automation with Dry Run disabled in Ship-it
2. Start the Early -> late beta merge automation

After the job runs, verify that the commit was pushed to the mozilla-beta repo, see example.

The following needs to be performed at the end of the Beta Cycle

1. Verfiy the automated mozilla-beta->mozilla-release migration no-op trial run
   • The job automatically runs every Friday and the link is shared on the #release-duty channel in Matrix.
   • See #do-migration-no-op-trial-runs for documentation.
2. Disable automated beta builds in Ship-it.
3. Please Note: Monitor beta/release uplift requests between the final beta build and the beta to release merge. It is possible that a late uplift request may come in for a regression/stability fix, evaluate if this is a blocker for the RC and if it should be uplifted to beta before the beta to release merge and RC build.

iOS XXX.0 Checklist

iOS XXX.0 preparation begins on the Friday before RC Week.

1. Check if there are any open PRs targettimg main for string imports or updates for effective_tld_names, initial experiments, AS, or Glean
   • Ideally, check the day before branching. If there are any pending PRs reach out on #firefox-ios-dev in Slack.
   • This query will help
2. The following steps can be performed by running ios-merge-day.py in your checkout of the firefox-ios repo:
   1. Create a firefox-ios release branch with the format release/vxxx off of the main branch
      • Create a branch name with the format release/vxxx off of the main branch through the GitHub UI
   2. Update the version in version.txt from XXX.0 to XXX.1 in the main branch
      • Edit version.txt directly in GitHub and commit as "Bump - Set version to XXX.1"
3. Notify the firefox-ios dev team on #firefox-ios-releases that the release branch is ready and the version bumped was merged
4. Create a Firefox iOS release in ship-it
   • Select Product: Firefox iOS, Repository: Firefox iOS, Branch: release/vXXX.0, and the latest revision
5. Start the Firefox iOS build in Ship-It via Promote
6. Push the Firefox iOS release in Ship-It via Push
   • This step pushes the Firefox iOS build to Testflight
7. Ensure the firefox-ios build is available in Testflight
   • Firefox Beta TestFlight
8. Create a Focus iOS release in ship-it
   • Note: Focus iOS XXX.0 may not ship; the purpose is to at least verify the Focus iOS pipeline once per cycle
   • Select Product: Focus iOS, Repository: Focus iOS, Branch: release/vXXX.0, and the latest revision
9. Start the Focus iOS build in Ship-It via Promote
10. Push the Focus iOS release in Ship-It via Push
    • This step pushes the Focus iOS build to Testflight
11. Ensure the focus-ios build is available in Testflight
12. Monitor for QA sign-off on Firefox iOS build validation
    • Build validation testing is tracked here
    • Preliminary sign-off is due to be sent by Monday EOD, which is sufficient for pushing to External Testers in TestFlight
13. Once QA has signed off, add the External Beta Testers group to the Firefox iOS TestFlight Build
    • Reply to the QA sign-off Slack message.
14. QA will aim to send a full manual testing report by EOD Tuesday so there is time to address any blocking issues identified.
15. Check if there are any pending backport PRs targeting the XXX.0 release branch
    • Example query https://github.com/mozilla-mobile/firefox-ios/pulls?q=is%3Apr+base%3Arelease%2Fv144.0+is%3Aopen+
    • After the branch was created iOS Engineers may have created backport PRs. Review the uplift request comment on the PR and merge when appropriate.
16. If an XXX.0 RC2 build is required, then cancel the previous RC build in Ship-It and repeat steps 3 to 6 and 11 to 12
17. Check if there are any pending backport PRs targeting the XXX.0 release branch
    • Example query https://github.com/mozilla-mobile/firefox-ios/pulls?q=is%3Apr+base%3Arelease%2Fv144.0+is%3Aopen+
    • After the XXX.0 RC2 was built iOS Engineers may have created backport PRs. Review the uplift request comment on the PR to determine if an XXX.0 RC3 build is required, or if the PR can wait for XXX.1
18. If an XXX.0 RC3 build is required, then cancel the previous RC build in Ship-It and repeat steps 3 to 6 and 11 to 12
19. Check if there are release notes
    • The deadline for release notes is Tuesday during the stabilization week. Product will provide release notes and let the Release Management team know.
    • If there are no release notes provided, then use the generic release notes
20. Submit Firefox iOS in the AppStore for review
    • Select a phased rollout when submitting.
    • Select to automatically release after App Store Review on Sunday night Eastern 10PM (Monday 2AM GMT) prior to go-live (i.e. go-live date -2)
      • Note: If the Monday of go-live week is a Canadian public holiday, then set the automatic release date to Monday night Eastern instead.
21. Ship Firefox iOS from Ship-It via Ship
    • Wait until the Friday of RC week to avoid shipping if another RC is needed
22. Verify that Firefox iOS rolled out
23. Send Slack message to #firefox-ios-releases to notify the release is live

RC Checklist

Once a release moves to the Release cycle, preparations for the release candidates begin. Until go-live, the daily tasks are similar to the beta cycle in terms of monitoring and if needed create a new release candidate.

The following tasks need to be performed at the start of RC week (week before go-live) for Desktop/Android

1. Review tracking-firefoxXXX+ bugs and approval requests
   • Verify that all approved uplift requests were uplifted to beta
2. Submit a Beta -> release merge automation with Dry Run disabled in Ship-it
3. Start the Beta -> release merge automation
4. Verify the changesets are visible on the release branch and Treeherder
   • It may take up to 45m before the changesets are on Treeherder
   • release should get a version bump and callsign changes
   • There will be two new tags. One tag should be in the form FIREFOX_BETA_xxx_END - FIREFOX_RELEASE_xxx_END - where the xxx is the major Gecko version that Release had prior to the merge. The other tag should be in the form FIREFOX_RELEASE_yyy_BASE - where the yyy is the major Gecko version that Release now has.
5. Send the merge complete email to release-drivers
6. Example email
7. Once the merge is complete, verify that the Treeherder tests green/starred
8. Set up Desktop build in Ship-It.
   1. Connect to the VPN.
   2. Log into Ship-It.
   3. Click Releases > Firefox > New.
      • 

        NewReleaseShipit
   4. Complete as follows and click Create Release:
      • 
      • Please note: Revision should be the tip. The Partial updates field is automatic.
   5. Click Submit.
      • 
9. Set up Firefox Android (AC, Fenix, Focus) release in Ship-It.
   • Follow the same steps as creating a beta build.
10. Start the Desktop build from Ship-It via Promote RC.
    • 
    • Click the Schedule button.
      • 
11. Start Firefox Android (AC, Fenix, Focus) release in Ship-It via Promote.
12. Confirm Desktop build has started.
    • Taskcluster email: “[desktop] Build of firefox xx.0 build 1”
13. Confirm the Firefox Android (AC, Fenix, Focus) builds have started
14. Confirm notification sent when Firefox Android builds finish.
15. Push Firefox Android (AC, Fenix, Focus) release in Ship-It via Push.
16. Confirm notification sent when the Firefox Android (AC, Fenix, Focus) push finishes.
17. Turn on managed publishing in the Play Console for Fenix/Focus/Klar.
18. Manually create a Fenix/Focus/Klar release on the Production track with a 5% rollout.
19. Submit the Fenix/Focus/Klar release for Play Store review.
20. Submit the Fenix/Focus release for Samsung Store review with a 5% rollout.
    • Please Note: The submission is automatically created as part of the push.
    • Delete the N-2 version so that only the previous release and RC packages remain.
    • Set to publish manually, the review should complete during RC week.
21. Submit the Fenix release for Huawei AppGallery Open Testing to release immediately (only needed if you need to send it to QA or users)
    • Please Note: The submission is separate from the release track
    • Create a new release (also considered an update) in Huawei App Gallery Connect
    • Make sure to select "Yes" for open testing.
    • In the packages section, Upload the arm64-v8a apk (Huawei only allows one build per submission).
    • 
    • Set to release immediately (only option when selecting open-testing)
22. Submit the Fenix release for Huawei AppGallery on the Release track review with a 5% rollout.
    • Make sure to select "No" for open testing
    • 
    • All other information should be copied over from the previous submission.
    • Set to at 5% with the end date being the Wednesday after the release date, this is when it will rollout to 100% automatically
      • 
23. Confirm notification sent when Desktop builds finish
24. Cancel pending Microsoft Store certification
    • Please Note: The submission is automatically created as part of the RC build pipeline.
25. Verify the Microsoft Store package version and enable manual update checks
    • Verify that correct package was uploaded, the version displayed in the submission matches the release version
    • Manual update checks allow users to choose to update
26. Re-submit the pending submission for Microsoft Store certification
    • Please Note: The Microsoft Store review can take several days, it’s recommended to submit for review at the end of RC week. If there is a blocker found later, you can always resubmit.
    1. Navigate to the Microsoft Partner Center
    2. Select Firefox from Windows & Xbox > Overview
    3. Select the pending submission
    4. In Packages enable gradual rollout and set to 25%
       • Ensure to enable the option to provide the newest package when users manually check for updates.
    5. In Submission Options set to Automatically Publish on the go-live date and time
    6. Submit to the store for review
    • Please see App submissions documentation for additional information on the Microsoft Store submission process.

As available, the following should be monitored/performed during RC week for Desktop/Android

1. Phabricator patches for remote-settings updates are generated automatically. Typically this is every Monday and Thursday. These patches must be included in release/esr uplifts.
   • Monitor email notifications sent by Phabricator: “No Bug, mozilla-release repo-update HSTS HPKP remote-settings tld-suffixes”/“No Bug, mozilla-esr91 repo-update HSTS HPKP remote-settings tld-suffixes”
   • Review and land the patch.
2. Update the Weekly Digest document, as requested on #fx-weekly-digest.
   • Update the Release Milestones (Details) section to add Status (On Track, Delayed), Ship Date.
   • Review the Release Milestones section dates and update the status.
3. Verify that QA have signed off manual testing for Desktop
   • QA will post a message to the #qa-coordination Slack channel when they complete update testing and another message when they complete functional testing.
   • Build validation testing is tracked here
4. Verify that QA have signed off update tests on Beta.
   • QA will post a message to the #qa-coordination Slack channel when they complete update testing and another message when they complete functional testing.
5. Verify that QA have signed off update tests on release-localtest
   • QA will post a message to the #qa-coordination Slack channel when they complete update testing and another message when they complete functional testing.
6. Monitor for QA sign-off on Fenix/Focus build validation before proceeding with Steps 7 to 10.
   • Please Note: Build Validation sign-off is usually provided the day after Fenix/Focus builds are produced.
   • Focus and Fenix QA sign-off are sent via #qa-coordination Slack Channel.
7. Turn off managed publishing in the Play Console for Fenix to allow it to roll out to 5% once the app is reviewed
   • See Firefox Fast & Private Browser
   • Reply to the QA sign-off slack message
     • “Thanks, pushed to Production at 5% rollout.”
8. Turn off managed publishing in the Play Console to allow it to roll out to 5% once the app is reviewed and push it to the Open Testing track at 100%
   • See Firefox Focus: No Fuss Browser and Firefox Klar Browser
   • Reply to the QA sign-off slack message
     • “Thanks, pushed to Production at 5% rolled out.
9. Publish Fenix/Focus on the Samsung Store.
   • Please note: It can take up to 2 days for review, so it might still be pending.
10. Publish Fenix to the Huawei App Gallery.
    • You can edit the percentage or release to all users

RC Uplifts

This tab is for tracking bugs that are being tracked for possible uplift to the mozilla-release repository for RC builds. The primary objectives are:

• Track whether there are any drivers for a respin of the RC builds during RC week.
• Assess whether Desktop, Mobile, or both are affected by the issues noted.
• Verify that all drivers have had an explicit decision made.

The following tasks need to be performed during RC week (the week before go-live)

1. Monitor Uplift Requests for Beta/Release
   • For Example
   • Track if there are any drivers for an RC respin during RC week.
   • Assess whether Desktop, Mobile, or both are affected by the issues noted.
   • Verify that all drivers have had an explicit decision made.
2. If a new RC is required, for example, a release blocker was identified, then create a new RC checklist tab and perform the same steps as for the initial RC but with the following exceptions:
   1. Cancel the previous RC build in ship-it
   2. Delete the Microsoft Store submission before creating a new RC build
      • Please Note: The RC build will fail if trying to create an MSIX submission when there is already a pending submission
      • To cancel and delete the pending submission you may first need to change it from publishing automatically to publishing manually
3. Determine if a new RC build impacts Android.
   • If a new Android build is not required, then there is no impact on the Fenix/Focus builds and no action is required.
   • If a new Android build is required, then repeat the RC steps with the following exceptions:
     • Cancel the previous Firefox Android (AC, Focus) build in Ship-It.
     • Cancel the previous Fenix build in Ship-It.
     • If Fenix/Focus/Klar already rolled out - Replace the 5% rollout of Fenix/Focus/Klar in Google Play and the Samsung Store.

Go-Live Checklist

Once RC week is over and the release candidate is ready to go-live, preparations for go-live begin. After go-live, monitoring continues and if needed preparing for potential dot releases.

The following tasks need to be performed prior to go-live

1. Ensure that feedback for release notes was gathered.
   • Monitor the #release-notes Slack channel
   • For more information on the release notes doc process see here
2. Run the new-contributor.py script and add to the release notes
   • For more information on the release notes doc process see here
3. Release notes RelMan peer review.
   • For more information on the release notes doc process see here
4. Incorporate feedback into draft release notes.
   • Add a release in Nucleus for Firefox, Firefox for Android, Firefox for iOS
   • Duplicate the previous release, edit the version, dates, and remove the previous release notes. For example Firefox, Firefox for Android, Firefox for iOS
   • Create release notes entries as outlined in the document shared on the #release-notes Slack channel
   • Please Note: If images are included with the release note then create a PR for Bedrock, example PR
   • Share a link to the staged release notes on #release-notes
     • Example
5. Ensure that Legal signoff on release notes was requested by UX/Product
   • Please Note: This step is not always required, if the release notes are not atypical
6. Review tracking-firefoxXX+ bugs and confirm there are no blocking issues
7. Check for crash spikes with RC builds
8. Schedule push to CDN from Ship-It
   • Click Push
   • 
   • Click Schedule
   • 
9. Ensure the automated email was sent after the push to release-cdntest
   • Taskcluster email: “firefox xx.0 buildx/mozilla-release has been pushed to cdntest.
10. Ensure QA have performed the Update test on release-cdntest
    • QA will post a message to the #qa-coordination channel in Slack when they complete update testing and another message when they complete functional testing.
    • Build validation testing is tracked here
11. Ensure Legal have signed off on release notes
    • Please Note: This is only required based on the outcome of Step 3.
12. Ensure that the security advisories are ready
    • Check with the security team on the #security Slack channel
13. Verify that the RC build candidate is available in the Ubuntu Snap Store

The following tasks need to be performed on go-live day

1. Make release notes live in Nucleus
   • Please Note: There is a 15-30min lag between making the change in Nucleus and the live website picking up this change, so plan to do this 15-30min prior to go-live.
2. Schedule push to release at 25% from Ship-It
3. Bump Fenix rollout rate in the play store to 25%
   • See Firefox Fast & Private Browser
4. Bump Focus/Klar rollout rate in the play store to 25%
   • See Firefox Focus: No Fuss Browser and Firefox Klar Browser
5. Bump Fenix rollout rate in the Samsung Store to 25%
6. Bump Focus rollout rate in the Samsung Store to 25%
7. Ship Firefox Android (AC, Fenix, Focus) from Ship-It via Ship
8. Upload security advisories to GitHub
9. Signoff on scheduled rule change in Balrog
   • Set the rollout percentage to 25%
   • Sync with release management for another team member to signoff on the rule change.
10. Verify that the new release is live on mozilla.org
11. Verify that the release notes are live here
12. Verify that security advisories are live here
13. Verify that the Microsoft Store is published at 25% rollout
14. Ship new Desktop release in Ubuntu Snap Store with a phased rollout starting a 25%
    • See Promote a snap to the stable channel for more information
15. Send Slack message to #release-coordination to notify of the release
    • For example “Firefox Desktop 115.0, ESR 115.0.0, Fenix/Focus/Klar Android 115.0, and Firefox/Focus/Klar iOS 115.0 are live”
    • Include links to the release notes.
16. Verify LATEST_FIREFOX_VERSION in firefox_versions.json
17. Verify that the release is live on Flathub
    • The `release-flatpak-push-firefox` job runs as part of the ship graph and the publication is handled by Flathub.
    • https://flathub.org/api/v1/apps/org.mozilla.firefox contains the current release available on Flathub.
    • It can take up to 120 minutes for the release to publish on Flathub. If the release is not updated then reach out in the #flatpaks Matrix channel.
18. Email release-drivers & release-signoff that updates are live at 25%
    • Example Email
19. Ensure QA have performed the Update tests on Release
    • QA will post a message to the #qa-coordination Slack channel when they complete update testing and another message when they complete functional testing.
    • Build validation testing is tracked here
20. Email announce list
    • Example Email
21. Schedule Desktop update rate to 0% in Balrog after 24 hours.
22. Signoff on scheduled rule change in Balrog.
    • Sync with release management for another team member to signoff on the rule change.

The following tasks need to be performed 24 hours after go-live

1. Verify Desktop update rate at 0% in Balrog
2. Email release-signoff & release-drivers to confirm 0% throttling.
   • Reply to the previous email from the go-live day.
   • Example Email

The following tasks need to be performed 48 hours after go-live

1. Review release crash rates and incoming bugs for new blockers.
2. Bump Desktop update rate to 100% in Balrog.
3. Signoff on scheduled rule change in Balrog.
   • Sync with release management for another team member to signoff on the rule change.
4. Bump the Microsoft Store rollout to 100%.
   • Click Finalize package rollout
5. End the phased rollout in the Ubuntu Snap Store
   • Only if it has not already rolled out to 100%.
6. Currently not supported via the web UI, but can be done via the snapcraft cli. See close the release for more information.
7. Review fenix/focus release crash rates, incoming bugs for new blockers, and spike/trend in negative play store reviews.
   • Check to see if there is anything concerning that would prevent completing the rollout.
8. Bump fenix/focus/klar rollout rate in the play store to 100%.
9. Bump fenix/focus rollout rate in the Samsung Store to 100%.
10. Email release-signoff & release-drivers to confirm full rollout.
    • Reply to the previous email from the go-live day.
    • Example Email

The following tasks need to be performed daily after go-live

1. Review release crash rates and incoming bugs for new blockers
2. Review uplift requests for dot release drivers and ride alongs

The following tasks need to be weekly after go-live

1. Update the Weekly Digest document, as requested on #fx-weekly-digest.
   • Update the Release Milestones (Details) section to add status on the release, dot release, etc.

iOS XXX.1 - XXX.3 Checklist

Perform the same steps as iOS XXX.0 Checklist but with the following exceptions:

• Do not build Focus iOS
• If the QA sign-off is still yellow/red by Thursday during the version stabilization week, then that version will not ship.
  • We will "burn" the version number and proceed with the next version branch on Friday as scheduled.

Dot Release Uplifts

Similar to the RC Uplifts tab. The primary purpose of this tab is to track any bugs driving a dot release and bugs that are under consideration to ride-along, It is also used to assess which products are affected by any drivers. It is up to the release manager what should go into a dot release, and we try to keep these uplift guidelines in mind. Adding even what looks like a trivial fix can add risk, both to the process (delay, extra testing, work for other teams) and to causing new regressions. For security issues, we may not want to take them in a dot release unless there is particular pressure to do so.

The following information needs to be added for each bug:

• Bug number
• Component
• Is the bug a dot release driver?
• Does the patch affect Desktop, Mobile, or both?
• Is the fix ready?
• Is the patch on mozilla-central?
• Is there an uplift request on the ticket?
• Is the uplift approved?
• Is the patch uplifted to release?
• Does the bug need a release note?
• Is the patch for a security bug?
• Any additional notes
  • For example, outline why the bug is being taken in a dot release.

Please Note: For each dot release, create a section in the Dot Release Uplifts tab to track the bugs in consideration: For example, “Desktop 101.0.1& Fenix/Focus 101.2.0 - GTB 2022-06-08”

Dot Release Checklist

The following tasks need to be performed at the start of preparing a dot release

Desktop

1. Duplicate the Desktop Dot Release Checklist tab and name it appropriately.
   • Example: Desktop 101.0.1
2. Email release-drivers group to inform them a dot release is being created.
   • Include information on the driver, if there will be an Android release, and a link to the sheet used to track the uplifts.
   • Example Email
3. Review any pending release uplift requests
   • Example Filter
   • If they are low risk consider including them in the dot release
4. Uplift patches to release.
5. If there are security bugs uplifted then sync with the security team member on rotation
   • The security team member on point is whoever is on rotation for the next release version to ship.
   • The security team member on will assign the CVEs and prepare the advisories. This also covers any Android advisories.
6. Create a release in Nucleus to add release notes.
   • Add a release note with a link to the major release for reference.
   • For bugs that have been identified as needing a release note:
     • If a release note is required and the release note text is clear from the bug content then create it directly in Nucleus.
     • Otherwise, request wording before adding the release note.
7. Follow the same process as creating an RC build.
   • If any uplifts impact Android then duplicate the Android Dot Release Checklist tab and name it appropriately.
8. Follow the same process as monitoring when an RC build has QA sign-off

Android

1. Duplicate the Android Dot Release Checklist tab and name it appropriately.
   • Example: Android 101.0.1
2. Follow the same process as creating an RC build with the following exceptions:
   1. Turn on managed publishing in the Play Console.
      • This allows you to submit an app for review but control when it is published.
   2. Manually create a Fenix/Focus/Klar release on the Production track with a 25% rollout.
      • If an app is still pending review on a closed track then you cannot promote it to a production track.
      • See Prepare and roll out a release for more information.
   3. Submit the Fenix/Focus/Klar release for Play Store review.
      • Keep the same release notes as the major release. Add any new notes at the top prefixed by the dot release version.
      • With managed publishing turned on, the app will not automatically release after the store review.
   4. Submit the Fenix/Focus release for Samsung Store review with a 25% rollout and set to publish automatically
      • This allows the review to go through slightly faster
      • Set the publish time to later in the go-live day. This gives enough time to react if the build fails QA
3. Follow the same process as monitoring when an RC build has QA sign-off with the following exceptions:
   1. Turn off managed publishing in the Play Console.
      • With managed publishing turned off, the app will be automatically released after store review.

The following tasks need to be performed during go-live of a dot release

1. Follow the same steps as an RC go-live, with the following exceptions:
   • The dot release is rolled out at 25% but there is no schedule drop to 0%
     • Please Note: Under chemspill situations, a greater rollout percentage may be required.
   • There may not be any security advisories.

The following tasks need to be performed 24 hours after go-live of a dot release

1. Follow the same steps as after an RC go-live, with the following exceptions:
   • If there are no blocking issues then bump the rollout to 100% after 24 hours

The following tasks need to be performed daily after go-live of a dot release

• Follow the same monitoring steps as after an RC go-live.