|
|
| Line 51: |
Line 51: |
|
| |
|
| == Timeline == | | == Timeline == |
| == Rapid Betas ==
| |
| The following documents the changes made to the above processes for adopting more frequent Beta builds. ''As of 2013-06-25 (Firefox 23) we are adopting twice-weekly Betas''.
| |
|
| |
| === Timeline ===
| |
| {| class="querytable" style="width:100%"
| |
| | class="header" style=:width:10%" | Week
| |
| | class="header" style=:width:18%" | Monday
| |
| | class="header" style=:width:18%" | Tuesday
| |
| | class="header" style=:width:18%" | Wednesday
| |
| | class="header" style=:width:18%" | Thursday
| |
| | class="header" style=:width:18%" | Friday
| |
| |-
| |
| | 1
| |
| | Beta migration
| |
| | Beta 1 built
| |
| |
| |
| | Beta 1 QA sign-off
| |
| |
| |
| |-
| |
| | 2
| |
| | Beta 2 built
| |
| | Beta 2 QA sign-off
| |
| |
| |
| | Beta 3 built
| |
| | Beta 3 QA sign-off
| |
| |-
| |
| | 3
| |
| | Beta 4 built
| |
| | Beta 4 QA sign-off
| |
| |
| |
| | Beta 5 built
| |
| | Beta 5 QA sign-off
| |
| |-
| |
| | 4
| |
| | Beta 6 built
| |
| | Beta 6 QA sign-off
| |
| |
| |
| | Beta 7 built
| |
| | Beta 7 QA sign-off
| |
| |-
| |
| | 5
| |
| | Beta 8 built
| |
| | Beta 8 QA sign-off
| |
| |
| |
| | Beta 9 built
| |
| | Beta 9 QA sign-off
| |
| |-
| |
| | 6
| |
| | Beta 10 built
| |
| |
| |
| | Beta 10 QA sign-off <br> Release migration <br> RC go-to-build
| |
| |
| |
| |
| |
| |-
| |
| | 7
| |
| | RC QA pre-sign-off <br> Mirror push
| |
| | RC QA sign-off <br> Live push <br> Live QA sign-off
| |
| |
| |
| |
| |
| |
| |
| |}
| |
|
| |
| === Sign-off Criteria ===
| |
| ==== Beta 1 ====
| |
| * Features signed off by feature owners as "beta ready"
| |
| * Functional automation uncovers 0 regressions, include 10 locales spread across all platforms
| |
| * >80% Critical bug fixes verified (as identified by QA lead)
| |
| * Manual regression tests in Moztrap uncovers 0 blocking regressions
| |
| ** Tests: Breakpad, Mac OSX 10.8 code-signing, NVidia 3D Vision, and Rel-noted features smoketests
| |
| ** Platforms: Windows 7 32-bit, Windows 8 64-bit, Mac OSX 10.8, Ubuntu Linux 32-bit
| |
| * Update automation uncovers 0 failures, include 4 previous versions and 10 locales spread across all platforms
| |
| ** betatest and releasetest channels tested before sign-off
| |
| ** beta channel tested after Release Engineering pushes the snippets
| |
| * Stability (Marcia/KaiRo) and Support (Matt_G/Tyler) uncovers no red flags from early adopters
| |
|
| |
| ==== Beta 2-9 ====
| |
| * Functional automation uncovers 0 regressions, include 10 locales spread across all platforms
| |
| * Update automation uncovers 0 failures, include 4 previous versions and 10 locales spread across all platforms
| |
| ** betatest and releasetest channels tested before sign-off
| |
| ** beta channel tested after Release Engineering pushes the snippets
| |
| * Manual testing to occur daily outside of the sign-off process:
| |
| ** Week 2
| |
| *** Tests: Breakpad, MD5 hash algorithm, and Plug-in compatibility exploratory smoketests
| |
| *** Platforms: Windows 7 64-bit, Windows 8 32-bit, Mac OSX 10.7, Ubuntu Linux 64-bit
| |
| ** Week 3
| |
| *** Tests: Breakpad, Mac OSX 32-bit mode, and Web compatibility exploratory smoketests
| |
| *** Platforms: Windows Vista, Windows 7 32-bit, Mac OSX 10.6, Ubuntu Linux 32-bit
| |
| ** Week 4
| |
| *** Tests: Breakpad, Software update for root-owned files, and Add-on compatibility exploratory smoketests
| |
| *** Platforms: Windows XP, Windows 7 64-bit, Mac OSX 10.8, Ubuntu Linux 32-bit
| |
| ** Week 5
| |
| *** Tests: Breakpad, What's New page for software update, and Plug-in compatibility exploratory smoketests
| |
| *** Platforms: Windows 7 32-bit, Windows 8 64-bit, Mac OSX 10.7, Ubuntu Linux 32-bit
| |
| * Bug fix verifications to occur daily outside of the sign-off process, supported by triage on Wednesdays & Fridays
| |
| * Testdays should be organized during Week 2 and Week 5 incorporating bug triage, regression testing, and feature testing
| |
| * Stability (Marcia/KaiRo) and Support (Matt_G/Tyler) consulted weekly to uncover any red flags from early adopters
| |
|
| |
| ==== Beta 10 ====
| |
| * Functional automation uncovers 0 regressions, include 10 locales spread across all platforms
| |
| * >90% critical bug fixes verified (as identified by QA lead)
| |
| * Manual regression tests in Moztrap uncovers 0 blocking regressions
| |
| ** Tests: Breakpad, NVidia 3D Vision, Mac OSX 10.5 & Windows 2000 de-support, Web compatibility exploratory smoketests
| |
| ** Platforms: Windows 7 64-bit, Windows 8 32-bit, Mac OSX 10.6, Ubuntu Linux 64-bit
| |
| * Update automation uncovers 0 failures, include 4 previous versions and 10 locales spread across all platforms
| |
| ** betatest and releasetest channels tested before sign-off
| |
| ** beta channel tested after Release Engineering pushes the snippets
| |
| * Stability (Marcia/KaiRo) and Support (Matt_G/Tyler) uncovers no red flags from early adopters
| |
|
| |
| ==== Release Candidate ====
| |
| * Features signed off by feature owners as "release ready"
| |
| * Functional automation uncovers 0 regressions, include 10 locales spread across all platforms
| |
| * All critical bug fixes verified (as identified by QA lead)
| |
| * Manual regression tests in Moztrap uncovers 0 blocking regressions
| |
| ** Tests: Breakpad, Partner repack update, and Windows share compatibility smoketests
| |
| ** Platforms: Windows XP, Windows 7 64-bit, Mac OSX 10.8, Ubuntu Linux 32-bit
| |
| * Update automation uncovers 0 failures, include 4 previous versions and 10 locales spread across all platforms
| |
| ** betatest tested before conditional sign-off
| |
| ** releasetest tested aftet Release Engineering pushes snippets to mirrors
| |
| ** release channel tested after Release Engineering pushes snippets to release
| |
| * Stability (Marcia/KaiRo) and Support (Matt_G/Tyler) uncovers no red flags from early adopters
| |
| * '''''Important''''': Due to an intermittent build-time stability issue related to Radeon drivers we need to do twin builds
| |
| ** Once build1 is available run functional and betatest update automation
| |
| ** Once automation is complete, send an email to Release Engineering to generate build2
| |
| ** Once build2 is available run functional and betatest update automation on it
| |
| ** build1 will be pushed to releasetest on Monday if it's not found to have the stability issue
| |
| ** build2 will be pushed to releasetest on Monday if build1 is is found to have the stability
| |
| ** All manual testing and bug fix verifications should still be tested against build1
| |
|
| |
| = Nightly = | | = Nightly = |
| Firefox starts on mozilla-central, this is where the initial code is landed to deliver new features to users. Every six weeks the current Nightly version is uplifted to the Aurora branch then Nightly becomes the next Firefox version. This "uplift" always happens on a Monday, except when people are unavailable due to statutory holidays. | | Firefox starts on mozilla-central, this is where the initial code is landed to deliver new features to users. Every six weeks the current Nightly version is uplifted to the Aurora branch then Nightly becomes the next Firefox version. This "uplift" always happens on a Monday, except when people are unavailable due to statutory holidays. |