Confirmed users
4,467
edits
Release Management/Rapid Betas: Difference between revisions
Jump to navigation
Jump to search
no edit summary
No edit summary |
|||
| Line 1: | Line 1: | ||
= Benefits of | = Benefits of Daily Betas = | ||
* shorter verification turn-around time | * shorter verification turn-around time | ||
* freed up RelEng/QA resources | * freed up RelEng/QA resources | ||
| Line 16: | Line 16: | ||
** Complete implementation, begin process testing on Nightly/Aurora | ** Complete implementation, begin process testing on Nightly/Aurora | ||
* Week of 7/17 | * Week of 7/17 | ||
** Push out our first | ** Push out our first daily beta while FF15 on mozilla-beta | ||
= In-Product = | = In-Product = | ||
== Questions == | == Questions == | ||
| Line 31: | Line 31: | ||
** Tracking {{bug|307181}}, {{bug|753103}} | ** Tracking {{bug|307181}}, {{bug|753103}} | ||
** Telemetry bugs (for getting uptime data): {{bug|727184}} - Increase granularity of telemetry uptime measurement, {{bug|733591}} - Send uptime with telemetry data | ** Telemetry bugs (for getting uptime data): {{bug|727184}} - Increase granularity of telemetry uptime measurement, {{bug|733591}} - Send uptime with telemetry data | ||
** Decreasing prompt level since download (if you don't close the browser): {{bug|754470}} - Once we have | ** Decreasing prompt level since download (if you don't close the browser): {{bug|754470}} - Once we have daily betas, we should change the beta channel to prompt users about a downloaded update once a week | ||
=Automation= | =Automation= | ||
| Line 42: | Line 42: | ||
== Questions == | == Questions == | ||
* breakdown of when sign off has uncovered new regressions in a beta? (and if it has, was it through automation or manual (human) tests?) | * breakdown of when sign off has uncovered new regressions in a beta? (and if it has, was it through automation or manual (human) tests?) | ||
** If we find that sign-off has not found new critical beta regressions, can we start shipping betas on Wednesdays after automated testing runs through (prior to | ** If we find that sign-off has not found new critical beta regressions, can we start shipping betas on Wednesdays after automated testing runs through (prior to daily betas being implemented)? Bug verification, exploratory testing, and other smoke tests could continue out-of-band. | ||
* can automating the release test plan be done against nightly/aurora in preparation for moving to faster beta releases? | * can automating the release test plan be done against nightly/aurora in preparation for moving to faster beta releases? | ||
* Can we automate mobile update testing? If so, we can even make the weekly push to the Play Store automatic. | * Can we automate mobile update testing? If so, we can even make the weekly push to the Play Store automatic. | ||
| Line 50: | Line 50: | ||
=Support= | =Support= | ||
== Questions == | == Questions == | ||
* any pain points in supporting Aurora? (expand those into to-do items for supporting | * any pain points in supporting Aurora? (expand those into to-do items for supporting daily betas) | ||
== Blockers == | == Blockers == | ||
= RelEng/IT = | = RelEng/IT = | ||
| Line 61: | Line 61: | ||
** Already is! (except for android) - [[User:Catlee|Catlee]] 12:39, 13 April 2012 (PDT) | ** Already is! (except for android) - [[User:Catlee|Catlee]] 12:39, 13 April 2012 (PDT) | ||
** note: upcoming osx10.8 signing is explicitly excluded | ** note: upcoming osx10.8 signing is explicitly excluded | ||
* Can we do the work associated with | * Can we do the work associated with daily betas be done in such a way that we could move back to weekly betas in the case of unforseen problems? | ||
* Do we need to tag/package source for | * Do we need to tag/package source for daily betas? | ||
== Blockers == | == Blockers == | ||
| Line 76: | Line 76: | ||
= Crash-Kill Team & Soccorro = | = Crash-Kill Team & Soccorro = | ||
== Questions == | == Questions == | ||
* What are the implications of switching to | * What are the implications of switching to daily betas? How would crash analysis be different than m-c or m-a? | ||
* Do we need to implement windowing over multiple build IDs to emulate having a large beta population on a single beta? | * Do we need to implement windowing over multiple build IDs to emulate having a large beta population on a single beta? | ||
* What is a unit of 'enough' user testing data (days?) for getting feedback on a regression fix? | * What is a unit of 'enough' user testing data (days?) for getting feedback on a regression fix? | ||
| Line 88: | Line 88: | ||
== Answers/Resolutions == | == Answers/Resolutions == | ||
* Keep sign-offs on release and beta channel, continue without signoffs on aurora and central | * Keep sign-offs on release and beta channel, continue without signoffs on aurora and central | ||
* Drop the use of milestones at least for | * Drop the use of milestones at least for daily betas | ||
* Get l10n-changesets from urls that give the current signed-off state for a product/version | * Get l10n-changesets from urls that give the current signed-off state for a product/version | ||
** These urls need to updated on migration day every six weeks | ** These urls need to updated on migration day every six weeks | ||
| Line 99: | Line 99: | ||
New questions: | New questions: | ||
* What happens with non- | * What happens with non-daily betas, notably TB/SeaMonkey? '''(No change, continue as usual)''' | ||
* Do we need a "sealed" status for release, or is that good to go without milestones, too? '''(That is up to the needs of the l10n drivers, do you want a tagged repo every 6 weeks when a release goes out?)''' | * Do we need a "sealed" status for release, or is that good to go without milestones, too? '''(That is up to the needs of the l10n drivers, do you want a tagged repo every 6 weeks when a release goes out?)''' | ||