Fennec/NativeUI/BetaReadinessReview: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(91 intermediate revisions by 4 users not shown)
Line 1: Line 1:
== [Fennec Native] Beta Readiness Review ==
== [Fennec Native] Beta Readiness Review ==


== Needed by: May 10-11, 2012 ==  
== May 14, 2012 ==  


=== Beta Release Plan ===
==== Friday- May 11 ====
<strike> [Kevin] analyze the Beta Sign-Off test matrix for an other couple hours so he is 150% sure </strike> <br>
<strike> [Aki] 13 beta 2 XUL build is kicked off for tablets </strike> <br>
<strike> [QA] Once build is available QA verifies 13 beta 2 XUL for tablets. No new regressions </strike> <br>
<strike> [Erin, Alex] Draft of release notes available for review </strike> <br>
<strike> [Erin] Status and timeline released </strike> <br>


==== Monday - May 14 ====
AM<br>
√ [QA] [https://wiki.mozilla.org/QA/Fennec/Milestones/13.0/Beta/Beta_2/Build_2 Complete 13 beta 2 XUL for tablet verification and sign-off]
√ [Alex] [https://wiki.mozilla.org/Releases/Fennec_Native_14_Beta/Final_Signoffs Formal in-person beta release sign-off confirming all x-functional teams have tasks complete]
[All] Submit (optional) blog posts for review to communications@mozilla.com<br>


=== [Erin] Blocking Bug Metrics ===
PM<br>
Find/Fix rate
√ [Alex] Final release notes on staging<br>
Blocking Bugs
√ [Jaclyn] All marketing web site changes on staging<br>
Bounce-Back Rate
√ [Michelle] All support materials complete<br>


=== [Erin] Deferred Bug Metrics ===  
==== Tuesday- May 15 ====  
List the most worrisome bugs that are being deferred.  List critical deferred bugs that may have Support impact, or which should be considered for the next beta. Explain why each bug isn't being or can't be fixed.
AM<br>
[Alex] Go LiveGoogle Play and Mobile Beta Website (target 9AM Pacific)


=== [Erin] Features Summary ===  
=== Weekly Beta Release Schedule ===  
Summarize new features that ship with this release


=== [Kevin] Test Coverage ===
[05/18] Beta 2<br>
#Summarize the test coverage status for each functional area
[05/25] Beta 3<br>
#Please note % level of completion of these items (the goal is assumed to be 100%)
[06/01] Beta 4<br>
#Note the number of days required to satisfy the goal of 100% completion. If the goal is not 100% then state the goal
[06/08] Beta 5<br>
[06/15] Beta 6<br>
[06/22] Beta 7<br>
[06/29] GA, LAST DAY OF Q2


=== [Naoki]Reliability ===
*Go-to build every Tuesday, at the latest on Wed afternoons <br>
#List the reliability criteria defined for the project
*Policy: Priority is frequency. However, we will block on serious issues reported by customers Google Play & regressions on the tree <br>
#State results of reliability testing (measured against the criteria)


=== [Clint, Wlach, Geoff, Erin] Performance ===  
=== Marketing Distribution Plan ===  
#List the performance criteria defined for the project
#State results of performance testing (measured against the criteria)


=== [Lukas] Mobile Test Drivers & Usability Testing ===  
==== Websites being updated/localized/launched on May 15th ====
#Number of testers
*[https://www.mozilla.org/mobile/beta/ hero image update]
#Number of bugs reported
*[https://play.google.com/store/apps/details?id=org.mozilla.firefox&hl=en Google Play Store content update]
#Validation (Area of focus and results)
*FAQ for versions and l10n plan
#Activity (Builds seeded and results)
List text (general location is fine) that is being updated and localized)<br>
* Bug 755411 - "About Firefox Beta" Release Notes link is 404<br>
* Bug 756908 - Link to FAQ on About Firefox Beta page points to a missing page<br>


=== [AaronMT, Lawrence] Web Compatibility ===  
==== Other Promos ====
#Where we are today
*Facebook ads for messaging testing
Investigation is ongoing (crowd sourcing test tool launched this week), actively working with select partners to fix site issues.
*Beta newsletter
#Where we want to be when we ship
*Beta channel Snippets
We are working on top apps and sites. Goals are aligned around k9o. Haven't received specific guidance from product about Fennec 1.0 launch.
*Core Mozillians communication
#If goals can't be obtained when we ship in June, provide high level for k9o
*Reddit IamA campaign
High level goals for k9o are captured in {{bug|747123}}. Are targeting active involvement with top apps/sites and mitigation strategies for Alexa top sites in specific locales.


=== [Erin, Alex] Release Plan ===  
=== PR Intro Plan ===  
#l10 details <== Erin
*5/15 Introduction date <br>
#Google Play Details<br>
*"Future of Firefox" blog post <br>
#Minimum System Requirements<br> <== Erin
*Friendly reporters have been given a head's up <br>
*FAQ for inbound press questions <br>


=== [Erin, Eng, QA] Schedule ===  
=== Support Plan ===  
#Weekly betas
*[https://www-dev.allizom.org/en-US/mobile/14.0beta/releasenotes/ Release notes]<br>
#Go-to build dates
*[https://wiki.mozilla.org/Mobile/Platforms/Android#System_Requirements Minimum System Requirements & Device Compatibility]
#Google Play release dates
*https://www.mozilla.org/en-US/mobile/platforms/ Updated Platforms page targeted for a beta very soon]
#We have flexibility until about June 5th
*en-us support docs complete for Google Play launch; l10n ready
*[http://support.mozilla.org/en-US/mobile#os=android&browser=m14 New Firefox 14 for Mobile Support Site; 60 new articles and new tutorials]
*firefoxforandroid@mozilla.com: one-time push notification about Flash
*army of awesome will monitor social sites and direct problem solving on sumo;
*Keep feedback flowing into Google Play for best support; monitoring throughout the day
*Daily reports for the first week or so, depending upon traffic:
-Positives<br>
-Negatives <br>
-Rating change <br>
-Interesting comments/trending <br>


=== [Jaclyn] Marketing Distribution Plan ===  
=== Issues and Risks ===  
#List websites being updated/localized/launched for this beta release
*en-us only build for beta 1<br>
#List text (general location is fine) that is being updated and localized)<br
*Web site compatibility effort is ongoing<br>
#What are the go-live dates for the above
*Crash rate (although trending in the direction)<br>
#List any bugs associated with the above
*GFX work is ongoing
*Add-On support for beta: {{Bug|744555}}, need add-on developer adoption for compatibility with new architecture


=== [Erica, Barry] PR & Marketing Intro Plan ===  
=== Deferred Bugs Scheduled for Beta 2 ===  
#[EJ]Introduction date, means by which the product will be introduced (press release), notable events, any blog or benchmark requirements from Dev/engineering and due dates
*[https://bugzilla.mozilla.org/buglist.cgi?quicksearch=717096%2C725389%2C736123%2C748531%2C752426%2C752444%2C752688%2C753534%2C753845%2C754276%2C725389%2C736123%2C739648%2C748531%2C750846%2C751479%2C752052%2C752444%2C752688%2C753534%2C753845%2C754276%2C738676%2C746860%2C747231%2C747857%2C751262%2C752492%2C752514;list_id=3083671 We will continue to approve until the cut-off for go-to build]
#[Barry] Any brand marketing activities/website updates planned and dates associated with them
*[https://bugzilla.mozilla.org/buglist.cgi?quicksearch=blocking-fennec1.0%3AbetaN%2B;list_id=3083812 tracking issues we want fixed early in the beta cycle with a specific flag
*[https://www-dev.allizom.org/en-US/mobile/14.0beta/releasenotes Other notable bugs in Known Issues]


=== [mluna, Erin, Alex] Support Plan ===  
=== Features Summary ===  
#Release notes [alex] https://www.mozilla.org/en-US/mobile/13.0beta/releasenotes/ <br>
*Performance
#[mluna] SUMO <br>  <== mluna, I believe you and I are meeting about this tomorrow. :)
*click-2-play Flash support
*Readability
*Refined UI, better tab management,smoother in-content UI


=== [All] Issues and Risks ===  
=== Test Coverage ===
• State any outstanding issues and/or risks
#Test cases based on each functional area 100% complete
#This includes verification of fixed blockers across a variety of devices, smoke testing, usability, and exploratory testing
#Installation, update, and migration testing 100% complete
#Google Play pre-testing 100% complete
 
=== Reliability ===
#[https://crash-stats.mozilla.com/products/FennecAndroid/versions/14.0a2?duration=3 As of May 14th, we are at 22 crashes per 100 ADU]
#Criteria initially defined was no more than 10 crashes per 100 ADU
#We have not met that goal 100%, however we are trending downward and are continuing to systematically fix top crashes and reproducible crashes
 
=== Performance ===  
====Goals====
*Fast start up time
*UI is up and running and ready for user interaction quickly
*Responsive browsing: page load, panning & zooming performance
*Better Memory Usage
====Performance State====
*[http://people.mozilla.org/~johnath/fnWe think we are within acceptable beta quality see state of Fennec Perf Section for more details]
*Latest numbers highlighted in weekly status
 
=== l10n ===
*Multi-locale builds not supported in beta 1 {{bug|753488}}
*Multi-locale builds have been turned on and are being qualified
*Spacing for multi-locale languages are too small in portrait mode {{bug|753880}}
*Committed to adding this support as early on in the beta cycle as possible
 
=== Mobile Test Drivers ===
*Number of testers: 500
*Number of bugs filed to date 144 (this not include bugs filed outside of the MTD form which auto-magically tags them for tracking).
*Areas of focus: Web compatibility, panning and zooming, flash,
*Builds released: 2 specific aurora builds with gfx changes we wanted extra validation on. Otherwise, the updater provides a new build every day.
*We are *keeping* MTD testers on Aurora
 
=== Web Compatibility ===
*Investigation is ongoing (crowd sourcing test tool launched 20120508), actively working with select partners to fix site issues.
*We are working on top sites. Goals are aligned around k9o. Coordinating with evangelism as needed top priority issues.
*High level goals for k9o are captured in {{bug|747123}}. Are targeting active involvement with top apps/sites and mitigation strategies for Alexa top sites in specific locales.
*Directed testing email was sent to Nightly testers on 05/14 to emphasize web compatibility testing and provided them with tips on reporting issues.

Latest revision as of 18:49, 23 May 2012

[Fennec Native] Beta Readiness Review

May 14, 2012

Beta Release Plan

Friday- May 11

[Kevin] analyze the Beta Sign-Off test matrix for an other couple hours so he is 150% sure
[Aki] 13 beta 2 XUL build is kicked off for tablets
[QA] Once build is available QA verifies 13 beta 2 XUL for tablets. No new regressions
[Erin, Alex] Draft of release notes available for review
[Erin] Status and timeline released

Monday - May 14

AM
√ [QA] Complete 13 beta 2 XUL for tablet verification and sign-off √ [Alex] Formal in-person beta release sign-off confirming all x-functional teams have tasks complete [All] Submit (optional) blog posts for review to communications@mozilla.com

PM
√ [Alex] Final release notes on staging
√ [Jaclyn] All marketing web site changes on staging
√ [Michelle] All support materials complete

Tuesday- May 15

AM
[Alex] Go LiveGoogle Play and Mobile Beta Website (target 9AM Pacific)

Weekly Beta Release Schedule

[05/18] Beta 2
[05/25] Beta 3
[06/01] Beta 4
[06/08] Beta 5
[06/15] Beta 6
[06/22] Beta 7
[06/29] GA, LAST DAY OF Q2

  • Go-to build every Tuesday, at the latest on Wed afternoons
  • Policy: Priority is frequency. However, we will block on serious issues reported by customers Google Play & regressions on the tree

Marketing Distribution Plan

Websites being updated/localized/launched on May 15th

List text (general location is fine) that is being updated and localized)

  • Bug 755411 - "About Firefox Beta" Release Notes link is 404
  • Bug 756908 - Link to FAQ on About Firefox Beta page points to a missing page

Other Promos

  • Facebook ads for messaging testing
  • Beta newsletter
  • Beta channel Snippets
  • Core Mozillians communication
  • Reddit IamA campaign

PR Intro Plan

  • 5/15 Introduction date
  • "Future of Firefox" blog post
  • Friendly reporters have been given a head's up
  • FAQ for inbound press questions

Support Plan

-Positives
-Negatives
-Rating change
-Interesting comments/trending

Issues and Risks

  • en-us only build for beta 1
  • Web site compatibility effort is ongoing
  • Crash rate (although trending in the direction)
  • GFX work is ongoing
  • Add-On support for beta: bug 744555, need add-on developer adoption for compatibility with new architecture

Deferred Bugs Scheduled for Beta 2

Features Summary

  • Performance
  • click-2-play Flash support
  • Readability
  • Refined UI, better tab management,smoother in-content UI

Test Coverage

  1. Test cases based on each functional area 100% complete
  2. This includes verification of fixed blockers across a variety of devices, smoke testing, usability, and exploratory testing
  3. Installation, update, and migration testing 100% complete
  4. Google Play pre-testing 100% complete

Reliability

  1. As of May 14th, we are at 22 crashes per 100 ADU
  2. Criteria initially defined was no more than 10 crashes per 100 ADU
  3. We have not met that goal 100%, however we are trending downward and are continuing to systematically fix top crashes and reproducible crashes

Performance

Goals

  • Fast start up time
  • UI is up and running and ready for user interaction quickly
  • Responsive browsing: page load, panning & zooming performance
  • Better Memory Usage

Performance State

l10n

  • Multi-locale builds not supported in beta 1 bug 753488
  • Multi-locale builds have been turned on and are being qualified
  • Spacing for multi-locale languages are too small in portrait mode bug 753880
  • Committed to adding this support as early on in the beta cycle as possible

Mobile Test Drivers

  • Number of testers: 500
  • Number of bugs filed to date 144 (this not include bugs filed outside of the MTD form which auto-magically tags them for tracking).
  • Areas of focus: Web compatibility, panning and zooming, flash,
  • Builds released: 2 specific aurora builds with gfx changes we wanted extra validation on. Otherwise, the updater provides a new build every day.
  • We are *keeping* MTD testers on Aurora

Web Compatibility

  • Investigation is ongoing (crowd sourcing test tool launched 20120508), actively working with select partners to fix site issues.
  • We are working on top sites. Goals are aligned around k9o. Coordinating with evangelism as needed top priority issues.
  • High level goals for k9o are captured in bug 747123. Are targeting active involvement with top apps/sites and mitigation strategies for Alexa top sites in specific locales.
  • Directed testing email was sent to Nightly testers on 05/14 to emphasize web compatibility testing and provided them with tips on reporting issues.
Retrieved from "https://wiki.mozilla.org/index.php?title=Fennec/NativeUI/BetaReadinessReview&oldid=433875"