Fennec/NativeUI/BetaReadinessReview

[Fennec Native] Beta Readiness Review

May 14, 2012

Beta Release Plan

Friday- May 11

[Kevin] analyze the Beta Sign-Off test matrix for an other couple hours so he is 150% sure
[Aki] 13 beta 2 XUL build is kicked off for tablets
[QA] Once build is available QA verifies 13 beta 2 XUL for tablets. No new regressions
[Erin, Alex] Draft of release notes available for review
[Erin] Status and timeline released

Monday - May 14

AM
√ [QA] Complete 13 beta 2 XUL for tablet verification and sign-off √ [Alex] Formal in-person beta release sign-off confirming all x-functional teams have tasks complete [All] Submit (optional) blog posts for review to communications@mozilla.com

PM
√ [Alex] Final release notes on staging
√ [Jaclyn] All marketing web site changes on staging
√ [Michelle] All support materials complete

Tuesday- May 15

AM
[Alex] Go LiveGoogle Play and Mobile Beta Website (target 9AM Pacific)

Weekly Beta Release Schedule

[05/18] Beta 2
[05/25] Beta 3
[06/01] Beta 4
[06/08] Beta 5
[06/15] Beta 6
[06/22] Beta 7
[06/29] GA, LAST DAY OF Q2

• Go-to build every Tuesday, at the latest on Wed afternoons
  
• Policy: Priority is frequency. However, we will block on serious issues reported by customers Google Play & regressions on the tree
  

Marketing Distribution Plan

Websites being updated/localized/launched on May 15th

• hero image update
• Google Play Store content update
• FAQ for versions and l10n plan

List text (general location is fine) that is being updated and localized)

• Bug 755411 - "About Firefox Beta" Release Notes link is 404
  
• Bug 756908 - Link to FAQ on About Firefox Beta page points to a missing page
  

Other Promos

• Facebook ads for messaging testing
• Beta newsletter
• Beta channel Snippets
• Core Mozillians communication
• Reddit IamA campaign

PR Intro Plan

• 5/15 Introduction date
  
• "Future of Firefox" blog post
  
• Friendly reporters have been given a head's up
  
• FAQ for inbound press questions
  

Support Plan

• Release notes
  
• Minimum System Requirements & Device Compatibility
• https://www.mozilla.org/en-US/mobile/platforms/ Updated Platforms page targeted for a beta very soon]
• en-us support docs complete for Google Play launch; l10n ready
• New Firefox 14 for Mobile Support Site; 60 new articles and new tutorials
• firefoxforandroid@mozilla.com: one-time push notification about Flash
• army of awesome will monitor social sites and direct problem solving on sumo;
• Keep feedback flowing into Google Play for best support; monitoring throughout the day
• Daily reports for the first week or so, depending upon traffic:

-Positives
-Negatives
-Rating change
-Interesting comments/trending

Issues and Risks

• en-us only build for beta 1
  
• Web site compatibility effort is ongoing
  
• Crash rate (although trending in the direction)
  
• GFX work is ongoing
• Add-On support for beta: bug 744555, need add-on developer adoption for compatibility with new architecture

Deferred Bugs Scheduled for Beta 2

• We will continue to approve until the cut-off for go-to build
• [https://bugzilla.mozilla.org/buglist.cgi?quicksearch=blocking-fennec1.0%3AbetaN%2B;list_id=3083812 tracking issues we want fixed early in the beta cycle with a specific flag
• Other notable bugs in Known Issues

Features Summary

• Performance
• click-2-play Flash support
• Readability
• Refined UI, better tab management,smoother in-content UI

Test Coverage

1. Test cases based on each functional area 100% complete
2. This includes verification of fixed blockers across a variety of devices, smoke testing, usability, and exploratory testing
3. Installation, update, and migration testing 100% complete
4. Google Play pre-testing 100% complete

Reliability

1. As of May 14th, we are at 22 crashes per 100 ADU
2. Criteria initially defined was no more than 10 crashes per 100 ADU
3. We have not met that goal 100%, however we are trending downward and are continuing to systematically fix top crashes and reproducible crashes

Performance

Goals

• Fast start up time
• UI is up and running and ready for user interaction quickly
• Responsive browsing: page load, panning & zooming performance
• Better Memory Usage

Performance State

• think we are within acceptable beta quality see state of Fennec Perf Section for more details
• Latest numbers highlighted in weekly status

l10n

• Multi-locale builds not supported in beta 1 bug 753488
• Multi-locale builds have been turned on and are being qualified
• Spacing for multi-locale languages are too small in portrait mode bug 753880
• Committed to adding this support as early on in the beta cycle as possible

Mobile Test Drivers

• Number of testers: 500
• Number of bugs filed to date 144 (this not include bugs filed outside of the MTD form which auto-magically tags them for tracking).
• Areas of focus: Web compatibility, panning and zooming, flash,
• Builds released: 2 specific aurora builds with gfx changes we wanted extra validation on. Otherwise, the updater provides a new build every day.
• We are *keeping* MTD testers on Aurora

Web Compatibility

• Investigation is ongoing (crowd sourcing test tool launched 20120508), actively working with select partners to fix site issues.
• We are working on top sites. Goals are aligned around k9o. Coordinating with evangelism as needed top priority issues.
• High level goals for k9o are captured in bug 747123. Are targeting active involvement with top apps/sites and mitigation strategies for Alexa top sites in specific locales.
• Directed testing email was sent to Nightly testers on 05/14 to emphasize web compatibility testing and provided them with tips on reporting issues.