From MozillaWiki
Jump to: navigation, search

[Fennec Native] Beta Readiness Review

May 14, 2012

Beta Release Plan

Friday- May 11

[Kevin] analyze the Beta Sign-Off test matrix for an other couple hours so he is 150% sure
[Aki] 13 beta 2 XUL build is kicked off for tablets
[QA] Once build is available QA verifies 13 beta 2 XUL for tablets. No new regressions
[Erin, Alex] Draft of release notes available for review
[Erin] Status and timeline released

Monday - May 14

√ [QA] Complete 13 beta 2 XUL for tablet verification and sign-off √ [Alex] Formal in-person beta release sign-off confirming all x-functional teams have tasks complete [All] Submit (optional) blog posts for review to communications@mozilla.com

√ [Alex] Final release notes on staging
√ [Jaclyn] All marketing web site changes on staging
√ [Michelle] All support materials complete

Tuesday- May 15

[Alex] Go LiveGoogle Play and Mobile Beta Website (target 9AM Pacific)

Weekly Beta Release Schedule

[05/18] Beta 2
[05/25] Beta 3
[06/01] Beta 4
[06/08] Beta 5
[06/15] Beta 6
[06/22] Beta 7
[06/29] GA, LAST DAY OF Q2

  • Go-to build every Tuesday, at the latest on Wed afternoons
  • Policy: Priority is frequency. However, we will block on serious issues reported by customers Google Play & regressions on the tree

Marketing Distribution Plan

Websites being updated/localized/launched on May 15th

List text (general location is fine) that is being updated and localized)

  • Bug 755411 - "About Firefox Beta" Release Notes link is 404
  • Bug 756908 - Link to FAQ on About Firefox Beta page points to a missing page

Other Promos

  • Facebook ads for messaging testing
  • Beta newsletter
  • Beta channel Snippets
  • Core Mozillians communication
  • Reddit IamA campaign

PR Intro Plan

  • 5/15 Introduction date
  • "Future of Firefox" blog post
  • Friendly reporters have been given a head's up
  • FAQ for inbound press questions

Support Plan

-Rating change
-Interesting comments/trending

Issues and Risks

  • en-us only build for beta 1
  • Web site compatibility effort is ongoing
  • Crash rate (although trending in the direction)
  • GFX work is ongoing
  • Add-On support for beta: bug 744555, need add-on developer adoption for compatibility with new architecture

Deferred Bugs Scheduled for Beta 2

Features Summary

  • Performance
  • click-2-play Flash support
  • Readability
  • Refined UI, better tab management,smoother in-content UI

Test Coverage

  1. Test cases based on each functional area 100% complete
  2. This includes verification of fixed blockers across a variety of devices, smoke testing, usability, and exploratory testing
  3. Installation, update, and migration testing 100% complete
  4. Google Play pre-testing 100% complete


  1. As of May 14th, we are at 22 crashes per 100 ADU
  2. Criteria initially defined was no more than 10 crashes per 100 ADU
  3. We have not met that goal 100%, however we are trending downward and are continuing to systematically fix top crashes and reproducible crashes



  • Fast start up time
  • UI is up and running and ready for user interaction quickly
  • Responsive browsing: page load, panning & zooming performance
  • Better Memory Usage

Performance State


  • Multi-locale builds not supported in beta 1 bug 753488
  • Multi-locale builds have been turned on and are being qualified
  • Spacing for multi-locale languages are too small in portrait mode bug 753880
  • Committed to adding this support as early on in the beta cycle as possible

Mobile Test Drivers

  • Number of testers: 500
  • Number of bugs filed to date 144 (this not include bugs filed outside of the MTD form which auto-magically tags them for tracking).
  • Areas of focus: Web compatibility, panning and zooming, flash,
  • Builds released: 2 specific aurora builds with gfx changes we wanted extra validation on. Otherwise, the updater provides a new build every day.
  • We are *keeping* MTD testers on Aurora

Web Compatibility

  • Investigation is ongoing (crowd sourcing test tool launched 20120508), actively working with select partners to fix site issues.
  • We are working on top sites. Goals are aligned around k9o. Coordinating with evangelism as needed top priority issues.
  • High level goals for k9o are captured in bug 747123. Are targeting active involvement with top apps/sites and mitigation strategies for Alexa top sites in specific locales.
  • Directed testing email was sent to Nightly testers on 05/14 to emphasize web compatibility testing and provided them with tips on reporting issues.