Website/Optimization Roadmap

Background

We are in the process of deploying an A/B and multivariate [1] solution for www.mozilla.com. Multivariate testing allows us to determine which designs and creative elements within a web page have an optimal effect on visitor behvior.

Initially, we'll optimize the Fx download rate (also known as “conversion rate” and determined by the equation downloads/visitors). We plan to extend our optimizations across a variety of Mozilla web properties, including AMO, SUMO, and Personas.

We're looking for experiment ideas from anyone and everyone. Please add your suggestions below.

Resources

• Metrics: Ken Kovash/Blake Cutler
• Marketing: John Slater/Laura Mesa
• Creative: The Royal Order
• Web Dev: Steven Garrity?
• QA: Web QA Team

Goals

• Increase the Firefox download/conversion rate
• Increase percentage of ADU using add-ons
• Increase add-ons installed per user
• Increase AMO developer contributions
• Improve page load speed
• Encourage users to update insecure plugins
• Increase number of new contributors

Testing Schedule

The goal for each quarter is to:

• Run two major page tests each quarter
• Run at least six minor tests per quarter as defined by the Metrics Team.

Q4 2009

• First Run [2]
  • Goals
    • Improve bounce rate
    • Increase Engagement with our product (Add-Ons, Personas)
    • Increase Engagement through Facebook, Twitter, Email lists
    • Learn what Fx users want
  • Project Plan
    • Test three different designs against the newest First Run page [3]
      • Task Oriented
      • Tab Oriented
      • New Concept
  • TRO Timeline
    • Creative Presentation: Thurs, 11.19
    • Mozilla Feedback Due: Mon, 11.23
    • Revisions Post: Tues, 12.01
    • Final Creative Approval: Wed, 12.02
    • Handoff Files: Thurs, 12.03

Q1 2010

• What's New [4]
  • Goals
    • Improve bounce rate
    • Increase Engagement with our product (Add-Ons, Personas)
    • Increase Engagement through Facebook, Twitter, Email lists
    • Learn what Fx users want
• Download [5]
  • Goals
    • Improve conversion rate
    • Learn what Fx users want

Q2 2010

• Getting Started [6]
  • Goals
    • Improve bounce rate
    • Increase Engagement with our product (Add-Ons, Personas)
    • Increase Engagement through Facebook, Twitter, Email lists
    • Learn what Fx users want
• Mobile [7]
  • Goals
    • Improve Download Conversions
    • Increase Engagement with our Mobile products (Add-Ons, Weave)
    • Increase Engagement through our Mobile Facebook, Twitter, Email lists
    • Learn what Fx Mobile users want

Q3 2010

• AMO [8]
• SUMO Home [9]

Experiment Results

We expect to begin experimenting in December 2009.

Name	URL	Test	Result
2009.10.09 Social Media #1	/en-US/firefox/ie.html	Social media links [10]	Design in progress
2009.10.02 Background IE	/en-US/firefox/ie.html	Each background image tested separately	Test in progress
2009.09.24 Download Buton #1	/en-US/firefox/firefox.html	Button copy and subtext	Removing button subtext lowered conversions [11]

Literature & Key Learnings

Resources

• Buttons [12]
• Links [13]
• Case study [14]

SiteSpect Implementation

• AMO and SUMO Implementation [15]. Status: 11/14
• Staging area implementation. Status: 12/01