1,623
edits
Support/StartPageOptimization/FirstTest: Difference between revisions
Support/StartPageOptimization/FirstTest (view source)
Revision as of 15:17, 25 February 2009
, 25 February 2009no edit summary
(New page: This page describes a simple test to determine if the high bounce rate on the SUMO inproduct start page compared to the normal start page is because of the article links under the search b...) |
No edit summary |
||
| Line 2: | Line 2: | ||
=Overview= | =Overview= | ||
People visiting Firefox Support (SUMO) from Firefox itself (aka from "inproduct") are more likely to just leave the website without interacting with it, compared to people visiting the site from www.mozilla.com. | |||
Another reason why we're seeing this difference could be because the two groups are actually ending up on different start pages. The inproduct start page lists simple tutorial and references, in an assumption that people visiting help from inproduct are more likely to just want to learn more about the browser, and not actually solve a problem. | |||
This test aims to determine whether that assumption is true or not. | |||
==Two Start Pages== | ==Two Start Pages== | ||
SUMO | There are two main ways to reach the SUMO website: | ||
===Inproduct Start Page=== | ===Inproduct Start Page=== | ||
First, there’s a support option within the Firefox browser itself (i.e., inproduct). Users clicking on "Help" from the Firefox menu are taken to the [http://support.mozilla.com/en-US/kb/Firefox+Help inproduct start page]. | First, there’s a support option within the Firefox browser itself (i.e., inproduct). Users clicking on "Help" from the Firefox menu are taken to the [http://support.mozilla.com/en-US/kb/Firefox+Help inproduct start page]. | ||
| Line 12: | Line 17: | ||
Below the search box, this start page lists references and tutorials that are originally coming from Firefox 2. This is based on the assumption that people using help from the product itself are more interested in learning about the program rather than solving a problem. | Below the search box, this start page lists references and tutorials that are originally coming from Firefox 2. This is based on the assumption that people using help from the product itself are more interested in learning about the program rather than solving a problem. | ||
This start page has an average bounce rate of 85%. | This start page has an average bounce rate of 85%. Among the people who stay on the page, 71% search. | ||
===Normal Start Page=== | ===Normal Start Page=== | ||
| Line 21: | Line 26: | ||
Below the search box, this start page lists the most popular support articles, and a blurb for new users, listing articles useful for new users. | Below the search box, this start page lists the most popular support articles, and a blurb for new users, listing articles useful for new users. | ||
This start page has a bounce rate of 53%. | This start page has a bounce rate of 53%. Among the people who stay on the page, 50% search. | ||
=The Test= | |||
'''Are people accessing help from inproduct really more interested in reading references and tutorials about Firefox, or are they actually trying to solve a support problem?''' | |||
That is the basic question this test tries to answer. Since a larger percentage of the visitors interact with the site on the normal start page (which features lists of the most popular support articles), this test will expose a group of people visiting SUMO from inproduct to a special start page with the article lists copied from the normal start page. | |||
[[Image:Sumo-a-b-test.png]] | |||
We will limit the test group to the en-US locale and steer e.g. 50% of the visitors of the inproduct start page to this special hybrid start page. | |||
The generic URL request from Firefox is http://support.mozilla.com/1/%PRODUCT%/%VERSION%/%PLATFORM%/%LOCALE%/firefox-help (the current .htaccess rules can be found [http://svn.mozilla.org/projects/sumo/trunk/webroot/htaccess.dist here]). | |||
=Measuring Results= | |||
The key metrics to compare in this test are: | |||
* bounce rate -- if the hybrid start page has a lower bounce rate, that should mean that the normal start page is more in line with what visitors of SUMO expect to see | |||
* search ratio -- if the number of people searching (rather than interacting/clicking on links) drops on the hybrid start page, that should mean that the provided article links are more useful | |||