1,623
edits
Support/Q2HelpfulnessTests: Difference between revisions
Jump to navigation
Jump to search
no edit summary
Michaelverdi (talk | contribs) |
No edit summary |
||
| Line 1: | Line 1: | ||
This quarter we gave ourselves the goal of increasing the average helpfulness score of knowledge base articles by 2%. | This quarter we gave ourselves the goal of increasing the average helpfulness score of knowledge base articles by 2%. There are two basic ways to approach that – make articles more helpful (obviously) and get more people to an article that helps them on the first shot (thereby decreasing the likelihood of rating an article unhelpful). | ||
Our plan is to run [http://en.wikipedia.org/wiki/Multivariate_testing multivariate tests] on the [https://support.mozilla.com/en-US/kb/ support home page] (getting people to the right article) and on 2-3 of the top support articles (making things more helpful). For our tests on KB articles, we want to use some of our most trafficked pages. We also want to choose an article with a high helpfulness rating and one with a low rating. | Our plan is to run [http://en.wikipedia.org/wiki/Multivariate_testing multivariate tests] on the [https://support.mozilla.com/en-US/kb/ support home page] (getting people to the right article) and on 2-3 of the top support articles (making things more helpful). For our tests on KB articles, we want to use some of our most trafficked pages. We also want to choose an article with a high helpfulness rating and one with a low rating. | ||
==Proposed articles to use for the test== | |||
*The [https://support.mozilla.com/en-US/kb/Private+Browsing Private browsing] article is our 2nd most popular article (58763 views, wk of 4/12) and has a helpfulness rating of 85%. | *The [https://support.mozilla.com/en-US/kb/Private+Browsing Private browsing] article is our 2nd most popular article (58763 views, wk of 4/12) and has a helpfulness rating of 85%. | ||
*The [https://support.mozilla.com/en-US/kb/Profiles Profiles] article is our 5th most popular article (24640 views, wk of 4/12) and has a helpfulness rating of 37% | *The [https://support.mozilla.com/en-US/kb/Profiles Profiles] article is our 5th most popular article (24640 views, wk of 4/12) and has a helpfulness rating of 37% | ||
| Line 9: | Line 11: | ||
If we only pick two pages to test we might go with Profiles over Pop-up blocker because it gets almost 40% more traffic and has a similar helpfulness rating. | If we only pick two pages to test we might go with Profiles over Pop-up blocker because it gets almost 40% more traffic and has a similar helpfulness rating. | ||
== Article tests== | |||
=== Changing KB Article language === | |||
=== Embeding screencasts inline === | |||
Screencasts can be very helpful. We'd like to test if embeding them inline will encourage more people to watch them when available. None of the articles we're considering for the tests currently have screencasts so they'll need some created before the test can be run. | |||
== Improving performance == | [[File:KBScreencast.png|700px]] | ||
=== Changing KB Article layout === | |||
Currently, KB articles have an "Actions" box at the top giving people the option to edit or translate an article. While this is nice option that helps to reinforce that the KB is a community effort, it may be a distraction for someone trying to solve a problem. Instead we'd like to test whether swapping the related articles and search sections with the actions box will help people get to the correct article faster. | |||
[[File:KBArticle.png|700px]] | |||
== Start page tests == | |||
=== Improving performance === | |||
Improving page load times should make using and navigating the site easier and therefore help prevent people from becoming frustrated while searching for a solution. After looking at the size and loading times for various elements of the home page, it appears that removing the [https://support.mozilla.com/styles/mozcommon/background-feature.jpg large background image (54k)] at the top of the page will make the biggest impact. The screenshot examples below are very subtle because there is actually a [https://support.mozilla.com/styles/mozcommon/background-tile.png tiled image (4k)] behind the large background image. | Improving page load times should make using and navigating the site easier and therefore help prevent people from becoming frustrated while searching for a solution. After looking at the size and loading times for various elements of the home page, it appears that removing the [https://support.mozilla.com/styles/mozcommon/background-feature.jpg large background image (54k)] at the top of the page will make the biggest impact. The screenshot examples below are very subtle because there is actually a [https://support.mozilla.com/styles/mozcommon/background-tile.png tiled image (4k)] behind the large background image. | ||
| Line 20: | Line 34: | ||
[[File:Sumo-kb-header2.jpg|700px]] | [[File:Sumo-kb-header2.jpg|700px]] | ||
== Simplifying the header and footer == | === Simplifying the header and footer === | ||
Another test we'd like to run is simplifying the page header and footer. This would get rid of a lot of unnecessary information (the links to the rest of Mozilla aren't crucial since the site is about getting help) on each page, making it easier to find what's important. | Another test we'd like to run is simplifying the page header and footer. This would get rid of a lot of unnecessary information (the links to the rest of Mozilla aren't crucial since the site is about getting help) on each page, making it easier to find what's important. | ||
| Line 26: | Line 40: | ||
[[File:Basicpage.png|700px]] | [[File:Basicpage.png|700px]] | ||
== Browsing Categories == | === Browsing Categories === | ||
Another idea that might help get people to the correct article faster would be to offer a better browsing option. Currently the home page is designed around searching with an additional list of popular articles for users who prefer to browse. One idea we'd like to test is to replace that list of popular articles with a few category sections. They could have headings like Crashes, Troubleshooting, or Tutorials with a short description of what other things are included. These would then link to a category page. We might also test whether categories work better above search (mockup on right). | Another idea that might help get people to the correct article faster would be to offer a better browsing option. Currently the home page is designed around searching with an additional list of popular articles for users who prefer to browse. One idea we'd like to test is to replace that list of popular articles with a few category sections. They could have headings like Crashes, Troubleshooting, or Tutorials with a short description of what other things are included. These would then link to a category page. We might also test whether categories work better above search (mockup on right). | ||
[[File:SUMOHomewCategories.png|325px|left]][[File:SUMOHomewCategoriesfirst.png|325px|right]] | [[File:SUMOHomewCategories.png|325px|left]][[File:SUMOHomewCategoriesfirst.png|325px|right]] | ||