Spectator: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(→‎Can I disable collection?: they are alternatives rather than succesive steps)
 
(10 intermediate revisions by 2 users not shown)
Line 1: Line 1:
= Problem Statement =
= About Spectator =
The Firefox team needs more data about how users actually use Firefox in the field.
Spectator is a Firefox extension that collects data about how Firefox is used. This information is used to better understand what people value in the browser. The extension is now available from [https://addons.mozilla.org/en-US/firefox/addon/6326 AMO].


= Proposal =  
= FAQ =
Although, we can obtain this data through detailed user studies. A more scalable solution is to technically instrument the browser to collect anonymous usability information and submit to a central site. A web application would aggregate this information into actionable summaries.
The Spectator extension collects usage information about the browser.


= Subject Areas =
=== What can Spectator collect? ===
There are numerous metrics that could be collected. For the initial build of this system, we will focus on feature use. In particular, the following questions:
Right now we are taking a snapshot of your profile (extensions installed, screen resolution, arch, installdate, etc... NOT history) every time you open the browser, logs all UI interactions, and log how/when documents are loaded (typed, link, etc). [[Browser_Metrics:Data_Collectors|See full technical details here]]. But don't be scared, keep reading....


== Feature Use ==
=== Can you see what websites I'm going to? ===
* How many times is the search box used?
No. The only URIs we capture are chrome document loads. Also, any string that comes from the user (a ui event name, an extension name, etc) is one-way hashed (md5) client side. This way we can only see data we already know about.
* Which search is being used?
* How many extensions do people use?
* Addons v. other extension distributors
* Number of themes, using a theme?
* How many bookmarks by type?
* How many pop-ups are blocked?
* How many tabs are opened/closed per session?


= Technical Implementation =
=== If you can't read what is being sent to back, how is it helpful? ===
The current proposal is to implement instrumentation as an extension. The advantages to this approach include the ability to update the extension independently and greater flexibility to promote/market the extension.  
Consider "Browser:Reload". In this case, the string "Browser:Reload" is hashed and then sent back to mozilla ( "fgKIKfJEiGq5arJfJCcOSQ==", base64). On the server side, we can hash the common stuff (like "Browser:Reload"), and connect the dots. Or consider a custom extension. If you're writing "super-secret-extension-1337", we have no idea what it is called, so we will not be able to resolve the hash. However, if you publish it on addons (or some other place we can find it), we have the extension-ID and can generate the md5.


This project can be divided into several components:
=== Can I see what you have collected? ===
Yes. You can see what has been collected since the last upload interval. To do this, hit: about:metrics, you will see the XML file we have collected since you last uploaded the data to mozilla. If you're very clever, you can make a snapshot of the data before it gets uploaded by copying it out of your profile. It is called metrics.xml.


== Data Collection Service ==
=== How do I know it is working? ===
Hit about:metrics and see if there is a log. If you do not see any XML kick the browser once. If it is still not working, let me know!


The data collection service aggregates the data and uploads it to the collection server.  Before uploading data, a manifest file will be fetched from the server to control which items will be uploaded. This allows us to tweak the volume of data collected as desired, without client-side changes.
=== What is mozilla going to do with the data? ===
We hope to use it to make Firefox better. For the first time, we'll be able to see how the user interacts with the browser. We'll know how many extensions people use. We'll be able to track if that fancy new star gets any action. From the 3 users we already have, we can say that the most common UI Event sequence is "close a window then open a tab". Lets use that info to make the best "close a window and open a tab" experience in the world!


[[Browser Metrics:Data Collection Service | Collection Service Design]]
=== Can I disable collection? ===
Yes. You have three options:


== Data Collectors ==
* Un-install or
* Disable or
* Toggle the add-on pref


Data collectors will hook in at a variety of locations in the backend and frontend.  Each collector submits ''events'' to the collection service, where each event can contain collector-specific key/value pairs.
However, the easiest way is to not install the extension in the first place!


[[Browser Metrics:Data Collectors | Data Collector Design]]
=== Where can I file bugs? ===
Toolkit, Data collection/metrics


== Server Infrastructure ==
= Technical Implementation =
 
The server is responsible for storing and providing a clean interface to the data.
 
[[User:Polvi/InstrumentationServer | Server Design]]
 
= Distribution =
 
We think that given the opportunity to opt-in to this data collection ("Help us improve Firefox"), a statistically significant number of users would enable this functionality. In addition, for prereleases, it may be feasible to enable this collection by default.


= Process =
Please see [[Browser Metrics]] for details regarding the technical implementation.
# Update [http://mxr.mozilla.org/seamonkey/source/extensions/metrics/ existing extension] [https://bugzilla.mozilla.org/show_bug.cgi?id=393881 to work with trunk]
# Setup data collector service to collect information about particular Feature Use areas
# Prototype a server to receive information

Latest revision as of 22:09, 18 March 2008

About Spectator

Spectator is a Firefox extension that collects data about how Firefox is used. This information is used to better understand what people value in the browser. The extension is now available from AMO.

FAQ

The Spectator extension collects usage information about the browser.

What can Spectator collect?

Right now we are taking a snapshot of your profile (extensions installed, screen resolution, arch, installdate, etc... NOT history) every time you open the browser, logs all UI interactions, and log how/when documents are loaded (typed, link, etc). See full technical details here. But don't be scared, keep reading....

Can you see what websites I'm going to?

No. The only URIs we capture are chrome document loads. Also, any string that comes from the user (a ui event name, an extension name, etc) is one-way hashed (md5) client side. This way we can only see data we already know about.

If you can't read what is being sent to back, how is it helpful?

Consider "Browser:Reload". In this case, the string "Browser:Reload" is hashed and then sent back to mozilla ( "fgKIKfJEiGq5arJfJCcOSQ==", base64). On the server side, we can hash the common stuff (like "Browser:Reload"), and connect the dots. Or consider a custom extension. If you're writing "super-secret-extension-1337", we have no idea what it is called, so we will not be able to resolve the hash. However, if you publish it on addons (or some other place we can find it), we have the extension-ID and can generate the md5.

Can I see what you have collected?

Yes. You can see what has been collected since the last upload interval. To do this, hit: about:metrics, you will see the XML file we have collected since you last uploaded the data to mozilla. If you're very clever, you can make a snapshot of the data before it gets uploaded by copying it out of your profile. It is called metrics.xml.

How do I know it is working?

Hit about:metrics and see if there is a log. If you do not see any XML kick the browser once. If it is still not working, let me know!

What is mozilla going to do with the data?

We hope to use it to make Firefox better. For the first time, we'll be able to see how the user interacts with the browser. We'll know how many extensions people use. We'll be able to track if that fancy new star gets any action. From the 3 users we already have, we can say that the most common UI Event sequence is "close a window then open a tab". Lets use that info to make the best "close a window and open a tab" experience in the world!

Can I disable collection?

Yes. You have three options:

  • Un-install or
  • Disable or
  • Toggle the add-on pref

However, the easiest way is to not install the extension in the first place!

Where can I file bugs?

Toolkit, Data collection/metrics

Technical Implementation

Please see Browser Metrics for details regarding the technical implementation.

Retrieved from "https://wiki.mozilla.org/index.php?title=Spectator&oldid=87304"