Summary

This document explains all of the charts I have prototyped since joining the GFX team at Mozilla. The purpose of this page is to document what has been prototyped with a goal to eventually put into production the metrics which are valuable to people making decisions about the quality of Firefox.

It should be noted that the use case is what's important. The prototypes are just a means to an end and the code itself should not be considered ready for production.

References:

• MetricsGraphics.js (used for charting)
• Socorro Supersearch API (used for gathering crash data)
• Bugzilla Rest API (used for gathering bug data)
• metrics-graphics-gfx code (source code for the dashboard)
• metrics-graphics-gfx dashboard (the public instance of the dashboard)

Supplemental

The development of this work gave birth to the Bugzilla Socorro Lens add-on which I developed at the London All-hands 2016 under Benoit Girard's guidance.

Explanation of Charts

Topline Charts

V2 Charts

Firefox Charts

Fennec Charts

Bugzilla

Prototypes

Charts behind VPN

(TO BE COMPLETED - Document the Mac Mini system)

Data Sources

Psuedocode

The bulk of the code comes from the MetricsGraphics.js library. Relevant to *this* project are the files contained in charts (html/js to load a specific chart) and data (json containing the specific chart data).

Charts

Each of the charts basically follows the same code:

1. Load data as JSON
2. Parse JSON data using D3.js into a JS Object
3. In some cases do calculations based on the data
4. Output the resulting Object as:
   {
   "date":date, (usually serves as X-axis)
   "category":value, (usually serves as Y-axis)
   ...}
5. Pass the output object to MetricsGraphics.js for graphing

Data

First Method

Originally I was using python to generate chart data as JSON. I would call the Socorro API, get the 6-months of data back as JSON, convert it to something I wanted to chart, then write out the resulting JSON to a file in the /data folder. My charts which use this method would simply load the appropriate JSON file from this /data folder. I would have to run the .py scripts periodically to update this files and push to github were I to want the chart to update. Any newly conceived queries essentially start from scratch with a 6-month snapshot and require building up data over time.

Second Method

The second method just loads the data in realtime from the Socorro API. This is slower than the first method as it means retrieving the remote data and processing it before charting, and doing all of this on the client side. However it does benefit from the fact that updating the data on the server side is unnecessary and therefore the data is much more recent. Another benefit is that a chart representing a new query or some new set of data can be spun up fairly quickly (simply a matter of changing the API URL in the chart).

Third Method

The third method essentially combines the two. I run a python script in a daily cron job which syncs relevant data to a local database. I then run a series of python scripts to update local JSON files for the charts from this local database. The charts point to the local JSON files which at most are a day out of date and contain more than 6 months of data. The downside is that this is only accessible behind the VPN and new queries are generally harder to architect as it requires updating the data generation scripts -- however we can generally get the older trend data even if the query is new so long as we aren't looking for data that has not already been synced to the database. For example if there's field I wasn't already syncing this would need to be added and would start from the most recent 6-months of available data.