Firefox OS/Performance/Boot Timing Automation

As per Bug 1045938, the plan for implementing automated boot info capturing will be described below. The end goal is the eventual creation of a system to support the continued tracking of the status of boot performance. That is to say, one should be able to simply do something like

make test-perf boottest

and receive the appropriate data. Furthermore we would like to get this information into datazilla along with all the other performance numbers we track.

Plan Outline

This primarily relates to Bug 1047118, but may well subsume Bug 1047125 as well due to some changes from initial ideas. The simplest approach we could take for accomplishing this is as follows:

The data emitted for each stage will be:

1. Capture and store pre-gecko timing to disk (something like "/data/local/.BOOTTIMES")
   • Currently setting an android property in init, and accessing it from b2g.sh to write out to disk
   • Also capturing the timing of when we launch b2g proper
   • File in JSON format to be compatible with the other tools.
2. settings.js/shell.js emit some "created" event near beginning.
   Questions:
   • Is make test-perf awake early enough to catch the events?
   • If not, we could send timing information in the outgoing event itself.
3. System app (in gaia) emits some "created" event
   • Do this right before iframe creation in shell.js ?
4. System App:
   • Emit moz-* events
     • Will need to implement above events for system app.
5. Vertical homescreen
   • moz-* are emitted
   • No way of actually observing these events for homescreen currently implemented.
   • Furthermore, the timing normally collected for these events are a bit different than the kind of timing I desire.

make test-perf will consume the .BOOTTIMES file, and all these events to give back meaningful data. In the pre-gecko timing one of the data points will be a pair of both "uptime" and "epoch time". Using this, we can calculate the time of boot in terms of epoch time, which is more compatible with times we have access to in the higher levels of the stack. Then, for each "created" event that fires, the current time will be recorded. There may need to be additional handling added for getting compatible times from the moz-* events.

For the system and homescreen apps, instead of emitting events for marionette to observe, the design taken will likely be marionette sending out an event to the app when it desires to know times, and the app in return firing an event carrying the relevant data.

Data Visualization

Once the data itself is collected it could be presented in datazilla. What this would look like would be for there to be a Boot test, and for each stage in the boot process to have its own line tracked over time. The Y-axis would be the cumulative time from boot, of each stage. This allows one to easily see how the duration of each section changes over time. Some parts (like the kernel) would likely stay relatively static, while we may see more fluctuations in others (gaia or gecko for example).