Firefox OS/Performance/Boot Timing Automation
As per Bug 1045938, the plan for implementing automated boot info capturing will be described below. The end goal is the eventual creation of a system to support the continued tracking of the status of boot performance. That is to say, one should be able to simply do something like
make test-perf boottest
and receive the appropriate data. Furthermore we would like to get this information into datazilla along with all the other performance numbers we track.
This primarily relates to bug 1047118. The simplest approach we could take for accomplishing this is as follows:
The data emitted for each stage will be:
- Capture and store pre-gecko timing to disk (something like
- settings.js/shell.js generate a time stamp (that is stored in some standard property of the window) near its entrypoint.
- System app (in chrome) records timing at some decided location (as in the prior step):
- System App:
- Vertical homescreen
- Record time stamp at entry point
- See implementation here.
- moz-* are emitted
- There is no way of actually observing these events for homescreen currently implemented, and there needs to be work done to make use of these events being fired in this test framework.
In the pre-gecko timing one of the data points will be a pair of both "uptime" and "epoch time". Using this, we can calculate the time of boot in terms of epoch time, which is more compatible with times we have access to in the higher levels of the stack. If there were ever to be some WebAPI to get device uptime, this would be a far preferable means, and would very much minimize the complexity currently needed in this implementation. .
Another problematic portion of the implementation is that as a result of the system date taking time to correct itself upon boot (sometimes this is even after gecko has started), depending on when one samples the date, it is possible to get a very incorrect date. As a stop gap measure, I collect a time "early" in the boot sequence, and another "late" in the boot sequence, and for the calculations use which ever is more appropriate as the reference.
In the current design of
make test-perf's startup_event_tests, the test designates code to be injected, and launches the app, thus having full control over the whole process. This is not quite a suitable approach for the system and verticalhome apps which are kind of special cases. Upon gecko finishing its initialization steps, shell.html is unconditionally loaded, which loads the system app directly (which then also goes and prepares the lockscreen and the homescreen). The approach of injecting certain handlers may be possible however, if one reloads b2g to give the code a chance to be injected into the relevant frames.
What the boot_test does is, instead of firing and catching events, it (in the future perhaps conditionally upon a setting) records time stamps in some predetermined object for the test harness to collect afterwards. This way there is minimal impact on the app's performance. The test looks for results in the system chrome, the system app, and the homescreen.
Once the data itself is collected it could be presented in datazilla. What this would look like would be for there to be a Boot test, and for each stage in the boot process to have its own line tracked over time. The Y-axis would be the cumulative time from boot, of each stage. This allows one to easily see how the duration of each section changes over time. Some parts (like the kernel) would likely stay relatively static, while we may see more fluctuations in others (gaia or gecko for example).