Firefox OS/Performance/Release Acceptance
< Firefox OS | Performance
Criteria
Definitions
- First Launch
- Milliseconds elapsed from the user-initiated launch of a not-already-in-memory app to the stable display of that app and its visible content. These timings are captured to the moz-app-visually-complete event documented on MDN.
- First Launch Memory
- Maximum bytes of RAM used by an application after completing its First Launch.
- Reference Workload
- A standardized collection of application data preloaded before the test is run.
Note: All measurements are the median results calculated from multiple iterations of relevant tests. For example, Datazilla's App Launch values are calculated from 30 such iterations.
Application cold launch
FxOS 2.5
- Reference Device: Flame-KK 319MB and Aries-KK
- Reference Workload: Light
| Application | First Launch | Regression | Nominated |
|---|---|---|---|
| Calendar | 1150 ms | Bug 1181019 | |
| Camera | 1550 ms | ||
| Clock | 1000 ms | Bug 1181021 | |
| Contacts | 1000 ms | ||
| Dialer | 1000 ms | ||
| 1000 ms | |||
| FM Radio | 1000 ms | ||
| Gallery | 1000 ms | ||
| Music | 1000 ms | ||
| Settings | 2600 ms | Bug 1181023 | |
| SMS | 1200 ms | Bug 1181017 | |
| Video | 1000 ms |
NOTE: These apps have been instrumented to emit events that support timing their launch states.
FxOS 2.0, 2.1, 2.2
- Reference Device: Flame 319MB
- Reference Workload: Light
- For Camera App, as discussed with the PM, the performance boundary will be based on the average of 3 performance runs done in Oct 2nd, 21st, and 30th. If the number regresses more than 100ms from the criteria, it will be considered as a regression.
| Application | FxOS 2.2 | FxOS 2.1 | FxOS 2.0 |
|---|---|---|---|
| Calendar | 1150 ms | 1150 ms | 1000 ms |
| Camera | 1550 ms | 1550 ms | 1000 ms |
| Clock | 1000 ms | 1000 ms | 1000 ms |
| Contacts | 1000 ms | 1000 ms | 1000 ms |
| Cost Control | 1000 ms | 1000 ms | 1000 ms |
| Dialer | 1000 ms | 1000 ms | 1000 ms |
| 1000 ms | 1000 ms | 1000 ms | |
| FM Radio | 1000 ms | 1000 ms | 1000 ms |
| Gallery | 1000 ms | 1000 ms | 1000 ms |
| Homescreen | 1000 ms | 1000 ms | |
| Marketplace | 1000 ms | 1000 ms | 1000 ms |
| Music | 1000 ms | 1000 ms | |
| Settings | 2600 ms | 2600 ms | 1000 ms |
| SMS | 1200 ms | 1200 ms | 1000 ms |
| Video | 1000 ms | 1000 ms | 1000 ms |
FxOS 1.4
- Reference Device: Buri/Hamachi
- Reference Workload: Medium
| Application | First Launch |
|---|---|
| Browser | No regression since 1.3 |
| Camera | No regression since 1.3 |
| Contacts | No regression since 1.3 |
| Dialer | No regression since 1.3 |
| No regression since 1.3 | |
| Gallery | No regression since 1.3 |
| Marketplace | No regression since 1.3 |
| Settings | No regression since 1.3 |
| SMS | No regression since 1.3 |
FxOS 1.3T
- Reference Device: Tarako
Acceptance for 1.3T will be determined by reviewing final performance results which will include cold launch time.