Firefox OS/Performance/Release Acceptance
From MozillaWiki
< Firefox OS | Performance
Contents
Criteria
Definitions
- First Launch
- Milliseconds elapsed from the user-initiated launch of a not-already-in-memory app to the stable display of that app and its visible content. These timings are captured to the moz-app-visually-complete event documented on MDN.
- First Launch Memory
- Maximum bytes of RAM used by an application after completing its First Launch.
- Reference Workload
- A standardized collection of application data preloaded before the test is run.
Note: All measurements are the median results calculated from multiple iterations of relevant tests. For example, Datazilla's App Launch values are calculated from 30 such iterations.
Reference Devices
Application cold launch
FxOS 2.5
- Reference Device: Flame-KK 319MB and Aries-KK
- Reference Workload: Light
Application | First Launch | Regression | Nominated |
---|---|---|---|
Calendar | 1150 ms | Bug 1181019 | |
Camera | 1550 ms | ||
Clock | 1000 ms | Bug 1181021 | |
Contacts | 1000 ms | ||
Dialer | 1000 ms | ||
1000 ms | |||
FM Radio | 1000 ms | ||
Gallery | 1000 ms | ||
Music | 1000 ms | ||
Settings | 2600 ms | Bug 1181023 | |
SMS | 1200 ms | Bug 1181017 | |
Video | 1000 ms |
NOTE: These apps have been instrumented to emit events that support timing their launch states.
FxOS 2.0, 2.1, 2.2
- Reference Device: Flame 319MB
- Reference Workload: Light
- For Camera App, as discussed with the PM, the performance boundary will be based on the average of 3 performance runs done in Oct 2nd, 21st, and 30th. If the number regresses more than 100ms from the criteria, it will be considered as a regression.
Application | FxOS 2.2 | FxOS 2.1 | FxOS 2.0 |
---|---|---|---|
Calendar | 1150 ms | 1150 ms | 1000 ms |
Camera | 1550 ms | 1550 ms | 1000 ms |
Clock | 1000 ms | 1000 ms | 1000 ms |
Contacts | 1000 ms | 1000 ms | 1000 ms |
Cost Control | 1000 ms | 1000 ms | 1000 ms |
Dialer | 1000 ms | 1000 ms | 1000 ms |
1000 ms | 1000 ms | 1000 ms | |
FM Radio | 1000 ms | 1000 ms | 1000 ms |
Gallery | 1000 ms | 1000 ms | 1000 ms |
Homescreen | 1000 ms | 1000 ms | |
Marketplace | 1000 ms | 1000 ms | 1000 ms |
Music | 1000 ms | 1000 ms | |
Settings | 2600 ms | 2600 ms | 1000 ms |
SMS | 1200 ms | 1200 ms | 1000 ms |
Video | 1000 ms | 1000 ms | 1000 ms |
FxOS 1.4
- Reference Device: Buri/Hamachi
- Reference Workload: Medium
Application | First Launch |
---|---|
Browser | No regression since 1.3 |
Camera | No regression since 1.3 |
Contacts | No regression since 1.3 |
Dialer | No regression since 1.3 |
No regression since 1.3 | |
Gallery | No regression since 1.3 |
Marketplace | No regression since 1.3 |
Settings | No regression since 1.3 |
SMS | No regression since 1.3 |
FxOS 1.3T
- Reference Device: Tarako
Acceptance for 1.3T will be determined by reviewing final performance results which will include cold launch time.