Firefox OS/Performance/Automation/Launch Latency: Difference between revisions
< Firefox OS | Performance | Automation
Jump to navigation
Jump to search
No edit summary |
|||
| Line 1: | Line 1: | ||
= Launch Latency = | = Launch Latency = | ||
== Summary == | == Summary == | ||
| Line 6: | Line 7: | ||
== Usage == | == Usage == | ||
== Test Cases == | === Test Cases === | ||
Launch each of the following apps from a cold state: | Launch each of the following apps from a cold state: | ||
| Line 35: | Line 36: | ||
* moz-app-loaded | * moz-app-loaded | ||
== Precision == | === Precision === | ||
TBD | TBD | ||
== | === How to Run On-Demand === | ||
=== 2.0 === | === Published Results === | ||
==== 2.0 ==== | |||
{| class="wikitable" | {| class="wikitable" | ||
! | ! | ||
! Flame | ! Flame | ||
|- | |- | ||
| '''startup_>_moz-chrome-dom-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-chrome-dom-loaded&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median | | '''startup_>_moz-chrome-dom-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-chrome-dom-loaded&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| '''startup_>_moz-chrome-interactive''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-chrome-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median | | '''startup_>_moz-chrome-interactive''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-chrome-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| '''startup_>_moz-app-visually-complete''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median | | '''startup_>_moz-app-visually-complete''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| '''startup_>_moz-content-interative''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-content-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median | | '''startup_>_moz-content-interative''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-content-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| '''startup_>_moz-app-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-app-loaded&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median | | '''startup_>_moz-app-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-app-loaded&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
|} | |} | ||
=== 2.1 === | ==== 2.1 ==== | ||
{| class="wikitable" | {| class="wikitable" | ||
! | ! | ||
! Flame | ! Flame | ||
|- | |- | ||
| '''startup_>_moz-chrome-dom-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-chrome-dom-loaded&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median | | '''startup_>_moz-chrome-dom-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-chrome-dom-loaded&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| '''startup_>_moz-chrome-interactive''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-chrome-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median | | '''startup_>_moz-chrome-interactive''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-chrome-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| '''startup_>_moz-app-visually-complete''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median link] | | '''startup_>_moz-app-visually-complete''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median link] | ||
|- | |- | ||
| '''startup_>_moz-content-interative''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-content-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median | | '''startup_>_moz-content-interative''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-content-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| '''startup_>_moz-app-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-content-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median | | '''startup_>_moz-app-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-content-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
|} | |} | ||
== | |||
=== References === | |||
* [https://developer.mozilla.org/en-US/Apps/Build/Performance/Firefox_OS_app_responsiveness_guidelines Firefox OS App Responsiveness Guidelines] | |||
== Development == | |||
=== Design === | |||
=== Setup === | === Setup === | ||
# Restart B2G process | # Restart B2G process | ||
| Line 90: | Line 102: | ||
=== Execution === | === Execution === | ||
# Test should be executed 30 times, take average and median | # Test should be executed 30 times, take average and median | ||
== Roadmap == | == Roadmap == | ||
Revision as of 12:16, 23 July 2014
Launch Latency
Summary
Measures elapsed time from the first launch of an application to a series of defined startup events.
Usage
Test Cases
Launch each of the following apps from a cold state:
- Browser
- Calendar
- Camera
- Clock
- Contacts
- Dialer
- FM Radio
- Gallery
- Marketplace
- Music
- Settings
- SMS
- Template
- Usage
- Video
For each app, report the time elapsed to the following events:
- moz-chrome-dom-loaded
- moz-chrome-interactive
- moz-app-visually-complete
- moz-content-interative
- moz-app-loaded
Precision
TBD
How to Run On-Demand
Published Results
2.0
| Flame | |
|---|---|
| startup_>_moz-chrome-dom-loaded | Datazilla |
| startup_>_moz-chrome-interactive | Datazilla |
| startup_>_moz-app-visually-complete | Datazilla |
| startup_>_moz-content-interative | Datazilla |
| startup_>_moz-app-loaded | Datazilla |
2.1
| Flame | |
|---|---|
| startup_>_moz-chrome-dom-loaded | Datazilla |
| startup_>_moz-chrome-interactive | Datazilla |
| startup_>_moz-app-visually-complete | link |
| startup_>_moz-content-interative | Datazilla |
| startup_>_moz-app-loaded | Datazilla |
References
Development
Design
Setup
- Restart B2G process
Test
- Invoke launch programmatically
- Inject the Performance Helper and save start time
- Instrumented throws timeline events at appropriate times
- Performance helper observes events
- Event emit times are compared with start time for time elapsed
Execution
- Test should be executed 30 times, take average and median
Roadmap
Overview
- Lead: Eli Perelman
- Tracking Bug: TBD
Milestone 1: Design
Tracking Bug: TBD
| Target Release | Test Design | Validity Review | |
|---|---|---|---|
| All | TBD | Bug | Bug |
Milestone 2: Test
Tracking Bug: TBD
| Target Release | Instrumentation | Workload | Single-run test | Results Review | Available | |
|---|---|---|---|---|---|---|
| Browser | TBD | bug 837668 | Bug | Bug | Bug | |
| Calendar | TBD | bug 837671 | Bug | Bug | Bug | |
| Camera | TBD | bug 1015385 | N/A | Bug | Bug | |
| Clock | TBD | bug 837668 | N/A | Bug | Bug | |
| Contacts | TBD | bug 1015388 | Light | Bug | Bug | |
| Dialer | TBD | bug 837668 | Light | Bug | Bug | |
| TBD | bug 837677 | Bug | Bug | Bug | ||
| FM Radio | TBD | bug 1015412 | Bug | Bug | Bug | |
| Gallery | TBD | bug 837674 | Light | Bug | Bug | |
| Marketplace | TBD | bug 837668 | Bug | Bug | Bug | |
| Music | TBD | bug 837675 | Bug | Bug | Bug | |
| Settings | TBD | bug 1015405 | N/A | Bug | Bug | |
| SMS | TBD | bug 1015390 | Light | Bug | Bug | |
| Template | TBD | bug 1015391 | N/A | Bug | Bug | |
| Usage | TBD | bug 837678 | Bug | Bug | Bug | |
| Video | TBD | bug 837658 | N/A | Bug | Bug |
Milestone 3: Publication
Tracking Bug: TBD
| Target Release | Published Results | Documentation | Available | |
|---|---|---|---|---|
| Browser | TBD | Bug | Bug | |
| Calendar | TBD | Bug | Bug | |
| Camera | TBD | Bug | Bug | |
| Clock | TBD | Bug | Bug | |
| Contacts | TBD | Bug | Bug | |
| Dialer | TBD | Bug | Bug | |
| TBD | Bug | Bug | ||
| FM Radio | TBD | Bug | Bug | |
| Gallery | TBD | Bug | Bug | |
| Marketplace | TBD | Bug | Bug | |
| Music | TBD | Bug | Bug | |
| Settings | TBD | Bug | Bug | |
| SMS | TBD | Bug | Bug | |
| Template | TBD | Bug | Bug | |
| Usage | TBD | Bug | Bug | |
| Video | TBD | Bug | Bug |