3,860
edits
m (Lakrits moved page FirefoxOS/Performance/Automation/Launch Latency to Firefox OS/Performance/Automation/Launch Latency: The official spelling of "Firefox OS" leaves a space between the two parts of the name. It's easier to find a page if the sp...) |
|||
(76 intermediate revisions by one other user not shown) | |||
Line 1: | Line 1: | ||
= Launch Latency = | = Launch Latency = | ||
== Summary == | == Summary == | ||
Measures elapsed time from the first launch of an application to a series of defined [https://developer.mozilla.org/en-US/Apps/Build/Performance/Firefox_OS_app_responsiveness_guidelines startup events]. | Measures elapsed time from the first launch of an application to a series of defined [https://developer.mozilla.org/en-US/Apps/Build/Performance/Firefox_OS_app_responsiveness_guidelines startup events]. | ||
== | == Usage == | ||
== | === Test Cases === | ||
Launch each of the following apps from a cold state: | |||
* Browser | * Browser | ||
* Calendar | |||
* Camera | * Camera | ||
* Clock | |||
* Contacts | * Contacts | ||
* Dialer | * Dialer | ||
* Email | * Email | ||
* FM Radio | |||
* Gallery | * Gallery | ||
* Marketplace | * Marketplace | ||
* Music | |||
* Settings | * Settings | ||
* SMS | * SMS | ||
* Template | |||
* Usage | |||
* Video | |||
For each app, report the time elapsed to the following events: | |||
* moz-chrome-dom-loaded | |||
* moz-chrome-interactive | |||
* moz-app-visually-complete | |||
* moz-content-interative | |||
* moz-app-loaded | |||
=== Precision === | |||
TBD | |||
=== | === How to Run On-Demand === | ||
TBD | |||
=== Published Results === | |||
=== | ==== FxOS 2.0 ==== | ||
{| class="wikitable" | {| class="wikitable" | ||
! | ! | ||
! Flame | |||
|- | |||
| '''startup_>_moz-chrome-dom-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-chrome-dom-loaded&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | |||
|- | |||
| '''startup_>_moz-chrome-interactive''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-chrome-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | |||
|- | |||
| '''startup_>_moz-app-visually-complete''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | |||
|- | |||
| '''startup_>_moz-content-interative''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-content-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | |||
|- | |- | ||
| ''' | | '''startup_>_moz-app-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame&range=7&test=startup_%3E_moz-app-loaded&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
|} | |} | ||
=== | ==== FxOS 2.1 ==== | ||
{| class="wikitable" | {| class="wikitable" | ||
! | ! | ||
! | ! Flame | ||
|- | |- | ||
| ''' | | '''startup_>_moz-chrome-dom-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-chrome-dom-loaded&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| ''' | | '''startup_>_moz-chrome-interactive''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-chrome-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| ''' | | '''startup_>_moz-app-visually-complete''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| ''' | | '''startup_>_moz-content-interative''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-content-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| ''' | | '''startup_>_moz-app-loaded''' || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame&range=7&test=startup_%3E_moz-content-interactive&app_list=calendar,camera,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,video&app=calendar&gaia_rev=8cb1a949f2e9650b&gecko_rev=c6f9f0396153&plot=median Datazilla] | ||
|- | |- | ||
| ''' | |} | ||
=== References === | |||
* [https://developer.mozilla.org/en-US/Apps/Build/Performance/Firefox_OS_app_responsiveness_guidelines Firefox OS App Responsiveness Guidelines] | |||
== Development == | |||
=== Design === | |||
==== Common Setup ==== | |||
# Set up workload | |||
# Restart B2G process | |||
==== All Cases ==== | |||
# Invoke launch programmatically | |||
# Inject the Performance Helper and save start time | |||
# App instrumentation throws timeline events at appropriate times | |||
# Performance helper observes events | |||
# Event emit times are compared with start time for time elapsed | |||
==== Results ==== | |||
* Result is time elapsed to each event | |||
* Execute test 30 times, for each event report mean, median, deviation | |||
=== Project Roadmap === | |||
* '''Lead:''' Eli Perelman | |||
* '''Tracking Bug:''' TBD | |||
{| class="wikitable" | |||
! width="120px" | | |||
! colspan="2" | Milestone 1: Design | |||
|- | |- | ||
! | |||
! Test Design | |||
! Validity Review | |||
|- | |- | ||
| ''' | | '''All Cases''' || Bug || Bug | ||
|- | |- | ||
|} | |} | ||
{| class="wikitable" | {| class="wikitable" | ||
! width="120px" | | ! width="120px" | | ||
! | ! | ||
! colspan="4" | Milestone 2: Test | |||
! colspan="2" | Milestone 3: Publication | |||
|- | |||
! | |||
! Availability | |||
! Instrumentation | |||
! Workload | |||
! On-Demand Test | |||
! Results Review | |||
! Published Results | ! Published Results | ||
! Documentation | ! Documentation | ||
|- | |- | ||
| '''Browser''' || TBD || Bug || Bug | | '''Browser''' || TBD || {{bug|837668}} || Bug || Bug || Bug || Bug || Bug | ||
|- | |||
| '''Calendar''' || TBD || {{bug|837671}} || Bug || Bug || Bug || Bug || Bug | |||
|- | |||
| '''Camera''' || TBD || {{bug|1015385}} || N/A || Bug || Bug || Bug || Bug | |||
|- | |||
| '''Clock''' || TBD || {{bug|837668}} || N/A || Bug || Bug || Bug || Bug | |||
|- | |||
| '''Contacts''' || TBD || {{bug|1015388}} || [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/make_options_reference#Reference_Workloads Light] || Bug || Bug || Bug || Bug | |||
|- | |||
| '''Dialer''' || TBD || {{bug|837668}} || [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/make_options_reference#Reference_Workloads Light] || Bug || Bug || Bug || Bug | |||
|- | |||
| '''Email''' || TBD || {{bug|837677}} || Bug || Bug || Bug || Bug || Bug | |||
|- | |||
| '''FM Radio''' || TBD || {{bug|1015412}} || N/A || Bug || Bug || Bug || Bug | |||
|- | |- | ||
| ''' | | '''Gallery''' || TBD || {{bug|837674}} || [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/make_options_reference#Reference_Workloads Light] || Bug || Bug || Bug || Bug | ||
|- | |- | ||
| ''' | | '''Marketplace''' || TBD || {{bug|837668}} || Bug || Bug || Bug || Bug || Bug | ||
|- | |- | ||
| ''' | | '''Music''' || TBD || {{bug|837675}} || [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/make_options_reference#Reference_Workloads Light] || Bug || Bug || Bug || Bug | ||
|- | |- | ||
| ''' | | '''Settings''' || TBD || {{bug|1015405}} || N/A || Bug || Bug || Bug || Bug | ||
|- | |- | ||
| ''' | | '''SMS''' || TBD || {{bug|1015390}} || [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/make_options_reference#Reference_Workloads Light] || Bug || Bug || Bug || Bug | ||
|- | |- | ||
| ''' | | '''Template''' || TBD || {{bug|1015391}} || N/A || Bug || Bug || Bug || Bug | ||
|- | |- | ||
| ''' | | '''Usage''' || TBD || {{bug|837678}} || N/A || Bug || Bug || Bug || Bug | ||
|- | |- | ||
| ''' | | '''Video''' || TBD || {{bug|837658}} || [https://developer.mozilla.org/en-US/Firefox_OS/Developing_Gaia/make_options_reference#Reference_Workloads Light] || Bug || Bug || Bug || Bug | ||
|- | |- | ||
|} | |} | ||
<small>[http://wiki.mozilla.org/FirefoxOS/Performance/Automation/ | <small>[http://wiki.mozilla.org/FirefoxOS/Performance/Automation/Metric_Template template]</small> |
edits