Firefox OS/Performance/Release Acceptance: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
(Added 2.1 Criteria and updated 2.0 & 2.1 to link to 319MB flame results)
(Moved 2.0 + 2.1 first launch datazilla links from application name to First Launch column)
Line 23: Line 23:
! Application !! [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,clock,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,sms,video&app=communications/dialer&plot=median First Launch] !! First Launch Memory <!-- Relaunch !! Max Memory !! Max Power -->
! Application !! [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,clock,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,sms,video&app=communications/dialer&plot=median First Launch] !! First Launch Memory <!-- Relaunch !! Max Memory !! Max Power -->
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar&app=calendar&plot=median Calendar] || 1000 ms
| Calendar || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar&app=calendar&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=camera&app=camera&plot=median Camera]  || 1000 ms
| Camera || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=camera&app=camera&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=clock&app=clock&plot=median Clock]  || 1000 ms
| Clock || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=clock&app=clock&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=communications/contacts&app=communications/contacts&plot=median Contacts] || 1000 ms
| Contacts || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=communications/contacts&app=communications/contacts&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=costcontrol&app=costcontrol&plot=median Cost Control]  || 1000 ms
| Cost Control || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=costcontrol&app=costcontrol&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=communications/dialer&app=communications/dialer&plot=median Dialer] || 1000 ms
| Dialer || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=communications/dialer&app=communications/dialer&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=email%20FTU&app=email%20FTU&plot=median Email] || 1000 ms
| Email || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=email%20FTU&app=email%20FTU&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=fm&app=fm&plot=median FM Radio] || 1000 ms
| FM Radio || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=fm&app=fm&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=gallery&app=gallery&plot=median Gallery] || 1000 ms
| Gallery || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=gallery&app=gallery&plot=median 1000 ms]
|-
|-
| Homescreen  || 1000 ms
| Homescreen  || 1000 ms
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=settings&app=settings&plot=median Settings] || 1000 ms
| Marketplace  || 1000 ms
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=sms&app=sms&plot=median SMS] || 1000 ms
| Settings || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=settings&app=settings&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=video&app=video&plot=median Video] || 1000 ms
| SMS || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=sms&app=sms&plot=median 1000 ms]
|-
| Video || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=video&app=video&plot=median 1000 ms]
|}
|}


Line 60: Line 62:
{| class="wikitable"
{| class="wikitable"
|-
|-
! Application !! [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=bookmark,browser,calendar,camera,clock,communications/contacts,communications/dialer,communications/ftu,contacts,costcontrol,email%20FTU,emergency-call,findmydevice,fm,fm_radio,gallery,homescreen,marketplace,messages,music,operatorvariant,pdfjs,phone,pss,rss,settings,sms,system_pss,system_rss,system_uss,system_vsize,template,usage,uss,video,vsize&app=settings&plot=median First Launch]
! Application !! [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar,camera,clock,communications/contacts,communications/dialer,costcontrol,email%20FTU,fm,gallery,settings,sms,video&app=communications/dialer&plot=median First Launch] <!-- !! First Launch Memory !! Relaunch !! Max Memory !! Max Power -->
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar&app=calendar&plot=median Calendar] || 1000 ms
| Calendar || [https://datazilla.mozilla.org/b2g/?branch=master&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=calendar&app=calendar&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=camera&app=camera&plot=median Camera]  || 1000 ms
| Camera || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=camera&app=camera&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=clock&app=clock&plot=median Clock]  || 1000 ms
| Clock || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=clock&app=clock&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=communications/contacts&app=communications/contacts&plot=median Contacts] || 1000 ms
| Contacts || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=communications/contacts&app=communications/contacts&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=costcontrol&app=costcontrol&plot=median Cost Control]  || 1000 ms
| Cost Control || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=costcontrol&app=costcontrol&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=communications/dialer&app=communications/dialer&plot=median Dialer] || 1000 ms
| Dialer || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=communications/dialer&app=communications/dialer&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=email%20FTU&app=email%20FTU&plot=median Email] || 1000 ms
| Email || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=email%20FTU&app=email%20FTU&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=fm&app=fm&plot=median FM Radio] || 1000 ms
| FM Radio || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=fm&app=fm&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=gallery&app=gallery&plot=median Gallery] || 1000 ms
| Gallery || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=gallery&app=gallery&plot=median 1000 ms]
|-
|-
| Homescreen  || 1000 ms
| Homescreen  || 1000 ms
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=settings&app=settings&plot=median Settings] || 1000 ms
| Marketplace  || 1000 ms
|-
| Settings || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=settings&app=settings&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=sms&app=sms&plot=median SMS] || 1000 ms
| SMS || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=sms&app=sms&plot=median 1000 ms]
|-
|-
| [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=video&app=video&plot=median Video] || 1000 ms
| Video || [https://datazilla.mozilla.org/b2g/?branch=v2.0&device=flame-319MB&range=7&test=startup_%3E_moz-app-visually-complete&app_list=video&app=video&plot=median 1000 ms]
|}
|}


Revision as of 17:45, 5 August 2014

Criteria

Definitions

First Launch
Milliseconds elapsed from the user-initiated launch of a not-already-in-memory app to the stable display of that app and its visible content. These timings are captured to the moz-app-visually-complete event documented on MDN.
First Launch Memory
Maximum bytes of RAM used by an application after completing its First Launch.
Reference Workload
A standardized collection of application data preloaded before the test is run.

Note: All measurements are the median results calculated from multiple iterations of relevant tests. For example, Datazilla's App Launch values are calculated from 30 such iterations.

2.1

  • Reference Device: Flame 319MB
  • Reference Workload: Light
Application First Launch First Launch Memory
Calendar 1000 ms
Camera 1000 ms
Clock 1000 ms
Contacts 1000 ms
Cost Control 1000 ms
Dialer 1000 ms
Email 1000 ms
FM Radio 1000 ms
Gallery 1000 ms
Homescreen 1000 ms
Marketplace 1000 ms
Settings 1000 ms
SMS 1000 ms
Video 1000 ms

NOTE: These apps have been instrumented to emit events that support timing their launch states.


2.0

  • Reference Device: Flame 319MB
  • Reference Workload: Light
Application First Launch
Calendar 1000 ms
Camera 1000 ms
Clock 1000 ms
Contacts 1000 ms
Cost Control 1000 ms
Dialer 1000 ms
Email 1000 ms
FM Radio 1000 ms
Gallery 1000 ms
Homescreen 1000 ms
Marketplace 1000 ms
Settings 1000 ms
SMS 1000 ms
Video 1000 ms

NOTE: These apps have been instrumented to emit events that support timing their launch states.

1.4

  • Reference Device: Buri/Hamachi
  • Reference Workload: Medium
Application First Launch
Browser No regression since 1.3
Camera No regression since 1.3
Contacts No regression since 1.3
Dialer No regression since 1.3
Email No regression since 1.3
Gallery No regression since 1.3
Marketplace No regression since 1.3
Settings No regression since 1.3
SMS No regression since 1.3

1.3T

  • Reference Device: Tarako

Acceptance for 1.3T will be determined by reviewing final performance results which will include cold launch time.

Retrieved from "https://wiki.mozilla.org/index.php?title=Firefox_OS/Performance/Release_Acceptance&oldid=1002868"