Project Eideticker/Profiling with Eideticker

From MozillaWiki
Jump to: navigation, search

Scenario 1: Measuring with video capture


  • Gives the most comprehensive and reliable results.


  • Slow. Each capture/analysis combination takes about 5-10 minutes. Multiplied by the number of tests that you would want to run and the number of runs you'd need to get a representative result and you're easily talking several hours.
  • Requires a dedicated machine with an HDMI-capable phone connected to it.

ssh into an Eideticker machine (as of this writing, we have several in Mountain View and a few on the way in Toronto, ask William Lachance for details) and run the "get-metric-for-build" script:

cd src/eideticker
source bin/activate
wget <apk of build>
./bin/ --num-runs 5 <apk of build> <path to test>

Tests that are good to run for a representative sample of performance include ('src/tests/cnn/'), nytimes-zoom ('src/tests/ep1/nytimes/') and nytimes-scroll ('src/tests/ep1/nytimes/').

You will get a bunch of output regarding amount of checkerboarding and number of unique frames. This may be compared to the existing Eideticker dashboard.

Scenario 2: Measuring with telemetry


  • Fast. Can do a test in under a minute.
  • Work on any suitably configured linux machine.


  • Not technically a black box test, relies on internal measurements that may be inaccurate
  • Only measures checkerboarding, not the smoothness of animations, etc.
  • Results are only relevant to Fennec, and are not comparable with other browsers.

The Eideticker README contains instructions on how to set up / run this type of test.