Necko/MobileCache/MicroBenchmarks: Difference between revisions

Jump to navigation Jump to search

Necko/MobileCache/MicroBenchmarks (view source)

Revision as of 14:04, 5 October 2011

427 bytes added ,  5 October 2011
→‎test_stresstest_cache_with_timing.js
Line 80: Line 80:


=== test_stresstest_cache_with_timing.js  ===
=== test_stresstest_cache_with_timing.js  ===
This test loads a number of '''different''' resources with uniform size. The resources are loaded sequentially in a tight loop and the test syncs with cache-io thread after finishing the loop. Each loop is executed for five different cache-configurations/setups:


# no cache enabled, loading all urls from server, not writing to cache (clearing cache before this step)
This test loads a number of '''different''' resources with uniform size. The resources are loaded sequentially in a tight loop and the test syncs with cache-io thread after finishing the loop. Each loop is executed for five different cache-configurations/setups:
# memory-cache enabled, loading all urls from server, write them to cache (clearing cache before this step)
# memory-cache enabled, read all urls from cache (resources cached in previous step)
# disk-cache enabled, loading all urls from server, write them to cache (clearing cache before this step)
# disk-cache enabled, reading all urls from cache (resources cached in previous step)


This whole procedure is repeated over a given list of datasizes, clearing the cache between each repetition.
#no cache enabled, loading all urls from server, not writing to cache (clearing cache before this step)
#memory-cache enabled, loading all urls from server, write them to cache (clearing cache before this step)
#memory-cache enabled, read all urls from cache (resources cached in previous step)
#disk-cache enabled, loading all urls from server, write them to cache (clearing cache before this step)
#disk-cache enabled, reading all urls from cache (resources cached in previous step)


Time is measured in two ways: 1) for each individual load by by using the nsITimingChannel interface of the channel, and 2) by measuring run-time of the complete loop from the test itself.
This whole procedure is repeated over a given list of datasizes, clearing the cache between each repetition.  


Informally one can say that this test loads a large number of uniformly sized resources at the same time, somewhat similar to what happens when starting up firefox with lots of open tabs.
Time is measured in two ways: 1) for each individual load by by using the nsITimingChannel interface of the channel, and 2) by measuring run-time of the complete loop from the test itself.  


==== Generated datafiles ====
Informally one can say that this test loads a large number of uniformly sized resources at the same time, somewhat similar to what happens when starting up firefox with lots of open tabs.  
This test writes data into a number of files. '''Note that''' all files are appended, meaning that if they exist prior to the run, you will have strange data...


; summary.dat : Format is multi-data with one datablock for each datasize run by the test. Each datablock has a header describing the datasize, then one row for each of the five configurations summing up the results from the loop. Datapoints are
==== Generated datafiles ====
# description of the configuration
# minimum value from the loop
# maximum value from the loop
# calculated mean over the results from the loop
# standard deviation over all data from the loop
# number of iterations in the loop
# time measured by the test using Data.now() for the loop
# time measured by the test using Data.now() for the loop, including syncing with the cache-io thread at the end


; telemetry-<bytes>.dat : One file for each datasize run by the test. The format is multi-data with one datablock for each configuration. Each row represents one resource-load in actual order. The datapoints in each row are
This test writes data into a number of files. '''Note that''' all files are appended, meaning that if they exist prior to the run, you will have strange data...  
# timestamp of create
# time from create to asyncOpen
# time from asyncOpen to cacheEntryAvailable
# time from cacheEntryAvailable to cacheReadStart
# time from cacheReadStart to cacheReadEnd
alternatively (if loaded from channel, not from cache)
# timestamp of create
# time from create to asyncOpen
# time from asyncOpen to cacheEntryAvailable
# time from cacheEntryAvailable to responseStart
# time from responseStart to responseEnd
I.e. referring to the Telemetry-data accumulated by nsLoadGroup, (2+3) corresponds to Telemetry::HTTP_PAGE_OPEN_TO_FIRST_FROM_CACHE alternatively Telemetry::HTTP_PAGE_OPEN_TO_FIRST_RECEIVED, and (2+3+4) corresponds to Telemetry::HTTP_PAGE_COMPLETE_LOAD.


==== Files to work with GnuPlot (requires Gnuplot >= 4.4) ====
;summary.dat&nbsp;
:Format is multi-data with one datablock for each datasize run by the test. Each datablock has a header describing the datasize, then one row for each of the five configurations summing up the results from the loop. Datapoints are


;settings.gnu: setup for the histogram, title of plot etc. Edit this to describe your platform.
#description of the configuration
#minimum value from the loop
#maximum value from the loop
#calculated mean over the results from the loop
#standard deviation over all data from the loop
#number of iterations in the loop
#time measured by the test using Data.now() for the loop
#time measured by the test using Data.now() for the loop, including syncing with the cache-io thread at the end


;plot-summary.gnu: Plots mean-values from summary.dat in one histogram, grouped by first column. Standard deviation in the dataset is indicated as an errorline. This is the recommended starting-point for studying results from a test-run.
;telemetry-&lt;bytes&gt;.dat&nbsp;
:One file for each datasize run by the test. The format is multi-data with one datablock for each configuration. Each row represents one resource-load in actual order. The datapoints in each row are


;plot-minmax-summary.gnu: Like above but indicates min and max values from the dataset instead of stddev.
#timestamp of create
#time from create to asyncOpen
#time from asyncOpen to cacheEntryAvailable
#time from cacheEntryAvailable to cacheReadStart
#time from cacheReadStart to cacheReadEnd


;plot-summary-per-size.gnu and plot-minmax-summary-per-size.gnu: Plots summaries like the two plots above, but makes one plot for each datasize to avoid scaling-issues in the plot.
alternatively (if loaded from channel, not from cache)


;plot-summary-from-javascript.gnu: '''This is WIP.''' Plots summaries of average loading-times seen from JS.
#timestamp of create
#time from create to asyncOpen
#time from asyncOpen to cacheEntryAvailable
#time from cacheEntryAvailable to responseStart
#time from responseStart to responseEnd


;plot-telemetry.gnu: You need to copy the file "telemetry-<size>.dat" to "telemetry.dat" for the size you want to study. Plots detailed information for each resource loaded, one plot for each configuration. The format used is a rowstacked histogram where each column shows total time from creating the channel to loading has finished. '''Note''' that columns show time for each load separately (all times starting at 0) as opposed to a real time-line for the dataset.
I.e. referring to the Telemetry-data accumulated by nsLoadGroup, (2+3) corresponds to Telemetry::HTTP_PAGE_OPEN_TO_FIRST_FROM_CACHE alternatively Telemetry::HTTP_PAGE_OPEN_TO_FIRST_RECEIVED, and (2+3+4) corresponds to Telemetry::HTTP_PAGE_COMPLETE_LOAD.
 
==== Files to work with GnuPlot (requires Gnuplot &gt;= 4.4)  ====
 
;settings.gnu
:setup for the histogram, title of plot etc. Edit this to describe your platform.
 
;plot-summary.gnu
:Plots mean-values from summary.dat in one histogram, grouped by first column. Standard deviation in the dataset is indicated as an errorline. This is the recommended starting-point for studying results from a test-run.
 
;plot-minmax-summary.gnu
:Like above but indicates min and max values from the dataset instead of stddev.
 
;plot-summary-per-size.gnu and plot-minmax-summary-per-size.gnu
:Plots summaries like the two plots above, but makes one plot for each datasize to avoid scaling-issues in the plot.
 
;plot-summary-from-javascript.gnu
:Plots summaries of average loading-times seen from JS. The term "INCLUDING sync" means that the time includes time to sync with the cache-io thread after finishing a loop.
 
;plot-telemetry.gnu
:You need to copy the file "telemetry-&lt;size&gt;.dat" to "telemetry.dat" for the size you want to study. Plots detailed information for each resource loaded, one plot for each configuration. The format used is a rowstacked histogram where each column shows total time from creating the channel to loading has finished. '''Note''' that columns show time for each load separately (all times starting at 0) as opposed to a real time-line for the dataset.
 
==== Sample results  ====
 
Here are some results provided by "plot-summary-from-javascript.gnu". The first is from a 64 bit Linux server
 
[[Image:Watson-100iter-summary.png]]
 
and this second is from a Nexus S
 
[[Image:Nexuss-100iter-summary.png]]
 
The Nexus S seems to generally perform well.


=== test_timing_cache.js '''(Obsolete)''' ===
=== test_timing_cache.js '''(Obsolete)''' ===
BjarneG
97

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/355177"

Navigation menu