Necko/MobileCache/MicroBenchmarks: Difference between revisions

Jump to navigation Jump to search

Necko/MobileCache/MicroBenchmarks (view source)

Revision as of 12:59, 24 October 2011

970 bytes removed ,  24 October 2011
→‎Sample results
Line 143: Line 143:
==== Sample results  ====
==== Sample results  ====


Here are two plots by "plot-summary-from-javascript.gnu". The leftmost is from a 64 bit Linux server and the rightmost from a Nexus S. ('''Only results with empty cache is reported here since filling the cache kills my Nexus at the moment.''')
This benchmark has currently been run successfully on two different Linux-systems and on a Nexus S. Below are extracts from the results, focusing on time to load resources as seen from JavaScript. The plots have been obtained by editing "summary.dat", sorting results by cache-type and splitting up in "hits" and "misses" which are plotted separately. Modifying the generated files and the GnuPlot-files to do this is trivial and I won't discuss it here.


{|  
The first set of graphs show "misses", i.e. it indicates time to search the cache for each resource, clear space for it, load it and finally set it up to be saved. Resources are served from the in-built httpd.js - loading with cache disabled is included first as a reference. Hence, the first group in the histogram show resources loaded without caching. The next three groups show results with disk-cache enabled and the last three groups show results for the memory-cache.
| [[Image:Watson-100iter-summary.png|400px]]
 
| [[Image:Nexuss-100iter-summary.png|400px]]
{|
| [[File:Watson-summary-misses-100iter.png|400px]]
| [[File:HPEnvy-summary-misses-100iter.png|400px]]
| [[File:Nexuss-summary-misses-100iter.png|400px]]
|}
|}


The Nexus S seems to perform very well compared to the Linux server, in fact it is generally faster! Note that the Nexus handles entries of size up to 8K pretty much as as fast as 128byte entries, whereas the Linux box maintains this performance only up to 4K. Note also the very first column in the Nexus-plot - most likely there is one outlier in this dataset. Let's study it closer and use "plot-timingchannel.gnu" with the "timingchannel-128.dat" file


[[File:Nexuss-100iter-128byte-telemetry-nocache.png|800px]]
Note that a memory-cache full of small entries (sixth group) seems to perform bad for large resources (surprising, because you'd expect the disk-cache to always be slower).


Observe that the first load takes a long time, caused by a delay between creating the channel to asyncOpen is called. I'm not entirely sure what can cause this, but I assume it's because it's the first load and certain things may have to be loaded and set up. It is not (should not be!) caused by the cache-service creating the disk-cache because caching is disabled in this test. It is easy to modify the test to load a couple of resources prior to starting the timed loop. The corresponding timingchannel-plot from the Linux server looks like this
Note that a disk-cache full of small entries (third group) is generally slower than the empty disk-cache (second group).


[[File:Watson-100iter-128byte-telemetry-nocache.png|800px]]


This is from another run on the Linux server, including results for using full caches. The first graph shows "misses", i.e. it indicates time to search the cache for each resource, clear space for it, load it and finally set it up for saving to the cache. Loading resources with cache disabled is included first as a reference.
The second set of graphs show "hits", i.e. the time spent to load the resource from cache. It indicates time to search for the resource as well as load it. Columns are sorted as in previous set of graphs.


[[Image:Watson-summary-misses-100iter.png|800px]]
{|
 
| [[File:Watson-summary-hits.png|400px]]
The second graph shows the hits, i.e. time when each resource is found in the cache. '''Note that no shuffling of the entries in the cache is done prior to measuring this - all 100 entries are nicely placed in sequence as the least recently used elements loaded in the previous test'''  Loading resources with cache disabled is included first as a reference.
| [[File:HPEnvy-summary-hits.png|400px]]
 
| [[File:Nexuss-summary-hits.png|400px]]
[[Image:Watson-summary-hits-100iter.png|800px]]
|}
 
By studying the two latter graphs a few points seems to require further investigation
 
; miss / mem-cache full of small entries: seems to perform really bad for large resources (surprising, because I'd expect the disk-cache to always be slower)
 
; miss / disk-cache full of small entries: performs clearly worse than the empty disk-cache for small resources (not surprising, but we may want to try optimizing this somewhow)


; hit / mem-cache full of small entries: seems to perform really bad for 2k and 8k resources (can be reproduced in subsequent runs) - could it be the access-pattern?
Note the time to load 2K and 8K entries from a memory-cache filled with small entries... Could this be caused by the access-pattern or is it generally true?


=== test_timing_cache.js '''(Obsolete)''' ===
=== test_timing_cache.js '''(Obsolete)''' ===
BjarneG
97

edits

Retrieved from "https://wiki.mozilla.org/Special:MobileDiff/360372"

Navigation menu