View source for Performance/Snappy/Testing:BFCache Sprint

= Overview =
The purpose of this sprint is to gather some initial baseline data to determine how often Firefox is actually using BFCache

= Background =
BF cache is what makes pages appear instantly when pressing back (ie. there is no incremental page layout, image loading, etc). Other types of cache include disk cache and memory cache.

For our needs, we just need to make sure disk cache is empty at startup (rebooting OS before-hand) so that we get a fair comparison. Once we have a stable test for comparing network traffic, we should also profile disk cache using xperf.

For more background, read [http://blog.mozilla.com/tglek/2010/10/04/diagnosing-slow-startup/ this blog post].

= Testing =
== Baseline Testing ==
=== Overview ===
* testing click-through perf on several pages
* time to click link to throbber stopping
** click next link when throbber stops
* screencast of testing (cell phone, or software)
* running test in parallel
** 2 people run the test on the same hardware, differing browsers
* clear cache between testing
* ebay, amazon, or something that is not so dynamic
* looking for one browser being slower than another, interest is in timing
* start with a couple QA then branch to community
* timeline ~2 weeks from now
* timings in spreadsheet, screencasts stored in a people account

=== Hardware ===
* low-end on Windows 7 (ie. netbook) -- x120e is the best candidate
* make sure harddrive light is off
* indexing / scans should not be running (ie. Antivirus)
* use ethernet on the same network
* run in QA lab

=== Reference Test Case ===
The following should be run twice, once on Firefox Nightly, once on Opera.

Preliminary
# Install browser
# Start with a new profile
# Create a bookmark for Amazon.com, Huffingtonpost.com and Tomshardware.com
# Clear the cache
# Make sure no hard drive activity
# Report all results in [https://docs.google.com/spreadsheet/ccc?key=0AmkRt0ylPb8zdGxPRDhUbGdoM3lwd3JQNXY3bW5nNHc&hl=en_US#gid=0 this spreadsheet] (times in ''ms'') -- if possible provide a link to a screencast

What to Test
# Start the timer (use your cellphone if it helps)
# Open one of the websites via bookmark
# Pause the timer once the throbber stops and record the time
# Resume the timer and click the specified link
# Pause the timer once the throbber stops and record the time
# Repeat steps 4 and 5 for each link specified

How to Test
# Run the test for all three sites in Firefox
# Quit Firefox and start Opera
# Run the test for all three sites in Opera
# Quit Opera and start Firefox
# Run the test for all three sites in Firefox
# Quit Firefox and start Opera
# Run the test for all three sites in Opera
# Quit Opera

Amazon.com Checkpoints
# Initial load of Amazon.com
# Help
# Shipping Rates & Information
# International
# Other Help Areas > Kindle
# Kindle Keyboard
# Wireless Coverage Map
# Careers
# Code Ninjas
# FAQ

Huffingtonpost.com Checkpoints
# Initial load of Huffingtonpost.com
# User Agreement (bottom of the page)
# Privacy (bottom of the page)
# About Us (bottom of the page)
# Front Page
# Politics
# Business
# World
# Living
# Tech

Tomshardware.com Checkpoints
# Initial load of Tomshardware.com
# Articles
# Components
# How to
# Modding
# ''"In Pictures: 19 Ways To Recycle Old, Unused Hardware"'' from Oct 16, 2010
# Cycle through all of the 19 pictures (use the '''>''' button above the picture)
# Legal (at the bottom of the page)
# Charts > HDD Charts 2012
# Desktop Performance benchmark

== Sprints ==
; Preparing your System
* Download and install the [http://www.microsoft.com/download/en/details.aspx?id=8279 Windows SDK] (this will give you the Windows Performance Toolkit which includes xperf.exe)
* Download [http://www.wireshark.org/ Wireshark]

; Phase 1 - xperf on Windows 7
* Start by restarting your computer
* Run ''cmd.exe'' as Administrator
* Run ''xperf -on latency+FILE_IO+FILE_IO_INIT+DISK_IO''
* Start Firefox
** Navigate to your test page
** Press the back button
** Press the forward button
* Run ''xperf -d report.etl''
* Run ''xperf report.etl'' to view the report
''Need somewhere for testers to post their reports''

; Phase 2 - wireshark
* Record a wireshark dump of your browsing session
* Filter out the non-web traffic
* Submit reports comparing Firefox to Opera

; Sites to Test
Top 10 sites not requiring authentication
* www.google.com
* www.youtube.com
* www.yahoo.com
* www.baidu.com
* www.wikipedia.com
* www.amazon.com
* www.qq.com
* www.msn.com
* www.ebay.com
* www.bing.com