Weave Load Test Plan/WeaveLiveUserLoadTestPlan

Overview

The load testing of the weave services, will rely on synthetic load generation against the server with concurrent live testers performing user sync operations and measuring the sync response times and ensuring basic weave operations are performing well. The live testers will be deployed over a number of geographical locations to measure the actual response times from a number global locations.

The test for the live testers to perform will be conducted in 2 phases. The 1st phase is to get systems initialized, create user synchronized data like history and bookmarks and to establish baseline performance sync measurements under light load server conditions. The second phase will be a rerun of the performance test under saturated high load levels. Performance measurements under noload and loaded conditions will compared and the difference will be checked against expected deltas.

Perquisites

• 10 testers located evenly in serveral geographic locations (north/south america, europe, africa, asia, australia)
• Tester should have access to 2 different firefox supported platforms (linux, win7, mac OSX 10.6)

Live User Initialization

1. Install version 3.5.10 Firefox browser on one system and 3.6.4 version of the Firefox browser on the other.
   1. Firefox 3.5.10 download is located here
   2. Firefox 3.6.4 download is located here
2. startup the browser with a clean profile on each system
   1. If you are a firefox user, it will be important not to use your personal data for testing.
   2. Instructions for starting the browser with a clean profile is located here
3. Install the latest weave sync addon in each of the browsers
   1. Install the latest weave sync client from this location
   2. Register with the weave services
   3. Instructions for installing and registering with the weave service is located here
4. On one of the Firefox browsers start creating sync data
   1. Browse to many sites
   2. Add lots of tabs
   3. Bookmark a number of urls
      1. add tags
      2. add toolbar bookmarks
   4. add login/passwords to pass protected sites (encrypted)
5. On the other system
   1. Login into weave service a sync down the server data captured from session above.
      1. Ensure synchronized data is duplicated on 2nd system.

Tests to Perform

Functionality Smoke Tests

• With both systems weave data synchronized, ensure aoperations are reflected in the secondary browser instance:
  • data deletion (tabs, bookmarks,history,passwords)
  • data addition (tabs,bookmarks, history,passwords)
  • data change (tabs,bookmarks,history,passwords)
  • data merge (bookmark tags)

Sync Performance Tests

• weave delete local and download weave server data synchronization performance test
  1. clear activity log
  2. for 5 times do
     1. Tools->Weave->Preference->Manage Account->Reset Sync->Replace all data on this computer with your Weave data
  3. save activity log as <user-name>_<phase>_sync-replace-local.log
• weave delete weave server data and upload local data synchronization performance test
  1. clear activity log
  2. for 5 times do
     1. Tools->Weave->Preference->Manage Account->Reset Sync->Replace all other devices with your local data
  3. save activity log as <user-name>_<phase>_sync-replace-server.log

Results Reporting

• The main reporting of user activity will be the Activity logs

References

• Overall weave client testplan: https://wiki.mozilla.org/QA/Weave_Test_Plan/1.3
• Build location: https://services.mozilla.com/sync/install.php
• Litmus Smoketests: https://litmus.mozilla.org/run_tests.cgi?test_run_id=83

• encourage people to search this list before filing a bug.
  • Known Weave bugs (minus dupes): http://bit.ly/btftoZ
  • Known Weave FIXED bugs: http://bit.ly/bxx7CT
  • Known Weave 1.3 final blockers: http://bit.ly/avVwal

• knowledge base articles