MozillaWiki

Weave Load Test Plan/WeaveLiveUserLoadTestPlan

< Weave Load Test Plan
Revision as of 17:44, 17 May 2010 by Mevans (talk | contribs) (→‎Results Reporting)

Overview

The load testing of the weave services, will rely on synthetic load generation against the server with concurrent live testers performing user sync operations and measuring the sync response times and ensuring basic weave operations are performing well. The live testers will be deployed over a number of geographical locations to measure the actual response times from a number global locations.

The test for the live testers to perform will be conducted in 2 phases. The 1st phase is to get systems initialized, create user synchronized data like history and bookmarks and to establish baseline performance sync measurements under light load server conditions. The second phase will be a rerun of the performance test under saturated high load levels. Performance measurements under noload and loaded conditions will compared and the difference will be checked against expected deltas.

Perquisites

  • 10 testers located evenly in serveral geographic locations (north/south america, europe, africa, asia, australia)
  • (ideally) Tester should have access to 2 different firefox supported platforms (linux, win7, mac OSX 10.6). Either physical platforms (preferred) or VMs.

Live User Initialization

  1. Install version 3.5.10 Firefox browser on one system and 3.6.4 version of the Firefox browser on the other. The download locations are ftp sites where builds for each supported platform and locale reside. Choose the appropriate build for your platform and locale. For example: http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.5.10-candidates/build1/win32/en-US/ is the location of the win32 3.5.10 build for English US.
    1. Firefox 3.5.10 download is located http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.5.10-candidates/build1
    2. Firefox 3.6.4 download is located http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/3.6.4-candidates/build3
  2. startup the browser with a clean profile on each system
    1. If you are a firefox user, it will be important not to use your personal data for testing.
    2. Instructions for starting the browser with a clean profile is located here
  3. Install the latest weave sync addon in each of the browsers
    1. Install the latest weave sync client from this location
    2. Register with the weave services
    3. Instructions for installing and registering with the weave service is located here
  4. On one of the Firefox browsers start creating sync data
    1. Browse to many sites
    2. Add lots of tabs
    3. Bookmark a number of urls
      1. add tags
      2. add toolbar bookmarks
    4. add login/passwords to pass protected sites (encrypted)
  5. On the other system
    1. Login into weave service a sync down the server data captured from session above.
      1. Ensure synchronized data is duplicated on 2nd system.

Tests to Perform

Phase 1

As mentioned above, the load test will be done in 2 phases. The first phase is to install firefox and create the user data followed by [1] Functionality Smoke Tests and then [2] Sync Performance Tests. After the first phase, user should save their user profile data.


Phase 2

The second phase will be conducted several days later when the weave systems is at the full load capacity. In the second phase the tests will be conducted in reverse order [2] Sync Performance Tests and the [1] Functionality Smoke Tests. Phase 2 should begin with the reload of the saved profile data.


[1] Functionality Smoke Tests

  • With both systems weave data synchronized, ensure user data update operations are reflected in the secondary browser instance after a weave sync:
    • data deletion (tabs, bookmarks,history,passwords)
    • data addition (tabs,bookmarks, history,passwords)
    • data change (tabs,bookmarks,history,passwords)
    • data merge (bookmark tags)

[2] Sync Performance Tests

  • weave delete local and download weave server data synchronization performance test
    1. clear activity log
    2. for 5 times do
      1. Tools->Weave->Preference->Manage Account->Reset Sync->Replace all data on this computer with your Weave data
    3. save activity log as <user-name>_<phase>_sync-replace-local.log
  • weave delete weave server data and upload local data synchronization performance test
    1. clear activity log
    2. for 5 times do
      1. Tools->Weave->Preference->Manage Account->Reset Sync->Replace all other devices with your local data
    3. save activity log as <user-name>_<phase>_sync-replace-server.log

Results Reporting

  • The main reporting of user activity will be the Activity logs
  • report any perceived errors (data loss, data corruption, http errors etc).
Tester Name Weave login P1 Sync Local Ave Time P1 Sync Server Ave Time P2 Sync Local Ave Time P2 Sync Server Ave Time Sync Local Diff Time Sync Server Diff Time
Pedro Gonzalez utest22 10.51 10.46
Travis Howell utest3 4.57 5.55
Bashra Kabir utest15 52.71 36.60
Eureka Lopez utest5 2.98 4.13
William Ricardi billricardi
Dany Ouellette utest24

References

Retrieved from "https://wiki.mozilla.org/index.php?title=Weave_Load_Test_Plan/WeaveLiveUserLoadTestPlan&oldid=223468"