QA/Weave Test Plan/Temp

From MozillaWiki
Jump to: navigation, search
Weave Test Plan

Overview

Weave Sync is an extension that encrypts and securely synchronizes the Firefox experience across multiple browsers, so that your desktop, laptop, and mobile phone can all work together.

Testing Weave will present some unique challenges in that data from many areas of the browser are to be stored and synced securely across multiple browser versions and multiple operating systems, including mobile.

Mozilla Labs - Weave Project Page
Weave Roadmap

Builds

Test Strategy

The Strategy for testing weave is evolving over time since it is a new product with a different kind of roadmap that Mozilla is accustomed to.  We will initially approach the major releases with this strategy:

  1. Weave Client Extension Testing (Lead: Tracy Walker)
    • This involves black box execution directly on the extension, testing the features and usage of the operation of the client.   Involves data syncing and cross browsers and OS platform support.   Will heavily include Fennec integration.
  2. Weave Client Load Testing (Lead: Matt Evans)
    • This involves a collective approach to organizing weave users around the globe to create a load generation through the client to generate network traffic to production servers. Still need to define measuring performance evaluation and instrumentation on what a successful network load would be. Likely outsourced.
  3. Weave Thick Client Server Testing (Lead: Jonathan Griffin)
    • Data Driven Testing
    • Parametrize user credentials (so that it can be cloned out to X vms for cloud testing)
    • Use multiple profiles on the same machine (so that it can be easily cloned for testing and can be easily setup and run by hand by developers).

Areas Covered

  1. UI/front end - Test case spreadsheet
    • Account management
      • Setup new account
      • Start over
      • Password and Passphrase changes
      • Account Deletion?
    • ensure sync is accurate in the following areas. No data loss is critical. Check each area by syncing item adds and item removals
    • Backward compatibility - 1.2b3 should be compatible to 1.2b2
    • cross platform/product sync
    • see <Tracy's testplan (link here)> for more
  1. Client Side Load Generation
    • See <Matt's Testplan (link here)> for more
  1. Thick Client load testing
    • See <Jgriffin's Testplan (link here)> for more

Areas not covered

  1. Server/back end
    • Weave server test script - run on changes and prior to pushes to stage.
    • Current implementation of server side tests include unit tests against the sync API.   There are plans for Weave 1.3 to include the following:
      • Registration API unit tests
      • Load Balancing tests
      • Failover/recovery tests?
      • others...  See Mike hanson and Tobi Elliot
  1. Unit Test Frameworks
    • Weave 1.3 will include a unit test framework hooked into tinderbox that includes nightly runs of tests.   This will be handled by development.  

Platform and product coverage

  • Browsers Versions covered
    • Firefox 3.6 (primary)
    • Fennec 1.0 (primary)
    • Firefox 3.5 (spot check if time allows)
    • Firefox next (spot check if time allows)
  • Operating systems covered
    • Windows
      • 7 (primary)
      • XP (spot check if time allows, ahead of any other spot checks)
      • Vista (spot check if time allows)
    • Apple
      • Snow Leopard (10.6) (primary)
      • Leopard (10.5) (spot check if time allows)
      • Tiger (10.4) (spot check if time allows)
    • Linux (spot check if time allows)
    • Mobile
      • fennec mobile device (primary)
      • fennec desktop versions on XP (spot check if time allows)

Schedule Scoping

Bugs

Litmus Test Cases added

Until Weave is integrated into Firefox test cases will be filed in a Weave dedicated testgroup. When weave becomes integrated into Firefox, I'll merge the test cases into the Firefox test groups.

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Weave_Test_Plan/Temp&oldid=215374"