TestEngineering/Services/SyncTestPlanV1: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 107: Line 107:
== TS 5: blah ==
== TS 5: blah ==


== TS 6: blah ==
== TS 6: Migration to "new" Sync ==
* At some point, all old Sync users will either have to
** Migrate to new Sync
** Stop using Sync
 
* Areas of focus for testing:
** QA will need to prepare a set of steps, before and after tests, to verify migration
** QA will need to interrogate/investigate account contents from old and new sync to verify migration
** All of this QA work can be done in Stage first to verify migration will actually run as expected.
** Dev and OPs suppport will be needed for the test and verification steps.
* Notes:
** From the user end, we want to make sure as much data as possible is migrated over to new sync.
** Old Sync data will exist in PHX1 datacenter, new Sync data may exist there or in AWS
** I assume all this work will be done manually by the user (since data is encrypted, OPs can not automatically move data over). Will be a challenge to see how this is handled per account vs.per device (if per device syncing is supported).
* Documentation review pertaining to old Sync, new Sync and migration steps/strategies
* See Migration Strategy for steps involved and timing
* Bug reporting will happen through Bugzilla
* REFs:
** https://wiki.mozilla.org/User_Services/Sync/v1#Migration_strategy
** https://services.etherpad.mozilla.org/sync-migration


== TS 7: Documentation ==
== TS 7: Documentation ==

Revision as of 22:48, 16 August 2013

This is the Sync/Picl Test Plan, V1

Intro/Summary/Notes

Intro

  • TBD

Summary

  • A complete snapshot of what it takes to test and release picl/sync: tools, processes, schedules, resources, etc

Notes

  • This is a living document. As such, assume very regular changes, updates, and rewrites as we go through the Dev milestones of v1.
  • This document is designed to accompany the v1 release. Although it will be used as a reference for the various Dev milestones for v1, it is not specific to any one milestone.
  • Milestone-specific test plans could be broken out from here to focus on very specific test sections, as defined below.
  • Requested sync-able collections for V1/M1:
    • Awesomebar/History
    • Passwords
    • Open Tabs
    • Bookmarks
    • (supported per account, but not per device)
  • Requested basic feature set for V1/M1:
    • Device compatibility (desktop/mobile only, not FFox)
    • Setup (Sign Up, Sign In)
    • Account Management
    • Password change/reset(forgot)
    • Detach a device (maintain the account)
    • Better security/encryption
    • Performance/stability

Schedule

QA Team

  • Services QA
    • Edwin Wong - Manager
    • James Bonacci - Sync/Picl Lead
    • John Morrison
    • Karl Thiessen
    • Peter deHaan
  • QA
    • Various members of the Firefox Desktop and Mobile teams will be involved in the testing of V1.
  • QA head count (DRAFT)
    • 1 FT automation/tools lead + QA team support
    • 1 FT android client lead + QA team support
    • 1 FT desktop client lead + QA team support
    • 1 FT FFos lead + QA team support
    • 1 FT Auth/Storage lead + QA team support
    • 1 FT server/security/db lead + QA team support - TBD
    • A subset of the FF and Services QA teams for testing when this goes to FF Nightly.
    • The entire FF and Services QA teams as part of normal testing of FF Aurora, Beta, Release channels
    • Challenges
      • Recruiting QA and SoftVision (or other contractors) for testing Try builds
      • Recruiting QA team for full testing during normal FF releases
      • Recruiting community for testing Firefox channels when they become available (esp. if they have experience with Sync 1.x)

Email Lists and IRC

  • Services QA: services-qa@mozilla.com
  • QA Teams: qa@mozilla.com
  • Sync and Picl: sync-dev@mozilla.org
  • IRC: #picl
    • #sync, #androidsync, #identity

Bug Reporting

Firefox Releases

  • Early stages
    • Desktop try builds, 32bit and 64bit, as required, source: TBD
    • Android Dev build, source: TBD
    • FFos Dev build, source: TBC
  • Nightly
    • Similar to above
  • Three-channel focus with latest features, fixes, etc in Nightly
    • But, basically, at this point, feature set should be frozen

Platforms/OS

  • Mac 10.7.x and 10.8.x
  • Windows XP 32bit, Windows 7 32bit, Windows 7 64bit)
  • Windows 8 (Metro mode and desktop mode). Does this come in a 32bit flavor?
  • Linux 32bit and 64bit - mostly likely one of Ubuntu latest, Mint latest, Fedora latest
    • The Goal is to stay consistent with requirements for FireFox and for FFos.

Test Sections/Suites

TS 1: blah

TS 2: blah

TS 3: blah

TS 4: blah

TS 5: blah

TS 6: Migration to "new" Sync

  • At some point, all old Sync users will either have to
    • Migrate to new Sync
    • Stop using Sync
  • Areas of focus for testing:
    • QA will need to prepare a set of steps, before and after tests, to verify migration
    • QA will need to interrogate/investigate account contents from old and new sync to verify migration
    • All of this QA work can be done in Stage first to verify migration will actually run as expected.
    • Dev and OPs suppport will be needed for the test and verification steps.
  • Notes:
    • From the user end, we want to make sure as much data as possible is migrated over to new sync.
    • Old Sync data will exist in PHX1 datacenter, new Sync data may exist there or in AWS
    • I assume all this work will be done manually by the user (since data is encrypted, OPs can not automatically move data over). Will be a challenge to see how this is handled per account vs.per device (if per device syncing is supported).
  • Documentation review pertaining to old Sync, new Sync and migration steps/strategies
  • See Migration Strategy for steps involved and timing
  • Bug reporting will happen through Bugzilla
  • REFs:

TS 7: Documentation

Why? Because current documentation (all of it really) is a pain point

  • Note: For QA, this is mostly a tracking responsibility
  • Updates to the old/current Sync information across locations
    • internal, app, wiki, SUMO, MDN, etc
  • Creation of new Sync information across locations
    • internal, app, wiki, SUMO, MDN, etc
  • Dev and QA can help define, write, and review documentation
    • UX/UI Demos and training
    • Usage demos and training (through SUMO?)
    • Documentation team support is needed
    • SUMO team support is needed
    • REFs: TBD

TS 8: Localization

  • Note: For QA, this is mostly a tracking responsibility
    • across UX/UI/clients
  • Mobile and Desktop clients - full UX/UI experience, wording, etc.
  • Dev and QA can help with string verfication before and after localization
  • Will need support from the FireFox Localization team and community
  • Bug reporting will happen through Bugzilla
  • REFs: TBD

TS 9: Firefox OS

  • TBD

TS 10: Metro (Win8/tablet) and Sync support

  • TBD

TS 11: Server/Backend Data Storage

  • TBD
    • Note this is the Sync Server portion of the product.
    • The Auth data store is covered in Section 1: Authentication.

Main References

Wikis and Etherpads

GitHub Repositories

Retrieved from "https://wiki.mozilla.org/index.php?title=TestEngineering/Services/SyncTestPlanV1&oldid=695801"