MozillaWiki

Identity/DDT Smoke Test

< Identity
Revision as of 12:05, 8 May 2012 by Stomlinson (talk | contribs) (Initial population)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)

This document is taken from content on QA BrowserID TestPlan - https://wiki.mozilla.org/QA/BrowserID/TestPlan

Sanity/Acceptance/Smoke

  • Small, repeatable set of tests with known, good, expected results.
  • Manual and automated testing on client and server to pass a minimal level of acceptance without which, QA testing of BrowserID can not proceed.
  • See the Test Cases (listed in the "Links and Documentation" section above) for more information.
  • Server-side (https://diresworb.org/)
    • Sign Up with a new account and one email
      • Verify correct email verification sent to correct account (email provider)
    • Sign In with a current account of one email
    • Sign Out from a current account of one email
    • Change the password on a current account with one email
      • Verify correct email verification sent to correct account (email provider)
    • Delete/Remove a current account with one email
  • Client-side (http://myfavoritebeer.org or http://123done.org using one desktop OS with one browser)
    • Sign In using one RP/client to create a new account with one email
      • Verify correct email verification sent to correct account (email provider)
    • Logout/Sign In/Logout using the same account from a different RP/client
    • Sign In and add one BID email to the same account from a different RP/client
      • Verify correct email verification sent to correct account (email provider)
    • Sign In and add one primary email to the same account
    • Sign In and change the password on the same account
      • Verify correct email verification sent to correct account (email provider)
    • Sign In to the same account and "sign out" by using the This is not me... link
    • Delete/Remove the account from the server-side
  • Client-side (http://myfavoritebeer.org or http://123done.org using Android with stock browser)
    • Sign In using one RP/client to create a new account with one email
      • Verify correct email verification sent to correct account (email provider)
    • Logout/Sign In/Logout using the same account from a different RP/client
    • Sign In and add one BID email to the same account from a different RP/client
      • Verify correct email verification sent to correct account (email provider)
    • Sign In and add one primary email to the same account
    • Sign In and change the password on the same account
      • Verify correct email verification sent to correct account (email provider)
    • Sign In to the same account and "sign out" by using the This is not me... link
    • Delete/Remove the account from the server-side

Basic Functional

Manual and automated testing on the client and the server to verify basic functionality of BrowserID:

  • Accounts and Email Verification
    • Creating an account from the server site using Sign Up
    • Creating an account inline (at first use of an RP/client Sign In)
    • Email notifications for new accounts: verification email through email provider with proper email account listed, live verification link, etc.
      • Test with emails/accounts on various, popular email servers/services
      • See more details in the next bulleted list
    • Creating multiple accounts with one or more emails
    • Deleting one or more accounts (cancellation) from the server site
  • Email Notifications for server or RP/Client
    • On the same OS
      • Email notification and verification using one browser
      • Email notification on one browser, verification on an another
    • Across multiple OS
      • Email notification on one specific browser on one specific OS
      • Verification on the same browser on a different OS
    • Other cases
      • Email notification on one specific browser on one specific OS
      • Verification on a different browser on a different OS
    • Mail Servers
      • Check functionality when the user does not verify by email (skips, forgets)
      • Check functionality when the user can not verify by email (email provider is down or user can not access email account for some reason)
    • BrowserID Server site
      • Check functionality when BrowserID server is unavailable (down or user is off the net)
      • Check functionality when BrowserID server is available but user has slow connection (like a public wifi)
    • Stage server logs (if you have access)
  • Accounts and Emails
    • Adding additional emails to an account
    • Attempt to add an email (that may or may not be yours) from another account
    • Deleting one or more emails (without actually deleting the account)
    • Leaving/returning to sites (while signed in, after signing out)
    • Browser restart after creation of account or access of an account
    • Always logging out from sites vs. never logging out from sites (session timeouts)
    • Shared access to same computer or profiles or accounts with different users
    • Browser settings and preferences, esp. pop-ups, cookies, security, privacy
    • Cancelling accounts
    • Copy/Pasting emails (names) or passwords from other sources
    • Auto-completion of emails and passwords
    • Merging one or more accounts (that may or may not be yours)
  • Email and Password fields
    • Email strings/types/limits
    • Verify all legal combinations of characters for both "local name" and "domain name" parts of the email string
    • Password strings/types/limits
  • More on Emails and Passwords
    • Use of passwords (strict) vs. pass phrases
    • Verify minimum/maximum sizes of emails and passwords (length)
    • Password reset, password remember/restore
    • Unique/unusual/edge case emails and passwords
    • Email and Password character compatibility
    • Valid vs. invalid email formats
    • Valid vs. invalid password formats
    • Different accounts using same email/password combos
    • Try to setup a new account with a password and/or email already in use
    • Copy/Pasting passwords from other sources
    • Verify that passwords are never stored in LocalStore on the user machine
    • Verify whether or not passwords are stored client-side (Stage RP)
    • Verify whether or not passwords are stored on the server (Stage server)
    • Verify proper formatting with very long emails and/or passwords.
  • Cross site activity
    • Creating an account on one RP/client, verifying the account/email on another RP/client
    • Adding an email on one RP/client while logged into another RP/client
    • Deleting an email on one while logged into other RPs/clients
    • Cancel account/delete email from the server while signed into an RP/client
    • Deleting the whole account while logged into one or more RPs/clients
      • This must be done from the server
    • Verify how account information on the server reflects the changes for each of these tests
  • Other Areas
    • Verify that the user cannot log in with an email if he/she did not confirm the used email
    • Log in simultaneously in two different browsers with the same email, then log out from one of the two browsers
    • Log in with different emails for different clients in the same browser/different browser
    • Log in with the same email for different clients in the same browser, then log out from one of the browsers
    • Verify that the log in is kept when restoring a session after a browser crash (or after closing a browser with the session restore option selected (is this Firefox only?)

Support for Primaries

  • Verify basic Primary support through the use of one or more test Primary sites
  • Primary: https://eyedee.me/
  • Other sites: TBD
  • Links
  • Verify all links on the Primary, especially if they open a new page/tab or replace the current page/tab
  • Email and Password fields
    • Email strings/types/limits
    • Verify all legal combinations of characters for both "local name" and "domain name" parts of the email string
    • Password strings/types/limits
  • Direct Primary usage
    • Signing in from Primary
    • Sign-in redirection to Primary from an RP/client
    • Sign-out from Primary
    • Sign-out from Primary while signed into an RP/client
    • Sign-out from Primary while signed into the Server site
  • BrowserID tests (from Server or from RPs)
    • Creating an account with Primary emails
    • Creating an account with mixed emails (Primary/BID)
    • Adding/Deleting a Primary email from a Primary account
    • Adding/Deleting a BID email from a Primary account
    • Adding/Deleting a Primary email from a BID account
    • Adding/Deleting a BID email from a BID account
    • Deleting an account with Primary emails
    • Deleting an account with mixed emails (Primary/BID)
    • Account Manager password changes on accounts with both primary and secondary emails
  • Network interruptions: latency or Primary site being down
  • Primary site UI flow
    • General tests for navigating the site
    • Include failures, cancellations, backing out
  • Local Installs: run the check_primary_support script on eyedee.me, other Primaries, and at least one non-primary
  • Regression: attempt Primary use with and without third-party cookie support set in the browser

UI

Manual and automated testing on the RP/client and the BID server to cover all aspects of the current UI.

  • Sign-in UI
  • Email field
  • Password field
  • "forgot your password?" link
  • "learn more" link
  • "This is not me" link
  • "Use a different email" link
  • "Terms of Service" and "Privacy" links off of RP sites
  • Account management: https://www.diresworb.org Account Manager page
  • Sign Up
  • Sign In
  • Edit button
  • Remove button
  • Password button
  • "cancel your account" link
  • UI for new user
    • Email field
    • Verify button
    • Verification email
    • Verification link
    • Password
    • Email selection - radio button vs. email string
  • Confirm your Email UI - verifying accurate "prove" link
    • Confirm email verification from client-side and server-side, as defined
  • Confirm Email verification UI
  • UI to Add new emails ("Use a different email") while still logged into RP/client
  • UI to Change a password ("Forgot your password?") on the RP/client
  • UI to select Terms of Services, Privacy, Learn More on the RP/client
  • UI to Edit the account on the Server from the Account Manager
  • UI to Remove an email on the Server from the Account Manager
  • UI to Change the Password on the Server from the Account Manager
    • Verify functionality with only BID emails
    • Verify functionality with only Primary emails
    • Verify functionality with a mix of email types
  • UI to Cancel an account (all emails) on the Server
  • General UI navigation to cover the following:
    • Error screens and dialogs
    • Email verification pop-ups and in browser (the fade/change to a server page)
    • Closing pop-ups manually rather than with a Cancel, OK, Continue, or other button
  • Various UI scenarios and navigation not covered above...
    • The affects of Stay logged in vs. always logging out
  • General ease of use
    • Mouse/pointer: left (click), right/ctrl click, hover over links, etc.
    • Keyboard: selection, tabbing, arrows, etc.
    • Mobile: touchscreen, keyboard, etc.


Accessibility - Desktop only

  • Verify minimal accessibility in the UI - keyboard only
    • Creating a new BrowserID account
      • Email verification from mail application and from BrowserID
    • Adding a new email to a current BrowserID account
      • Email verification from mail application and from BrowserID
    • Changing/resetting a password - RP flow
      • Email verification from mail application and from BrowserID
    • Changing a password - Account Manager
    • Deleting an email from an account - Account Manager
    • Deleting an account - Account Manager
    • Other UI flows: This is not me, BrowserID links, etc.
  • Note: this testing may require changes at the OS-level (like Mac OS) or changes at the browser level to more fully support keyboard-only access to a site.
Retrieved from "https://wiki.mozilla.org/index.php?title=Identity/DDT_Smoke_Test&oldid=428412"