Identity/DDT Smoke Test: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
Line 12: Line 12:
staging: http://beta.myfavoritebeer.org/
staging: http://beta.myfavoritebeer.org/


== Tests ==
== Basic Developer Driven Tests ==


=== Sanity/Acceptance/Smoke ===
=== Dialog ===
* Small, repeatable set of tests with known, good, expected results.
* Create a "secondary" address based account.
* See the Test Cases (listed in the "Links and Documentation" section above) for more information.
** Create a new account using a "secondary" email address.
** Add a secondary address to the account.
** Add a primary address to the account.
** Log out and sign back in using both primary and secondary addresses.


* Main Site (https://diresworb.org/)
* Forgot Password
** Sign Up with a new account and one email
** Sign out of dialog, enter known secondary address, and click "forgot password".  User needs to enter new password and verify email.  No other emails are associated with account.
*** Verify correct email verification sent to correct account (email provider)
** Sign In with a current account of one email
** Sign Out from a current account of one email
** Change the password on a current account with one email
*** Verify correct email verification sent to correct account (email provider)
** Remove email on account with one email closes account.
** Delete account


* Dialog (http://myfavoritebeer.org or http://123done.org using one desktop OS with one browser) - create account using secondary email
* Create a "primary" address based account.
** Sign In using one RP/client to create a new account with one secondary email
** Create a new account using a "primary" email address.
*** Ensure set password screen displayed and works as expected
** Add a different primary address
*** Verify correct email verification sent to correct account (email provider)
** Add a "secondary" address - user is required to set password.
** Logout/Sign In/Logout using the same account from a different RP/client
** Log out and sign back in using both primary and secondary addresses.
** Sign In and add one BID email to the same account from a different RP/client
*** Verify correct email verification sent to correct account (email provider)
** Sign In and add one primary email to the same account
*** Verify user is correctly sent to IdP to verify credentials
** Sign In and change the password on the same account
*** Verify correct email verification sent to correct account (email provider)
** Sign In to the same account and "sign out" by using the This is not me... link
** Logout/Forgot Password
*** Ensure set password screen displayed and works as expected
*** Verify correct email verification sent to correct account (email provider)
** Delete/Remove the account from the server-side


* Dialog (http://myfavoritebeer.org or http://123done.org using one desktop OS with one browser) - create account using primary email
* Assertion/Password Authentication Levels.
** Sign In using one RP/client to create a new account with one primary email
** Sign out of dialog, enter known "primary" address.  Verify with IdP if needed. Open dialog again, select "secondary" address.  User should now have to enter password.
*** Verify user does not see set password screen but is correctly sent to IdP to verify credentials
** Logout/Sign In/Logout using the same account from a different RP/client
** Sign In and add second primary email to the account
*** Verify user is correctly sent to IdP to verify credentials
** Sign In and add first secondary address to account
*** Ensure set password screen displayed and works as expected
*** Verify correct email verification sent to correct account (email provider)


=== Main Site ===
* Check all pages for formatting


* Dialog (http://myfavoritebeer.org or http://123done.org using mobile browser)
* Sign Up/Sign In/Forgot Password
** Sign In using one RP/client to create a new account with one email
** Sign up using "secondary" address.
*** Verify correct email verification sent to correct account (email provider)
** Sign in using "secondary" address.
** Logout/Sign In/Logout using the same account from a different RP/client
** Forgot password for "secondary" address.
** Sign In and add one BID email to the same account from a different RP/client
** Sign up using "primary" address.
*** Verify correct email verification sent to correct account (email provider)
** Sign in using "primary" address.
** Sign In and add one primary email to the same account
** Sign Out
** Sign In and change the password on the same account
*** Verify correct email verification sent to correct account (email provider)
** Sign In to the same account and "sign out" by using the This is not me... link
** Delete/Remove the account from the server-side


=== Support for Primaries ===
* Management page
* Verify basic Primary support through the use of one or more test Primary sites
** Remove Address
* Primary: https://eyedee.me/
** Change password
 
** Remove Account
* BrowserID tests (from Server or from RPs)
** Creating an account with Primary emails
** Creating an account with mixed emails (Primary/BID)
** Adding/Deleting a Primary email from a Primary account
** Adding/Deleting a BID email from a Primary account
** Adding/Deleting a Primary email from a BID account
** Adding/Deleting a BID email from a BID account
** Deleting an account with Primary emails
** Deleting an account with mixed emails (Primary/BID)
** Account Manager password changes on accounts with both primary and secondary emails
 
* Primary site UI flow
** General tests for navigating the site
** Include failures, cancellations, backing out
 
 
 
 
=== Basic Functional ===
Manual and automated testing on the client and the server to verify basic functionality of BrowserID:
 
* Accounts and Email Verification
** Creating an account from the server site using Sign Up
** Creating an account inline (at first use of an RP/client Sign In)
** Email notifications for new accounts: verification email through email provider with proper email account listed, live verification link, etc.
*** Test with emails/accounts on various, popular email servers/services
*** See more details in the next bulleted list
** Creating multiple accounts with one or more emails
** Deleting one or more accounts (cancellation) from the server site
 
* Email Notifications for server or RP/Client
** On the same OS
*** Email notification and verification using one browser
*** Email notification on one browser, verification on an another
** Across multiple OS
*** Email notification on one specific browser on one specific OS
*** Verification on the same browser on a different OS
** Other cases
*** Email notification on one specific browser on one specific OS
*** Verification on a different browser on a different OS
** Mail Servers
*** Check functionality when the user does not verify by email (skips, forgets)
*** Check functionality when the user can not verify by email (email provider is down or user can not access email account for some reason)
** BrowserID Server site
*** Check functionality when BrowserID server is unavailable (down or user is off the net)
*** Check functionality when BrowserID server is available but user has slow connection (like a public wifi)
 
* Accounts and Emails
** Adding additional emails to an account
** Attempt to add an email (that may or may not be yours) from another account
** Deleting one or more emails (without actually deleting the account)
** Leaving/returning to sites (while signed in, after signing out)
** Browser restart after creation of account or access of an account
** Always logging out from sites vs. never logging out from sites (session timeouts)
** Shared access to same computer or profiles or accounts with different users
** Browser settings and preferences, esp. pop-ups, cookies, security, privacy
** Cancelling accounts
** Copy/Pasting emails (names) or passwords from other sources
** Auto-completion of emails and passwords
** Merging one or more accounts (that may or may not be yours)
 
* Email and Password fields
** Email strings/types/limits
** Verify all legal combinations of characters for both "local name" and "domain name" parts of the email string
** Password strings/types/limits
 
* More on Emails and Passwords
** Use of passwords (strict) vs. pass phrases
** Verify minimum/maximum sizes of emails and passwords (length)
** Password reset, password remember/restore
** Unique/unusual/edge case emails and passwords
** Email and Password character compatibility
** Valid vs. invalid email formats
** Valid vs. invalid password formats
** Different accounts using same email/password combos
** Try to setup a new account with a password and/or email already in use
** Copy/Pasting passwords from other sources
** Verify that passwords are never stored in LocalStore on the user machine
** Verify whether or not passwords are stored client-side (Stage RP)
** Verify whether or not passwords are stored on the server (Stage server)
** Verify proper formatting with very long emails and/or passwords.
 
* Cross site activity
** Creating an account on one RP/client, verifying the account/email on another RP/client
** Adding an email on one RP/client while logged into another RP/client
** Deleting an email on one while logged into other RPs/clients
** Cancel account/delete email from the server while signed into an RP/client
** Deleting the whole account while logged into one or more RPs/clients
*** This must be done from the server
** Verify how account information on the server reflects the changes for each of these tests
 
* Other Areas
** Verify that the user cannot log in with an email if he/she did not confirm the used email
** Log in simultaneously in two different browsers with the same email, then log out from one of the two browsers
** Log in with different emails for different clients in the same browser/different browser
** Log in with the same email for different clients in the same browser, then log out from one of the browsers
 
 
=== UI ===
Manual and automated testing on the RP/client and the BID server to cover all aspects of the current UI.
* Stage: Account creation UI in https://www.diresworb.org/
* Stage: Account creation inline using http://beta.myfavoritebeer.org/ or http://123done.org
** Stage: Account creation inline using Primary support
 
* Sign-in UI
* Email field
* Password field
* "forgot your password?" link
* "learn more" link
* "This is not me" link
* "Use a different email" link
* "Terms of Service" and "Privacy" links off of RP sites
 
* Account management: https://www.diresworb.org Account Manager page
* Sign Up
* Sign In
* Edit button
* Remove button
* Password button
* "cancel your account" link
 
* UI for new user
** Email field
** Verify button
** Verification email
** Verification link
** Password
** Email selection - radio button vs. email string
 
* Confirm your Email UI - verifying accurate "prove" link
** Confirm email verification from client-side and server-side, as defined
* Confirm Email verification UI
 
* RP/Client-side UI after sign-in (http://beta.myfavoritebeer.org, http://123done.org)
** There is a hello message of sorts: Yo, <NAME>!
** A dummy logo
** A logout link
 
* RP/Client links:
* BrowserID: https://diresworb.org
* Source code: https://github.com/lloyd/myfavoritebeer.org/
* Source code: https://github.com/mozilla/123done
* Mozilla Labs: http://mozillalabs.com/
 
* Server side UI:
* BrowserID: https://diresworb.org/
* How It Works: https://diresworb.org/about
* Take The Tour: https://diresworb.org/about
* Developers: https://github.com/mozilla/browserid/wiki/How-to-Use-BrowserID-on-Your-Site
* Identity Team: http://identity.mozilla.com/
* Mozilla Labs: http://mozillalabs.com/
* Privacy: https://diresworb.org/privacy
* TOS: https://diresworb.org/tos
* Need Help: https://support.mozilla.com/en-US/kb/what-browserid-and-how-does-it-work
* Sign In: https://diresworb.org/signin
* Sign Up: https://diresworb.org/signup
* Account Manager (once you are signed in)
 
* UI to Add new emails ("Use a different email") while still logged into RP/client
* UI to Change a password ("Forgot your password?") on the RP/client
* UI to select Terms of Services, Privacy, Learn More on the RP/client
* UI to Edit the account on the Server from the Account Manager
* UI to Remove an email on the Server from the Account Manager
* UI to Change the Password on the Server from the Account Manager
** Verify functionality with only BID emails
** Verify functionality with only Primary emails
** Verify functionality with a mix of email types
* UI to Cancel an account (all emails) on the Server
 
* General UI navigation to cover the following:
** Error screens and dialogs
** Email verification pop-ups and in browser (the fade/change to a server page)
** Closing pop-ups manually rather than with a Cancel, OK, Continue, or other button
 
* Various UI scenarios and navigation not covered above...
** The affects of Stay logged in vs. always logging out
 
* General ease of use
** Mouse/pointer: left (click), right/ctrl click, hover over links, etc.
** Keyboard: selection, tabbing, arrows, etc.
** Mobile: touchscreen, keyboard, etc.
 
 
=== Accessibility - Desktop only ===
* Verify minimal accessibility in the UI - keyboard only
** Creating a new BrowserID account
*** Email verification from mail application and from BrowserID
** Adding a new email to a current BrowserID account
*** Email verification from mail application and from BrowserID
** Changing/resetting a password - RP flow
*** Email verification from mail application and from BrowserID
** Changing a password - Account Manager
** Deleting an email from an account - Account Manager
** Deleting an account - Account Manager
** Other UI flows: This is not me, BrowserID links, etc.
 
* Note: this testing may require changes at the OS-level (like Mac OS) or changes at the browser level to more fully support keyboard-only access to a site.

Revision as of 11:24, 8 June 2012

This document is taken from content on QA BrowserID TestPlan - https://wiki.mozilla.org/QA/BrowserID/TestPlan

Purpose

As BrowserID grows and the product becomes increasingly complex, it is a difficult task to introduce new features without also introducing bugs. To help reduce the number of bugs that make it to prod as well as to ease some of the pain on our own QA team, we are introducing a WWE move - the DDT. DDT (developer driven testing) is like jumping off the top rope to smash bugs before they have a chance to cause pain.

QA has a well defined set of tests they perform before qualifying any release as "ready for production." We are going to use a very slimmed down version of the Sanity/Acceptance/Smoke tests.

Tests should be performed against both dev and staging. 123done.org will be the RP of choice once it is set up to handle all environments. 123done.org currently only supports staging. Until 123done.org is ready, testing will take place using myfavoritebeer.org as the RP.

dev: http://dev.myfavoritebeer.org/ staging: http://beta.myfavoritebeer.org/

Basic Developer Driven Tests

Dialog

  • Create a "secondary" address based account.
    • Create a new account using a "secondary" email address.
    • Add a secondary address to the account.
    • Add a primary address to the account.
    • Log out and sign back in using both primary and secondary addresses.
  • Forgot Password
    • Sign out of dialog, enter known secondary address, and click "forgot password". User needs to enter new password and verify email. No other emails are associated with account.
  • Create a "primary" address based account.
    • Create a new account using a "primary" email address.
    • Add a different primary address
    • Add a "secondary" address - user is required to set password.
    • Log out and sign back in using both primary and secondary addresses.
  • Assertion/Password Authentication Levels.
    • Sign out of dialog, enter known "primary" address. Verify with IdP if needed. Open dialog again, select "secondary" address. User should now have to enter password.

Main Site

  • Check all pages for formatting
  • Sign Up/Sign In/Forgot Password
    • Sign up using "secondary" address.
    • Sign in using "secondary" address.
    • Forgot password for "secondary" address.
    • Sign up using "primary" address.
    • Sign in using "primary" address.
    • Sign Out
  • Management page
    • Remove Address
    • Change password
    • Remove Account
Retrieved from "https://wiki.mozilla.org/index.php?title=Identity/DDT_Smoke_Test&oldid=439315"