: Etherpad users! We are developing an extension that will allow you to create pages from etherpads quickly and easily. Please visit our sandbox and help us test it.

Difference between revisions of "QA/Test Days/Stories"

From MozillaWiki
< QA‎ | Test Days
Jump to: navigation, search
(Organizer and Date)
(Organizer and Date)
 
Line 16: Line 16:
 
== Desktop and Android Web Apps Test Day ==
 
== Desktop and Android Web Apps Test Day ==
  
=== Organizer and Date ===
+
=== Event Organizer and Date ===
 
'''Organizer:''' [mailto:jsmith@mozilla.com Jason Smith]
 
'''Organizer:''' [mailto:jsmith@mozilla.com Jason Smith]
  

Latest revision as of 23:16, 27 February 2012

Summary

The following is a repository of "stories" about testdays. What was tried, what was successful, what wasn't, what was learned, ideas new and old.

I recommend that you include the following information in your story as points of reference:

  • Date/Time of your testday
  • Subject/Goals of your testday
  • Links to testplan or any important documents
  • Your name and contact info (ex. email) so people can ask you follow up questions

WebQA 'Live' Test Day

On February 23, 2012 at 18:30, the WebQA team held a combined test day and meetup event. The purpose was to invite web testers from all skill levels to our San Fransisco space to learn hands-on how we test our websites.

This event was organized by Dave Hunt.

Desktop and Android Web Apps Test Day

Event Organizer and Date

Organizer: Jason Smith

Date: February 17th, 2012

Time: 1am - 5pm PST

Goals

  • Simulate an end-to-end customer scenario with the Web Apps product
  • Determine what manual test cases pass and fail
  • Verify fixed bugs
  • Exploratory test the product

What was tried

  • Starting test day organization a week and half ahead
  • Utilizing a template built by past test days
  • Following a checklist of items posted by Anthony
  • Publicizing the test day to many different avenues
  • Requesting moderators from waverly and QA staff
  • Building test accounts for using the product
  • Posting links to major builds, test sites, bug queries
  • Asking community members to join Mozillians
  • Posting a "how to" for non-obvious tasks
  • Indicating known issues with the product
  • Posting a key customer scenario to model a demo script with detailed steps
  • Asking community members to run documented manual test cases
  • Asking community members to verify fixed bugs
  • Proposing different areas to exploratory test the product
  • Asking community members to free write with including test results & bug numbers
  • Being highly active on the channel to answer any question that arises
  • Helping new people to testing get ramped up on how to get started on IRC
  • Getting major test day documents and processes reviewed by QA staff members

What was successful

  • Well-prepared mostly for the test day overall
  • Had more enough work for community members to do on the test day
  • Publicity was good - Lots of people knew about this test day
  • Had enough moderators to answer questions
  • Community members were active asking questions
  • Very few setup issues with community members in testing the product
  • Some community members added themselves to Mozillians
  • Many existing resolved bugs were verified on the test day (>10 bugs)
  • Free writing - Allowed community members to be more open about what they found, even if they were unsure
  • Reviewing the test day before starting - caught issues before the test day
  • New people seemed interested and helped out on the test day

What was not successful

  • Lots of issues brought up in free writing, but few bugs filed
  • Configuration issues with the build and deployment on test day
    • Some areas were unavailable to test since the code was not ready
  • Mostly desktop testers, not that many Android testers
  • Did not seem like community members saw that the manual test cases spreadsheet had multiple sheets to it
    • Most tests ran seemed to reflect one of the sheets typically
  • Even with documented known issues, community members hit them and did not realize it was a known issue
  • Facebook integration for test days was found to have some issues with QMO integration

Links