User:Ashughes/Bugdays/20100913
Contents
in-litmus? Bug Burn
Date | Sept 13 - 17, 2010 |
Time | 1800Z - 2400Z (11am - 6pm PDT) |
Place | irc.mozilla.org #bugday |
What | We will be triaging the in-litmus? bug queue and writing test cases for these bugs. This will be a great opportunity to hone or learn bugzilla and litmus skills, test case writing skills, and investigative skills. |
Goal | The goal will be to resolve ALL of the bugs, meaning no bugs with in-litmus? at the end of the week. |
Artifacts | Bug List, Spreadsheet, Scorecard, Pre-bugday Survey, Post-bugday Survey, Testcase Writing Documentation |
Schedule
All times are in Z-time (GMT). Subtract 7 hours for PDT.
Monday | Tuesday | Wednesday | Thursday | Friday | |
1800Z | WORKSHOP: Learning to Write Testcases | STRIKE TEAM: Tabbed Browsing | STRIKE TEAM: Session Store | STRIKE TEAM: Panorama | SPRINT & TESTDAY |
1900Z | |||||
2000Z | |||||
2100Z | |||||
2200Z | |||||
2300Z | |||||
2400Z |
Prizes & Tiers
Prizes will be awarded based on individual contributions. In the case of teams, each contributor gets equal credit for the team effort. Mozilla QA employees do not qualify for prizes
Level | Achievement | Prize |
1 | 10 points | Mozilla wristband, entry into MozillaQA draw |
2 | 20 points | T-shirt of your choosing |
3 | 50 points | $25 USD credit for the Mozilla Store |
Top Tester | ||
$100 USD gift card (store TBD) |
- Scoring
- SCORECARD: link
- WORKSHOP: Educators give score based on lessons accomplished
- STRIKE: Team Leads track team members and score; awarding each member with the team score at the end of the day
- SPRINT: Moderator gives score based on tasks complete
- TESTDAY: Moderator gives score based on tasks complete
- See individual testplans for details on scoring for each day
Sign Up
Please put your name (or IRC nick) and the times you will be willing to help out. The roles are moderator, assistant, and contributor (see below for a more detailed description).
During the first day we will be running workshops to help community members gain the skills needed to successfully triage bugs, navigate Litmus, and write good test cases. Please sign up as an assistant if you would like to help me with moderator duties. Please sign up as an educator if you would like to help train community members.
During the next three days, we will apply the skills learned in the first day. This will give the community the hands on experience with resolving in-litmus bugs and writing test cases. Each day we will be focusing on a particular feature are of Firefox. Please sign up as an assistant if you would like to help me with moderator duties. Please sign up as a striker if you would like to be part of a strike team.
Each day we will be focusing on a particular feature area of Firefox. The following is a synopsis of each feature area:
Tuesday | Tabbed Browsing |
LEAD | juanb |
SUMMARY | Tabbed Browsing is a long standing feature in Firefox which allows users to browse multiple web pages in a single window by placing each page in its own tab. Today we will be vetting the current set of testcases, creating testcases for any in-litmus? bugs, and writing exploratory testcases for any fixed bugs. |
LINKS | Litmus Testcases, FIXED bugs, in-litmus? bugs |
Wednesday | Session Store |
LEAD | ashughes |
SUMMARY | Session Store is a long standing feature in Firefox which allows users to recover the browser from a crash. Today we will be vetting the current set of Litmus testcases and creating exploratory testcases against any fixed bugs. |
LINKS | Litmus testcases, FIXED bugs |
Thursday | Panorama |
LEAD | tracy |
SUMMARY | Panorama (aka Tab Candy) is a new feature in Firefox which allows users to group and organize their tabs. Today we will be creating testcases for any in-litmus? bugs, and creating exploratory testcases against any open bugs. |
LINKS | in-litmus? Bugs, Open Bugs, Litmus Testcases, Feature Summary |
Friday | Test Day |
MODERATORS | whimboo, juanb, ashughes, marcia, ... |
SUMMARY | We will hold a Testday on Friday, Sept 17. By then we should have candidate builds available for beta 6. There have been a ton of landings in several areas, but it has been suggested we concentrated on the Add-ons Manager. |
LINKS | Testday Details, Bugs |
During the final day, we will have a 4-hour contest. We will let the community members work on their own at their own pace. This will be the community's opportunity to work toward prizes. Please sign up as an assistant if you would like to help me with moderator duties. Please sign up as a sprinter if you will just be around to work on test cases and answer some community questions.
Please sign up for a timeslot and role:
Monday | Moderator | Assistant | Educators |
1800Z - 2000Z | ashughes | krupa | |
2000Z - 2200Z | ashughes | mw22 | tracy |
2200Z - 2400Z | ashughes | tracy |
Tuesday: Tabbed Browsing |
Moderator | Assistant | Strikers Lead: juanb |
1800Z - 2000Z | ashughes | Cameron Dawson | |
2000Z - 2200Z | ashughes | rbillings | |
2200Z - 2400Z | ashughes |
Wednesday: Session Store |
Moderator | Assistant | Strikers Lead: ashughes |
1800Z - 2000Z | tracy | ||
2000Z - 2200Z | tracy |
Thursday: Panorama |
Moderator | Assistant | Strikers Lead: tracy |
1800Z - 2000Z | ashughes | AaronMT | |
2000Z - 2200Z | ashughes | ||
2200Z - 2400Z | ashughes | tchung |
Friday | Moderator | Assistant | Sprinters |
1800Z - 2000Z | ashughes | stephend | |
2000Z - 2200Z | ashughes |
Roles-specific Tasks
- Moderating the Channel
- Great people as they join
- Make sure they have filled out the pre-bugday survey
- Make sure they have read the documentation and this test plan
- Be proactive and ask them if they know what they want to do. If not, give them something to do
- If they ask questions, answer them. If they don't ask questions, ask if they need help
- If they are working with a specific component, introduce them to the component owner
- Don't forget to broadcast the hourly advertisement in other IRC channels:
- QA is running a week-long bugzilla event to burn through bugs needing test cases. Want to help write test cases or learn how? Join us in #bugday
- Channels: #firefox, #developers, #qa
- Assisting the Moderator
- Be sure to greet newcomers and point them to the survey if they haven't already filled it out
- Be prepared to complete any of the above tasks
- Be proactive and ask if the moderator needs help
- If the moderator seems idle, be proactive and assist community members
- Cover for the moderator if a break is needed
- Assisting Contributors
- If someone asks a questions, help them as best you can
- If they don't ask questions, ask if they need help
- Review contributor test cases and give them advice on good writing techniques
- Help contributors navigate Litmus and Bugzilla by sharing your personal tips, tricks, and techniques
- Educators
- Make sure contributors are aware of and familiar with the documentation
- Make sure they know how to use the spreadsheet
- Answer questions
- Possible topics to teach to community:
- Finding a testcase in Litmus
- Anatomy of a testcase
- Understanding how to reproduce a bug
- Determining if a bug needs a testcase
- Strike Team
- Make sure teammates are familiar with the documentation, spreadsheet and bug list
- Help teammates work through the bug list
- Help teammates navigate Litmus to find existing test cases
- Review test cases added to the spreadsheet and give teammates feedback
- Make changes in Litmus on behalf of teammates and give them credit in the spreadsheet
- Answer any questions from contributors
- Sprinters
- Work through the spreadsheet adding/revising test cases in Litmus for contributors
- Answer any questions from contributors
- Component Owners
- Answer questions people may have specific to your component areas
Common Tasks
- Identify Steps to Reproduce
- Sometimes a bug will have clearly defined steps to reproduce it
- If the steps are not well defined, read through the comments to see if you can formulate your own steps
- If you are finding it difficult, ask for help from QA on IRC
- If all else fails, comment on the bug to get a clear set of steps from the developer:
- I'm having a hard time understanding the steps to reproduce this bug. Could someone please indicate how this bug was originally reproducible?
- Identify Expected Results
- Sometimes a bug will have a clearly defined result
- If the expected result (ie. the fix) is not well defined, read through the comments to see if you can formulate your own assumptions of what should happen
- If the bug has a well defined set of steps to reproduce, run through those steps and note the behaviour of the browser
- Ask someone from QA on IRC if they can help you verify what the expected result should be
- Ask a developer on the bug if
- I'm having a hard time understanding what the desired behaviour is now that this is fixed. Could someone please indicate more clearly what the expected result is?
- I ran through the steps and got this result, is that expected?
- Find existing test in Litmus
- Firstly, check with the component owner. She may know if a test case already exists from memory. If not, she will be able to direct you to the testgroup in Litmus.
- You could also check by going to the Firefox 3.6 test run or the Firefox 4 test run and looking for adequate test cases in the subgroup matching your component
- Otherwise, you'll have to do a manual search on litmus by going to view/search tests and use the following criteria:
- Product: Firefox
- Branch: 3.6 or 4.0 (you can get the version from the bug -- if unsure, assume 4.0)
- Testgroup: Select Full Functional Tests
- Subgroup: Select the subgroup matching the component of the bug
- Revising a test
- If a test case already exists but the behaviour has slightly changed, simply note in the spreadsheet the changes that need to be made
- Disabling a test
- If a test case already exists but is obsolete or no longer needed, simply note in the spreadsheet the test should be disabled
- Write a new test
- If the test case cannot be found and you think a new one needs to be written, simply write your steps and expected results in the spreadsheet
- Make sure you notify the component owner of the entry so that he can vet it and add it to Litmus
- Using the Spreadsheet
- Link
- Component Owners
- work using the Overview sheet
- track progress as you vet test cases and add/revise/disable in Litmus
- create a new sheet for you component for contributors to add test cases
- don't forget to flip the in-litmus flag
- Contributors
- add test cases to the specific component sheet
- if it's a revision, simply note the changes that need to be made
- if it's a test case that needs to be disabled, simply state the test no longer applies and should be disabled
- if no test case is needed for the bug, flip the in-litmus flag to in-litmus- (or ask the component owner to do it)
Documentation
Before writing any test cases you should familiarize yourself with our Test Case Writing Primer. It will tell you everything you need to know about writing good test cases.
The document is pretty extensive so try not to be overwhelmed or intimidated. Just try your best to understand and remember some of the key points it tries to convey. The following is a quick set of important guidelines you should take away from reading this document:
1. Be descriptive and concise
2. Make the steps easy to follow
3. Make the expected results clear to understand
4. Don’t get hung up on terminology
5. Write your test cases as if the reader has never used Firefox
6. Test case summaries should include a component header and briefly but clearly describe the test:
- [component:subgroup] Summary
- Example: [addons:themes] Install a theme using the Add-on Manager
7. Test case steps should be a numbered-list and expected results should be a bullet-list
- Example Test Case
[toolbar:customize] Add a Print icon to the toolbar | |
Steps:
1. Right click on the toolbar, and select “Customize” from the dropdown menu. |
Expected Results:
|
The above test could probably stand to be a bit more concise, but it is a "good" test case, in general.
Components & Owners
Component | Owner | IRC Nick |
Add-ons | Henrik Skupin | whimboo |
Audio & Video | Anthony Hughes | ashughes |
Awesomebar | Aakash Desai | aakashd |
Content Handling | Martijn Wargers | mw22 |
Cookies | Marcia Knous | marcia |
Downloading | Al Billings | abillings |
Find in Page | Juan Becerra | juanb |
Form Manager | Juan Becerra | juanb |
General | Tomcat | tomcat |
Geolocation | Marcia Knous | marcia |
Import | Marcia Knous | marcia |
Install/Migration | Marcia Knous | marcia |
Layout | Martijn Wargers | mw22 |
Library (History & Bookmarks) | Tracy Walker | tracy |
Lightweight Themes | Aravind | aravind |
Menu Bar | Henrik Skupin | whimboo |
Microsummaries | Aravind | aravind |
Online/Offline Events | Juan Becerra | juanb |
Options | Aakash Desai | aakashd |
Password Manager | Juan Becerra | juanb |
Plugins | Juan Becerra | juanb |
Popup & Annoyance Blocking | Al Billings | abillings |
Printing | Matt Evans | mevans |
Private Browsing | Marcia Knous | marcia |
RSS | Tomcat | tomcat |
Search | Henrik Skupin | whimboo |
Security | Anthony Hughes | ashughes |
Session Store | Anthony Hughes | ashughes |
Sync | Tony Chung | tchung |
Tabbed Browsing | Matt Evans | mevans |
Technical Tools | Aravind | aravind |
Toolbar Customization | Juan Becerra | juanb |