Calendar:QA TestDay:2006-10-31

From MozillaWiki
Jump to: navigation, search

<< Back to Calendar QA TestDays

RESULTS!

  • 57 total bugs addressed
  • 3 new bugs found

We had a successful bug day on the 31st. We were able to confirm about 20 bugs, and we addressed (asked for more information, reassigned components, works for me, verified fixes etc) over 30 bugs. In the course of the day, we found 3 new bugs.

Thanks to everyone that helped out. The list of helpers has gotten so long, I'm sure I'll miss someone, and I apologize. But, thanks to: jminta, lilmatt, ssitter, celina63, sebo, nightrat, xfallenangel for moderating and helping set up the test day. Thanks also to all the folks that participated: lambert, worzel, mc, mschroeder, unlogic, koen, damian, mattheus123.

Congratulations to our Winners!

Once again, our friends at Linpro AS have agreed to sponsor the prizes for the test day. This was a close race and we nearly had a tie, but the winners are:

  • mschroeder
  • damian


When is the test day?

Where do I start?

OK, I am in the channel, now what?

Let the moderators know if you have any questions or encounter any problems. If you are not familiar with our bugzilla bug tracking system, feel free to report your problems to a moderator and they can log any bugs for you.

Who will be moderating the #calendar-qa channel on the Test Days?

The moderator's nicks for this test day will be:

  • ctalbert
  • ssitter
  • lilmatt
  • dmose
  • xFallenAngel
  • sebo
  • jminta

What build should I test?

We will be testing the latest nightly builds of Sunbird and Lightning:

Sunbird:

http://ftp.mozilla.org/pub/mozilla.org/calendar/sunbird/nightly/latest-trunk/

Lightning (for Thunderbird 1.5.x and 2.0 candidates):

  • Note: to find the Lightning.xpi, browse to your Operating System subfolder from this directory.

http://ftp.mozilla.org/pub/mozilla.org/calendar/lightning/nightly/latest-mozilla1.8/

What specifically will we be testing?

We will be working to clean out the bugzilla backlog so that we can begin testing the new functionality that will soon be landing for the next calendar version.

We need to:

  • Confirm unconfirmed bugzilla bugs - ensure that the bugs are reproducible and not duplicates of known issues
  • Verify existing fixes - ensure that the fixes did fix the problem in the bug report and did not cause other regressions
  • Investigate defects that are flagged as QAWanted - provide more details, analysis, and clearer or simpler steps to reproduce the issue.

If you've already backed up your profile, you can skip directly to the Testing Instructions area.

BEFORE YOU START--BACKUP YOUR PROFILE

Once you have downloaded the version of the application you want to test, but before you start it, you MUST back up your profile and create a testing profile, otherwise, you run a very good chance of losing your data.

Backing up a profile is easy:
1. Find your profile directory:

OS Thunderbird Profile Sunbird Profile
Windows  %AppData%\Thunderbird  %AppData%\Mozilla\Sunbird
Linux ~/.thunderbird ~/.mozilla/sunbird
MacOSX ~/Library/Thunderbird ~/Library/Application Support/Sunbird

2. Copy the root of it somewhere else, for example copy the Thunderbird or Sunbird directory and everything below it.

Creating a testing profile is also easy:

  1. Launch your application with a -P option
  2. Create a New Profile by clicking the Create a New Profile button
  3. Click Next, Give it a name like "Testing Profile", click Finish
  4. Select the testing profile and click the "Start Calendar" (Or Start Mail in Thunderbird) button.

Remember that you will have to start the application with the -P option during testing so you can pick your Testing profile

Once you've finished testing, you can delete the extra profiles and you'll be back to normal.

If something goes terribly wrong (although we don't expect it to), close the application and just replace the profile with the backed up copy. That will reset the application to the way it was before you started testing.

Testing Instructions

  1. Download Sunbird or Lightning from the above URLs
  2. Create your testing profile (see above)

To Confirm Unconfirmed Bugs:

  1. Click on a Bugzilla query below for a component that you would like to work on.
  1. Pick a bug that sounds interesting (confirm NORmal bugs before ENHancements), and tell one of the moderators in the channel that you're working on it. (They will let you know whether or not someone's already taken it).
  2. To confirm the bug, follow these steps:
    1. Do some bugzilla searching first to see if the bug might be a duplicate of an existing issue. If it is a duplicate, please add a comment stating the existing bug that this is a duplicate of, and include your IRC nickname. Also, let a moderator know in the channel, so that they can change its status to DUPLICATE.
    2. Attempt to perform the steps reported using a nightly build from above. Be certain that you can reproduce the bug using the nightly build.
    3. If you were able to reproduce the bug in the last step, then add a comment to the bug containing detailed steps on how you confirmed the bug. Be certain to include the buildID of the nightly (see table below), your operating system, the steps you followed, your IRC nickname (for Test Day tracking), and any other information that might be pertinent to the developer when they attempt to fix it.
    4. If you have bugzilla priviledges, please move the bug to the "New" state, if not, alert one of the moderators and they will do it for you.

There is more information about confirming bugs here. 

Build ID for October 31 Test Day
--------------------------------
OS      | Sunbird    | Lightning
Linux   | TODO | TODO 
Mac     | TODO | TODO
Windows | TODO | TODO


To Verify Fixed Bugs:

  1. Select a bug off of this list to verify.
  2. Attempt to reproduce the bug using the operating system that the bug was reported in.
  3. If you cannot reproduce the bug, please mark that with a comment in the bug (be sure to include your IRC nickname), and let the moderators know in the channel that you believe the bug to be fixed.
  4. If a bug has been sufficiently tested on the operating systems that it was reported on, please mark the bug as VERIFIED, or ask one of the moderators of the channel to do that for you.

To aid with QA wanted Bugs:

  1. If you have not worked with Mozilla QA or the Calendar QA team before, we do ask that you work closely with a core calendar QA team member when working with QA Bugs. It is crucial that the very best feedback is supplied for QA Wanted items.
  2. Test the bug and confirm that it still occurs
  3. Come up with suggestions of why it occurs, clear steps to reproduce, error messages, ICS files that exhibit the behavior, etc. Try to include as much relevant information as you possibly can about the bug.
  4. Ensure it is filed against the proper component and note what build you had to use in order to reproduce the bug.

BUG FOUND!

  1. In the process of working with the Bugzilla bugs, you may find another bug. When you think you've found a bug, try to reproduce it. Restart Sunbird/Lightning and do the same steps again. The most valuable bugs are those that can be reproduced.
  2. Once the bug has been reproduced, search for it or something similar on Bugzilla. This can easily be done by searching for the error message, window name, or some other identifying feature of the bug. If you found an existing bug that is just like the bug you found, read the comments. Maybe your situation is slightly different and your input could help developers find the origin of the bug.
  3. If you can't find the bug in Bugzilla, talk about it in the channel. Maybe others can confirm your bug or find a similar bug in bugzilla. This also gives moderators a chance to guide you to either provide more details, update an existing bug or file a new bug.
  4. If you are certain you found a unique bug and have read the Bug Writing Guidelines and Bugzilla Etiquette, go ahead and file the bug in Bugzilla. Remember to be as specific as possible.

Some other ways you can help:

  • Think of some test cases that aren't already included in Litmus (we love suggestions!).
  • Ask your friends to join you. The more people help the better!

Where do I find Litmus and Bugzilla?

  • Bugzilla can be found at http://bugzilla.mozilla.org/
  • A list of Calendar specific bugs (without enhancements) can be found here
  • A list of already filed enhancement requests can be found here

Tips:

  • Make sure you enter the correct information for your platform, version of Thunderbird, Lightning etc. Please include the full build identifier (i.e Thunderbird 1.5.0.7 (20060909) with Lightning 0.3 (2006100206)).
  • Be sure to read the Testing Instructions above before filing a bug.
  • Feel free to ask on IRC if you need help or have any more questions.

How do I send feedback?

Please use email us at dev-apps-calendar at lists.mozilla.org to send feedback about how the testday went, any overall thoughts about Sunbird and Lightning, and to follow up on issues found and bugs filed. We really appreciate your help today and your feedback is very valuable.

Moderator Instructions

To keep track of who is working on what, we'll make use of firebot's memory. That way we will have a current listing regardless of when we log into the channel, and we don't have to have each moderator give the next moderator a list of who's doing what. Testers, if no moderator is available, you can perform these actions yourselves as well.

  • Let's say JohnDoe logs into the channel. JohnDoe wants to work on bug 334455.
  • The moderator will write:
firebot bug334455 is JohnDoe
  • This will cause firebot to associate "bug334455" with "JohnDoe".
  • Later, if Jane logs into the channel, and she wants to work on bug 334455, the moderator can ask firebot: 
firebot who is bug334455.
  • And firebot will respond:
bug334455 is JohnDoe
  • At that point, the moderator can direct Jane to a different bug.
  • When JohnDoe confirms bug 334455, the moderator should tell firebot to forget the association by typing this into the channel: 
firebot forget bug334455

Tip: Do NOT use a space between "bug" and the bug number. If you do that, firebot will misunderstand your request and respond with the bug information.

Rewards!

Our friends at Linpro AS have decided to sponsor one more test day. There will be a first place and second place reward. They will be awarded to the two people that confirm and/or verify the most bugs. Moderators will not be eligible for prizes, so this gives everyone in the community an equal chance of winning.

When you edit a bug to confirm or verify it, please mention your IRC nick name in the comment so that your contribution will be properly counted.

Scoring

I have never mentioned it previously, but we use a simple point system to determine the winners of these Test Days. The number of points awarded for each action changes based on the focus of the day. For this day, it will work like this:

  • You will get 2 points for each bug confirmed
  • You will get 2 points for each bug verified
  • If you find a new bug, you will get 1 point. (This is a low number because the focus of the day is on confirming and verifying bugs).

The two people with the most points (who have confirmed and/or verified the most bugs) will win the prizes.

Retrieved from "https://wiki.mozilla.org/index.php?title=Calendar:QA_TestDay:2006-10-31&oldid=43501"