Thunderbird:Testing/Testing

From MozillaWiki
Jump to: navigation, search
Quality through testing

One of the easiest way to help make Thunderbird a great product is by testing it.

Here are several different ways you can help.

Punctual testing

This may be the easiest way to participate, learn how to do more - and get more confident about helping. To help here, just subscribe yourself to the Thunderbird-testers mailing list. This is a read-only mailing list with one or two messages per month. We usually send detailed instructions on what to do and test and how to get help when you sign up for one of those events.

Once on the mailing list just read the emails coming in. They'll either ask for volunteers for a shared event or will give very specific instructions on how to test a new feature on which we want a lot of coverage. When we ask for volunteers we'll usually use the following tools for testing :

Having accounts on both on these systems is needed in order to participate.

Finding regressions

While testing you might find some portions of Thunderbird that used to work, but do not work anymore. These are called regressions.

Daily testing

To participate in this effort, you'll need to run a version different from the standard release version of Thunderbird linked on the mozilla.com website.

Daily testing is can be done with "newer" versions of Thunderbird that needs testing. Mozilla produces updates of these versions of Thunderbird every night, with new fixes, but also sometimes causing new bugs.

  • We need many people on the stable branch - as its name implies, the stable branch only contains fixes and doesn't see any new features. But by using these builds, you'll help make sure that we don't introduce new bugs in the next security and stability release. To use these, either:
    1. Download one stable nightly build and update it regularly
    2. Or switch your update channel to beta instead of release.
  • We also need people to use Alphas and Betas of the next major release. To run these just download one when they are announced and just keep using them.
  • And finally we also need people to run trunk builds. These builds will contain all the new features of the next major release of Thunderbird and you don't need to wait for the next beta to see them. These builds are less stable than the others and get broken once in a while (I would say maybe once or twice per calendar year).
    1. We also announce major issues when we are aware of them on Tb-planning - Some coverage about less critical issues is available on the Daily build thread at Mozillazine

When doing daily testing, it's a good idea to have an account on Bugzilla to log the issues you find (and please make sure to search bugzilla before filling new defects)

Maintaining the Test cases

A main part of our manual and community driven testing relies on Test cases. What we do is we publish a call for help from volunteers and then split the test cases between those volunteers.

Having a nice set of up to date tests is important. The work here involves litmus, which is our tool to manage test cases. We have a fairly large number of tests and we need to maintain them as the product evolves.

This means :

  • Adding tests
  • Editing tests
  • Removing obsolete tests

Not too technical, but a good level of English is needed.