Thunderbird:Testing/Testing: Difference between revisions

From MozillaWiki
Jump to navigation Jump to search
 
(11 intermediate revisions by 5 users not shown)
Line 1: Line 1:
<center>Quality through testing</center>
<small>This article was moved inside [[Thunderbidr:Testing]]</small>


One of the easiest way to help make Thunderbird a great a great product is by testing it.
<center>Quality through testing


One of the easiest way to help make Thunderbird a great product is by testing it.
Here are several different ways you can help.</center>
=== Keywords ===
* testace<ref>[https://en.wikipedia.org/wiki/Test_case]</ref>
== Punctual testing ==
== Punctual testing ==


This is the easiest way to participate, and learn how to do more - and get more confident about helping.
This may be the easiest way to participate, learn how to do more - and get more confident about helping. To help here, just subscribe yourself to the [https://mail.mozilla.org/listinfo/thunderbird-testers Thunderbird-testers] mailing list. This is a read-only mailing list with one or two messages per month. We usually send detailed instructions on what to do and test and how to get help when you sign up for one of those events.
 
To help there , just subscribe yourself to the [https://mail.mozilla.org/listinfo/thunderbird-testers Thunderbird-testers] mailing list. This is a read-only mailing list with one or two messages per month.


Once on the mailing list just read the emails coming in. They'll either ask for volunteers for a shared event or will give very specific instructions on how to test a new feature on which we want a lot of coverage. When we ask for volunteers we'll usually use the following tools for testing :
Once on the mailing list just read the emails coming in. They'll either ask for volunteers for a shared event or will give very specific instructions on how to test a new feature on which we want a lot of coverage. When we ask for volunteers we'll usually use the following tools for testing :


* [https://litmus.mozilla.org Litmus] to manage our [https://secure.wikimedia.org/wikipedia/en/wiki/Test_case testcase]
* [https://moztrap.mozilla.org/ MozTrap] to manage our [https://secure.wikimedia.org/wikipedia/en/wiki/Test_case testcase]
* [https://bugzilla.mozilla.org Bugzilla] to manage our defects and [https://secure.wikimedia.org/wikipedia/en/wiki/Software_bug bugs]
* [https://bugzilla.mozilla.org Bugzilla] to manage our defects and [https://secure.wikimedia.org/wikipedia/en/wiki/Software_bug bugs]


Having accounts on both on these system is needed in order to participate.
Having accounts on both on these systems is needed in order to participate.
 
We usually send detailed instructions on what to do and test and how to get help when you sign up for one of those events.


=== Finding regressions ===
While testing you might find some portions of Thunderbird that used to work, but do not work anymore. These are called [[Regression_Testing|regressions]].


== Daily testing ==
== Daily testing ==


To participate in this effort, you'll need to run different versions of Thunderbird than the one linked on the mozilla.com website.
To participate in this effort, you'll need to run a version different from the standard release version of Thunderbird linked on the mozilla.com website.


Daily testing is can be divided by [[Thunderbird/Active_Branches |versions of Thunderbird]] that needs testing. Mozilla produces versions of Thunderbird every night, with new fixes and sometimes new bugs.
Daily testing is can be done with [https://www.mozilla.org/en-US/thunderbird/channel/ "newer" versions of Thunderbird] that needs testing. Mozilla produces updates of these versions of Thunderbird every night, with new fixes, but also sometimes causing new bugs.


* We need a number of people on the stable branch - as its name implies, the stable branch only contains fixes and doesn't see any new features. But by using these builds, you'll help make sure that we don't introduce new bugs in the next security and stability release. To use these either:
* We need many people on the stable branch - as its name implies, the stable branch only contains fixes and doesn't see any new features. But by using these builds, you'll help make sure that we don't introduce new bugs in the next security and stability release. To use these, either:
*# Download one stable nightly build and update it regularly
*# Download one stable nightly build and update it regularly
*# Or switch your update channel to beta instead of release.
*# Or switch your update channel to beta instead of release.
* We need people to use Alphas and Betas of the next major release. To run these just download one when they are announced and just keep using them.
* We also need people to use Alphas and Betas of the next major release. To run these just download one when they are announced and just keep using them.
*# When we are about to release a new version of Thunderbird we usually announce a  beta release on [https://mail.mozilla.org/listinfo/tb-planning Tb-planning] - which means you'll get updated within 48 hours.
* And finally we also need people to run trunk builds. These builds will contain all the new features of the next major release of Thunderbird and you don't need to wait for the next beta to see them. These builds are less stable than the others and get broken once in a while (I would say maybe once or twice per calendar year).
* And finally we also need people to run trunk builds. These builds will contain all the new features of the next major release of Thunderbird and you don't need to wait for the next beta to see them. These builds are less stable than the others and get broken once in a while (I would say maybe once or twice per calendar year).
*# We also announce major issues when we are aware of them on [https://mail.mozilla.org/listinfo/tb-planning Tb-planning] - Some coverage about less critical issues is available on the [http://forums.mozillazine.org/viewforum.php?f=29 Daily build thread at Mozillazine]
*# We also announce major issues when we are aware of them on [https://mail.mozilla.org/listinfo/tb-planning Tb-planning] - Some coverage about less critical issues is available on the [http://forums.mozillazine.org/viewforum.php?f=29 Daily build thread at Mozillazine]
Line 37: Line 39:
== Maintaining the Test cases ==
== Maintaining the Test cases ==


A main part of our manual and community driven testing relies on [https://secure.wikimedia.org/wikipedia/en/wiki/Test_case Test cases]. What we do is we publish a call for help from volunteers and then split the test cases between those volunteers.
A main part of our manual and community driven testing relies on test cases .<ref name="testcase">https://secure.wikimedia.org/wikipedia/en/wiki/Test_case</ref>. What we do is we publish a call for help from volunteers and then split the test cases between those volunteers.
 
Having a nice set of up to date tests is important. The work here involves [https://litmus.mozilla.org litmus], which is our tool to manage test cases. We have a fairly large number of tests and we need to maintain them as the product evolves.


This means :
This means :
Line 48: Line 48:


Not too technical, but a good level of English is needed.
Not too technical, but a good level of English is needed.
== References ==
<references/>

Latest revision as of 21:50, 15 January 2024

This article was moved inside Thunderbidr:Testing

Quality through testing

One of the easiest way to help make Thunderbird a great product is by testing it.

Here are several different ways you can help.

Keywords

  • testace<ref>[1]</ref>

Punctual testing

This may be the easiest way to participate, learn how to do more - and get more confident about helping. To help here, just subscribe yourself to the Thunderbird-testers mailing list. This is a read-only mailing list with one or two messages per month. We usually send detailed instructions on what to do and test and how to get help when you sign up for one of those events.

Once on the mailing list just read the emails coming in. They'll either ask for volunteers for a shared event or will give very specific instructions on how to test a new feature on which we want a lot of coverage. When we ask for volunteers we'll usually use the following tools for testing :

Having accounts on both on these systems is needed in order to participate.

Finding regressions

While testing you might find some portions of Thunderbird that used to work, but do not work anymore. These are called regressions.

Daily testing

To participate in this effort, you'll need to run a version different from the standard release version of Thunderbird linked on the mozilla.com website.

Daily testing is can be done with "newer" versions of Thunderbird that needs testing. Mozilla produces updates of these versions of Thunderbird every night, with new fixes, but also sometimes causing new bugs.

  • We need many people on the stable branch - as its name implies, the stable branch only contains fixes and doesn't see any new features. But by using these builds, you'll help make sure that we don't introduce new bugs in the next security and stability release. To use these, either:
    1. Download one stable nightly build and update it regularly
    2. Or switch your update channel to beta instead of release.
  • We also need people to use Alphas and Betas of the next major release. To run these just download one when they are announced and just keep using them.
  • And finally we also need people to run trunk builds. These builds will contain all the new features of the next major release of Thunderbird and you don't need to wait for the next beta to see them. These builds are less stable than the others and get broken once in a while (I would say maybe once or twice per calendar year).
    1. We also announce major issues when we are aware of them on Tb-planning - Some coverage about less critical issues is available on the Daily build thread at Mozillazine

When doing daily testing, it's a good idea to have an account on Bugzilla to log the issues you find (and please make sure to search bugzilla before filling new defects)

Maintaining the Test cases

A main part of our manual and community driven testing relies on test cases .<ref name="testcase">https://secure.wikimedia.org/wikipedia/en/wiki/Test_case</ref>. What we do is we publish a call for help from volunteers and then split the test cases between those volunteers.

This means :

  • Adding tests
  • Editing tests
  • Removing obsolete tests

Not too technical, but a good level of English is needed.

References

<references/>