Canmove, confirm, emeritus
3,627
edits
Changes
Initial version
== Where we are ==
We have two sets of automated tests:
* xpcshell-tests
* Leak and Bloat tests
More info [[MailNews:Home_Page#Automatic_Testing]].
We are also starting to write MozMill tests. At the moment we have a few tests checked into comm-central plus a runtest.py that will run those tests within Thunderbird if you have built Thunderbird in a dev environment.
== The Plans ==
=== xpcshell-tests ===
xpcshell-tests are now moving along nicely. As we have leak stats as well, there isn't much more to do, except for fixing the remaining reported leaks and writing new tests as we fix bugs.
I think expanding facilities such as fake server will most likely happen as there is demand for it.
=== MozMill and Leak & bloat tests. ===
We have a great opportunity here to start automated UI testing, something which we haven't been able to do up until now.
The rough plan is this:
# {{Bug|458352}} Ludovic has been working on extending the runtest.py that is used as an aid to run the MozMill tests. The two main things we're adding are:
#* support for different profiles when running tests
#* being able to generate leak and bloat logs just by specifying one or two command line arguments.
# {{Bug|500142}} The next step would be to make it so that runtest.py and the mozmill tests can be packaged via "make package-tests". This would mean that anyone could download a Thunderbird build and a test package and run the two together without build environment (python + mozmill would be the only requirements). This would also make it easier for non-developers to write tests.
# Once package-tests works, we can then set up additional tinderboxes (or just tinderbox steps) to download a build and a test package and run the two. With the package-tests step we can do this on release or debug builds (or both) hence providing confidence in our packaged release builds but also we can get debug output and leak stats.
# {{Bug|458352}} Once we've got that support, I'd like to replace the existing Leak and Bloat tests (which currently use js files installed in chrome) with a mozmill version. This will help with the extensibility of the tests and give us a better base for which we can start extend those tests {{Bug|458351}}.
#* This step currently blocked by {{bug|500201}} - MozMill is leaking lots of items. Hence moved to end.
We have two sets of automated tests:
* xpcshell-tests
* Leak and Bloat tests
More info [[MailNews:Home_Page#Automatic_Testing]].
We are also starting to write MozMill tests. At the moment we have a few tests checked into comm-central plus a runtest.py that will run those tests within Thunderbird if you have built Thunderbird in a dev environment.
== The Plans ==
=== xpcshell-tests ===
xpcshell-tests are now moving along nicely. As we have leak stats as well, there isn't much more to do, except for fixing the remaining reported leaks and writing new tests as we fix bugs.
I think expanding facilities such as fake server will most likely happen as there is demand for it.
=== MozMill and Leak & bloat tests. ===
We have a great opportunity here to start automated UI testing, something which we haven't been able to do up until now.
The rough plan is this:
# {{Bug|458352}} Ludovic has been working on extending the runtest.py that is used as an aid to run the MozMill tests. The two main things we're adding are:
#* support for different profiles when running tests
#* being able to generate leak and bloat logs just by specifying one or two command line arguments.
# {{Bug|500142}} The next step would be to make it so that runtest.py and the mozmill tests can be packaged via "make package-tests". This would mean that anyone could download a Thunderbird build and a test package and run the two together without build environment (python + mozmill would be the only requirements). This would also make it easier for non-developers to write tests.
# Once package-tests works, we can then set up additional tinderboxes (or just tinderbox steps) to download a build and a test package and run the two. With the package-tests step we can do this on release or debug builds (or both) hence providing confidence in our packaged release builds but also we can get debug output and leak stats.
# {{Bug|458352}} Once we've got that support, I'd like to replace the existing Leak and Bloat tests (which currently use js files installed in chrome) with a mozmill version. This will help with the extensibility of the tests and give us a better base for which we can start extend those tests {{Bug|458351}}.
#* This step currently blocked by {{bug|500201}} - MozMill is leaking lots of items. Hence moved to end.
