User:Tsk/Docs/QA Plan

Gross Initial QA plans/ Thoughts which needs to be correlated with all the other bits and pieces floating around MailNews:Home_Page and Thunderbird wiki pages. I need to start somewhere so here are the thoughts I've been having on how the QA should be done for Thunderbird. There are three main areas in which the QA effort should be divided.

Community / triage / testing

Triagging bugs making sure that each bugs is being taken care of is a huge taks when many bugs are being filled daily. In order to keep the bug database clean we need more man power to be able to do the following tasks :

• Triagge new bug
• Follow-up on old bugs
• Clean up the database which contains very old bug that were never being taken care of.

And for this there is no automatic alternative It can only be done by hand (the other choice being to force the bug closure for bugs prior to date XX/XX/XXXX - but some of those bugs might still be valid - this is for now not an option for me). In order to get more people we need to tell them that we are in the need for people willing to spend time reading, commenting and changing bugs.

• for that we can use the actual mechanism -> blog post, newsgroup post ... (and at some point we should)
• Interact with other communities that might have an interest in the product :
  • Linux people -> OpenSuse and Ubuntu are good candidates
  • Extension users : people using Lightning, people using enigmail
  • OpenSolaris, as solaris ships with Gnome and moz apps as defaults.

Daily triage

So triagge is done in many different ways. Some people do it per component - some people do it once in a while. Would be nice to come up with a team that would be big enough to make sure that all bug entered in a single day would get some attention, and be triagged right away. To do so we need more people and we need to train them. Ideally I would like that each bug that gets entered on a daily basis get some attention and love.

Bug days

Will not change until we reach a community of people doing QA that would be big enough for the bug days to become training days for new comers or for triagging the left overs untriagged bugs that are in bugzilla.

Test days

Test days are here to test new features. Might also be a great way to improve coverage by adding some tests into litmus while some testing is happening. So two things would be happening during bug days, coverage would get bigger in our TC management tools and testing and bugs would be filled.

Bugzilla Clean Up

Bugzilla is full of old bugs related to old releases of thundrbird this needs to be cleaned up - this also means that you needs people that understand the product enough to do it. This is not High priority at the moment.

Tooling / Test Cases/ Reports

Tooling

Playing with Litmus and QMO. Litmus is slow but has some stuff in it. Need to get more information on what's the mozilla plan for that.

Test Cases

We definitively need more TC, and some of them are dups - (I also need to spend a lot more time in litmus) - Need to get some information on Mozilla's qa plans for litmus in 2009 and act accordingly. Also need to groups TC - Initial Idea would be something like - can't ship without - Shouldn't ship without. We could also group based on what the candidate is - Testing needed for beta, testing needed for RC, testing needed for etc .... Would also be nice to know against what server the test is being run - ie Zimbra, Gmail etc ... Nice to have would be where to file the bug in bugzilla - ie if TC 1234 fails -bugzilla xxx where the component is already selected. I need to figure out how much % we cover of he application.

Reports

• We needs stats on bugs - (make sure to save those URLs)
• We needs stats on Coverage and what has been covered for a particular release.
• Stats on memory usage - need to figure out how to exploit them