QA/Fennec

Home of those who want to Propel Fennec Quality!

Milestones

Current Releases

• 8.0: Testplan
• 7.0: Testplan
• 6.0: Testplan

Past Releases

• 5.0: Testplan
• 2.0: Testplan
• 1.1: Testplan
• 1.0: Testplan

Planning

• 2011
• 2010

What We Do

Testing Strategy

Manual:

* Daily Smoketests on the latest nightly build on the current development branch and trunk
   1. Verifications of overnight resolved bugs on the current development branch
   2. Run BFT(s)/Focused testing around areas that are affected by resolved bugs
* Weekly triage of feedback received via the Fennec Reporter. URLs to the current lists can be found here: 1.1a2pre and 1.0.0
* Weekly triage of filed bugs and those that are nominated and in a New state
* Before string freeze, update the String Guide in the latest Localizer Test Run with new features added for that release
* Bi-Quarterly Testdays

Automation:

* Unittests on Fennec
   1. Mochitests on Fennec
   2. Reftests on Fennec
* For more information, take a look at the TestDev Wiki for Mobile

Unique Areas of Testing:

* Test Guide on Daily Browsing
* Website Compatibility
* Device Integration Checklist
* OS Integration Checklist
* Updates Testing

Release Testing

• Mobile Release Checklist
• Template for results tracking page

Release	Smoketest	BFT	Blocker Verifications	L10n	Website	Updates
Early-Stage Betas	YES	NO	YES	1 RTL, 1 non-Latin, 1 Multi-L10n Switch	NO	YES
Late-Stage Betas (3+)	YES	YES	YES	1 RTL, 1 non-Latin, Full Multi-L10n Switch	NO	YES
Major Releases	YES	YES	YES	2 RTL, 2 non-Latin, Full Multi-L10n Switch	YES	YES

What We Use

Tools

• Reporter for Fennec: Extension to allow users to inform Mozilla of problems during their browsing session. For triagers, reports are filed to the following queries, ____ and ____.
• QA Companion: Currently allows users to painlessly report bugs if they don't have a desktop on hand.
• Litmus Fennec 1.0 Test Run
• Litmus Fennec 1.0 Localizer Test Run
• Performance Graphs

Resources

Device Anywhere Tutorial

• Device Anywhere Tutorial

Mobile (HG) Repository

• mozilla-central

Mobile Source Code

• Fennec MXR Source Code Web-Based Viewer

Builds

• Mobile Nightly Builds
• Hourly Tinderbox Builds
• Fennec L10n Dashboard

Where Do We Need Help

Test Tool Development

• Write automated testcases
• Port Automation Frameworks to WinCE
• Find, Fix, Verify UnitTest Failures

Execution

• Triage Unconfirmed Fennec Bugs
• Create Litmus testcases for bugs flagged as in-litmus?
  • Fennec Component Ownership in Litmus
• TestDay Promotions within Mobile Community Channels
• Write technical documentation to perform quality execution

Performance Benchmarking

• Scratch pad of notes
• Performance Testplan

Website Compatibility Testing

• Scratch pad of notes
• Web Compatibility Testplan

Sync Performance Instrumentation

Fennec sync continues to cause performance issues on first time syncing with very large amounts of data. How you can help, is to provide sync logging through a custom extension, and submit your data to the Mozilla Services team. Detailed instructions can be found here.

More historical information on the project:

• Philikon's Sync Instrumentation Research
• Blog on profiling Javascript Objects

How You Can Contact Us

If you're interested in helping out with anything below (or want to help with something else), please contact jmaher or aakashd. If you want to talk to the Mozilla Mobile Community, you can find us in our IRC Channel.

• Mailing Lists
  • mobile@mozilla.com
• Newsgroups
  • mozilla.dev.platforms.mobile
  • mozilla.dev.quality
• 3rd Party Testing (Waverley)
  • Execution testplan for Smoketests & BFTs