QA/Fennec

Home of those who want to Propel Fennec Quality!

Milestones

Current Releases

• 8.0: Testplan
• 7.0: Testplan
• 6.0: Testplan

Past Releases

• 5.0: Testplan
• 2.0: Testplan
• 1.1: Testplan
• 1.0: Testplan

Planning

• 2011
• 2010

What We Do

Testing Strategy

Manual

• QA/Fennec/Waverley on the latest [nightly.mozilla.org/mobile.html Nightly build] on the current development branch and trunk
• Verification of overnight resolved bugs on the current development branch
• Run BFT(s)/Focused testing around areas that area affected by resolved bugs
• Weekly triage of new and unconfirmed bugs
• Bi-Monthly Testdays

Automation

• Browser Chrome, Reftests, Crashtests, Talos Performance Tests

Unique Areas of Testing

Project	Description
Device Compatibility Test Plan	Tracks Tests and Full device list of compatibility with Mobile Firefox
Website Compatibility Test Plan	Tracks top Website compatibility with Mobile Firefox
Performance Benchmarking Test Plan	Tracking performance benchmarking numbers for Startup times and Page Loads
Mobile Addon for Unit Testing Automation	Project to run unit tests via a test-harness addon and disperse via crowdsourcing techniques

Release Testing

• Mobile Release Checklist
• Template for results tracking page

Release	Smoketest	BFT	Blocker Verifications	L10n	Website	Updates
Early-Stage Betas	YES	NO	YES	1 RTL, 1 non-Latin, 1 Multi-L10n Switch	NO	YES
Late-Stage Betas (3+)	YES	YES	YES	1 RTL, 1 non-Latin, Full Multi-L10n Switch	NO	YES
Major Releases	YES	YES	YES	2 RTL, 2 non-Latin, Full Multi-L10n Switch	YES	YES

What We Use

Tools

Litmus

• Fennec Test Runs

Device Anywhere

• DeviceAnywhere

Performance Graphs

• Performance Graphs

Resources

Device Anywhere Tutorial

• Device Anywhere Tutorial

Mobile (HG) Repository

• mozilla-central

Mobile Source Code

• Fennec MXR Source Code Web-Based Viewer

Builds

• Mobile Nightly Builds
• Hourly Tinderbox Builds
• Fennec L10n Dashboard

Where Do We Need Help

Test Tool Development

• Write automated testcases
• Port Automation Frameworks to WinCE
• Find, Fix, Verify UnitTest Failures

Execution

• Triage Unconfirmed Fennec Bugs
• Create Litmus testcases for bugs flagged as in-litmus?
  • Fennec Component Ownership in Litmus
• TestDay Promotions within Mobile Community Channels
• Write technical documentation to perform quality execution

Performance Benchmarking

• Scratch pad of notes
• Performance Testplan

Website Compatibility Testing

• Scratch pad of notes
• Web Compatibility Testplan

Sync Performance Instrumentation

Fennec sync continues to cause performance issues on first time syncing with very large amounts of data. How you can help, is to provide sync logging through a custom extension, and submit your data to the Mozilla Services team. Detailed instructions can be found here.

More historical information on the project:

• Philikon's Sync Instrumentation Research
• Blog on profiling Javascript Objects

How You Can Contact Us

If you're interested in helping out with anything below (or want to help with something else), please contact jmaher or aakashd. If you want to talk to the Mozilla Mobile Community, you can find us in our IRC Channel.

• Mailing Lists
  • mobile@mozilla.com
• Newsgroups
  • mozilla.dev.platforms.mobile
  • mozilla.dev.quality
• 3rd Party Testing (Waverley)
  • Execution testplan for Smoketests & BFTs