Firefox:Automated Builds:Test Plan

General Firefox Automated Builds Test Plan

Description of Release

The build team has been working to generate builds via automation. This automation removes the necessity of human intervention during a normal build process. This is the first time that this has been done but this process will be used for the 2.0.0.7 release.

In order to validate that the builds that are generated have no issues, the build team has asked that QA run basic tests on the builds.

Specific Test Plan for this Release

QA will run smoke tests on all of the standard operating systems (Windows XP, Windows Vista, Mac OS X, and Linux). BFTs will be run on Windows XP.

In addition to this testing, the build team has specifically requested that additional testing be done with add-ons because of the nature of the automated builds. Top add-ons will be tested on Windows XP, Mac OS X, and Linux.

Additionally, localization should be spot checked as well since localized builds are being generated. This can be done through the Litmus l10n-localization test suite.

Builds

Because of naming and versioning requirements, the automated build was generated as a 2.0.0.7 RC1 build. It is not the 2.0.0.7 RC1 but lives in that directory on ftp.mozilla.org:

• ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2.0.0.7-candidates/rc1/

Assumptions

The automated build should behave exactly the same as the normal 1.8 nightly builds from 2007-08-21, but with official branding (eg Firefox rather than Bon Echo), and release update channel instead of nightly.

Current Tasks

• Smoke Tests:
  • Windows XP (tomcat) - PASS
  • Windows Vista (abillings) - PASS
  • Mac OS X (tchung) - PASS
  • Linux (ashughes) - PASS
• BFTs
  • Windows XP (tomcat) - PASS
  • Mac OS X (tomcat) - PASS
  • Linux (tomcat) - PASS
• FFTs
  • Windows XP (tomcat) - PASS

Schedule

• First Builds Available: 8/23
• Testing: 8/24, 8/27

Test Results

• Smoketests, BFTs, and FFTs
• l10n related testing
• Addon Test Results