QA/Firefox3/TestPlan

Top Blocking issues for Beta 1

If you have any issues in these areas, please enter them here:

• Design still incomplete?
• Major bugs blocking your feature that should be addressed?
• Major bugs blocking overall regression of the trunk that should be fixed before shipping?

Collected Notes

• Critical Printing Bugs
• Bug 399238 and Bug 397428. Google maps prints horribly on windows and mac.
• Bug 324857 – MathML all screwed up in Cairo builds. Blocking MathML usage
• Download Manager designs are still changing at this stage. Testing has halted due to changes
• Addons Manager UI has a spec, but no dev resource or timeline to work on. Is this going to make Firefox 3?
• Java plugin bug is still open.
• Mac Focus issues still a problem.
• Accessibility: Download Manager, being redesigned
• Accessibility: Bookmark/Places Panel not Accessible, need menu items to duplicate toolbar entries. Beltzner says it's being worked on.

Overview

Firefox QA Test Strategy (draft)

The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.

Major QA Activities:

• Litmus Cleanup
• Litmus Triage Team
• New feature testing - see below
• L10n testing - TBD
• Milestone testing - TBD

[Firefox 3 Central]

Requirements and Ownership

• Fx3 Product Requirements
• Fx3 Dev team up-to-date feature tracking status

Archives:

• Firefox 3 TestPlan Tracking page (Google Spreadsheet)
• Fx 3 PRD Wiki page

Test Areas

Feature Focused Areas

Key: L = Large (3-6 weeks) M = Medium (2-4 weeks) S = Small (1-2 weeks)

Test Plan Template

Feature Area and Spec	Test Plan Location	QA Contact	Size	Description
Add ons	Addons Test Plan	tchung (new Addons Manager UI, Extensions), stephend (Unified Plugin Manager)	M	Add-on management system, plugin management system, localized user-facing addon text, configuration, updates, Installation, Discovery, FUEL 0.2
Content Handling	Microformat Test Plan Web Application Test Plan Download Manager Test Plan	ctalbert, stephend (Download Manager)	L	microformats, Content Type Processing, download manager, MIME type support, User Interface, Media Plugins (needs definition), firebug (at risk)
Partner Distribution	TBD	tracy, ctalbert	M	High-Touch/Low-Touch distros, regression, updates, test automation
OS Platform Integration	Vista Test Plan	Tomcat, marcia, ss/ispiked (breakpad)	M	Vista Support (At Risk), MSI bundling, Airbag / Breakpad, parental controls (At Risk), Notification, Vista Installer
Gecko Platform Integration	Gecko Platform Test Plan	martijn, marcia	M	Cairo graphics, nxTextFrame migration to Thebes, Reflow Branch, Caret painting rewrite, Cross-domain XMLHttpRequest (Cocoa widgets covered under Mac Test Plan)
Mac OS Platform Integration (includes Gecko)	Mac OS Integration Test Plan Leopard Test Plan	Tomcat, marcia	M	Leopard support, Cocoa implementation, widgets, form controls, Growl Integration on Mac, Mac Installer
Places	Places Test Plan	Tracy, ABillings	L	Bookmarks, History Indexing, Data Management, User Interface, Extensibility, Performance
Search / Tabbed Browsing / Visual Refresh	Search Tab / Visual Refresh Test Plan	Tomcat (search), Marcia (tabbed browsing)	S	User Interfacing, Animations, Data management
Security, Privacy / Password, Identity	Security / Privacy Test Plan Password Manager Testplan	Tomcat, juanb	L	password management, Phishing protection, Private web browsing, Blacklisting (At Risk), password Manager, Password Security
User Support	TBD	Tomcat, ss	S	Help content, Reporter, feedback, Customization options Now owned by SUMO team - David Tensor.
L10N Testing	L10N Test Plan	ctalbert, tchung	S	test automation, new locales, string lengths, bookmarks, rss, search plugins
UI Accessibility	UI Accessibility Test Plan	TimK	S	UI accessibility convention testing
Major Update Tests (2.0.x -> FF3)	TBD	tchung	S	update verification from 2.0.x builds to 3.0b1+. includes upgrades/downgrades/sidegrade cases

Regression Areas

• Top Web accessibility Tests
  • Plugins (eg. Flash, java, AJAX)
  • Extensions (eg. Firebug, NoScript, Fotofox)
  • Themes (eg. Vista, Joga, Simpsons)
  • Top Site Tests (eg. Alexa top 50)
  • Financial Sites (eg. banks, online trading)
• Accessibility Tests
• User Performance Testing
• Major / Minor Update Testing
• L10N Testing (may coincide with Feature area)
• Distribution Testing (may coincide with Feature area)

Ownership Needed Litmus Test Case Areas

These are Litmus test case areas that are currently unowned.

• RSS->marcia
• Printing -> Tomcat, tchung
• Location Bar -> Stephend
• Find -> TBD
• Customize Toolbar -> TBD
• Mac Leopard -> marcia

Test Results

Tests results in this section are recorded at each milestone. Current locale coverage is en-US only. They will be covering:

• Smoketests
• BFTs
• FFTs
• Focused testing (Test areas that have landed)

Alpha 4 Results
Alpha 5 Results
Alpha 6 Results
M7 Results
M8 Results
M9 Results
Beta 1 Results
Beta 2 Results

Daily Litmus Smoketest Schedule and Results

Daily Litmus Smoketest Schedule and Results

Test Planning Notes

Test Planning Notes

Extensions

For Firefox 2 we had Bug 353206 as extension Tracking bug.

The following extensions were never fixed from their extension compatibly authors to be able to install in Firefox 2.

• Slim Extension List [1]
• Bookmarks Synchronizer 3 [2]
• AutoMarks [3]
• Extended Link Properties [4]
• Snapper [5]
• Cacheout[6]
• Resize Search Box [7]
• superT 0.7.9.1 https://addons.mozilla.org/en-US/firefox/addon/2224
• Reveal extension [8]

Top Crash reports

TBD. ss to fill in chart here.

Banking and Financial Sites

Banking and Financial Sites

Results of Tab Switching Performance Tests

Tab Switching results

Additional References

New FF3 Feature list - An incomplete list of what Firefox 3 developers are working on.