QA/Firefox3.5/TestPlan

« Firefox3.5

Top Critical issues for QA

This is a working list that QA is using to request be fixed or completed for the Upcoming Fx3 Milestones.

If you have untracked issues in these areas, please enter them here:

• Design still incomplete?
• Major bugs blocking your feature that should be addressed?
• Major bugs blocking overall regression of the trunk that should be fixed before shipping?

Overview

Firefox 3 QA Test Strategy

The goal of this Overall Test Plan is to lay out our testing strategy, communicate it to others, and track the status.

Major QA Activities:

• New feature testing - see below
• L10n testing
• Milestone testing

Firefox 3.5 Central

Requirements and Scheduling

• Fx3.5 Firefox Product Requirements
  • A more update list of features
• Fx3.5 Schedule
• Test Plan Review Schedule

Test Results

• Live Results
• RC3 Results
• RC2 Results
• RC1 Results
• Beta Preview Results
• Beta 4 Results
• Beta 3 Results
• Beta 2 Results
• Beta 1 Results
• Alpha 2 Results
• Alpha 1 Results

Test Areas

Feature Focused Areas

Key: L = Large (3-6 weeks) M = Medium (2-4 weeks) S = Small (1-2 weeks)

Test Plan Template

Feature Area and Spec	Test Plan Location	QA Contact	Dev Contact	Size	FF or Platform?	Description
Downloadable Fonts (@font-face rule)	here	Akira, ctalbert	jdaggett/zack	L	Pf	Primary support backend: bug 70132 Primary CSS @font-face support: bug 441469 unified font entry bug 437356 have user font object hooked into @font-face parsing code bug 441473 Spec
moz-transform (CSS Transform)	here	martijn, ctalbert	Keith Schwarz	L	Pf	Doing Transforms on HTML elements via CSS. Does webkit team have tests for this? Demo at (requires patch or webkit): http://tekkie.flashbit.net/tmp/css3_transform_sample.html CSS Transforms Spec Primary Bug: bug 435293
NPAPI Cocoa Event Model	?	marcia	Josh Aas	L	Pf	UPDATE - feature cut for 1.9.1 due to high risk and extra work bug 350109 - Need to check Info.plist for plugin metadata bug 435041 New event model for NPAPI plugins NPAPI Event Models Spec
Places	here	Tracy	dietrich	L	Ff	tag autocomplete (bug 415960 bulk tagging/editing (bug 412002 reducing fsyncs and dependencies (bug 442967 transaction manager perf improvements bug 432706 bug 450290 – Sync the temp tables to the permanent tables
Private Browsing Mode	Test Plan	Marcia	Ehsan Akhgari [:ehsan]	L	Ff	Private Browsing Mode Spec Unit Test Plan bug 248970 - Private Browsing mode bug 411929 - Private Browsing UI
Query Selectors	?	martijn, ctalbert	bzbarsky@mit.edu	L	Pf	Decided that John Resig's tests provide enough coverage here, no further work required on our part, right now bug 416317 Spec at http://dev.w3.org/2006/webapi/selectors-api/ John Resig wrote a test Webkit devs wrote a performance test Security Review
SVG CSS	here	Martijn	roc	L	Pf	SVG in CSS is mostly covered by two bugs: bug 448831 and bug 450340 bug 276431 - external SVG not loaded from img tag CUT SVG Image Decoding
WHATWG <canvas> text API	here	Tchung	ebutler	L	Pf	bug 310682 – Implement shadows for <html:canvas> and dependencies spec draft implement text bug 436904 default computed values bug 438695 browser zoom bug 442133 bidi resolution bug 402276 CUT font baselines bug 438702
Border Image	?	martijn, ctalbert	robarnold, dbaron	M	Pf	bug 378217 spec draft
Cross Site XHR	here	bc, ctalbert	sicking@mozilla.com	M	Pf	bug 389508 cross site scripting allowed in certain cases http://wiki.mozilla.org/Cross_Site_XMLHttpRequest
Drag and Drop of Tabs (to new Windows)	here	juanb	Bz	M	Ff	Drag and Drop of tabs to new windows is now supported bug 113934 Boris' Blog Post
Color Management	here	Ezadkiel Villarico Marbella (seneca community)	bholley	M	Pf	bug 418538 - Enable color management support by default bug 455077 - Enable full color_management by default Evmarbella's Blog Bholley's Blog
HTML 5 Drag and Drop API	here	Joelm	Ndeakin	M	Pf	bug 356295 - Drag and Drop (WHATWG) WHATWG DnD Spec Neal Deakin's Blog Post DnD Documentation
Media Queries	here	Joelm, (formerly Hasham)	dbaron	M	Pf	Allows for html to query for a media type based on specific attributes in order to determine which style sheet or style rule to apply Primary Bug: bug 156716 spec draft
Native JSON	here	Mikeal	rsayre@mozilla.com	M	Pf	hook webpages that use JSON, to begin using native JSON streams, native to the browser. Currently, need to install a third party prototype. bug 408838 Spec: http://wiki.ecmascript.org/doku.php?id=es3.1:es3.1_proposal_working_draft Dojo JSON Tests
Worker Threads	here	Adam or bc	bent@mozilla.com	M	Pf	currently, everything is on one UI thread. affects performance of browser if its a slow thread. Want Multi-threading look for security sided, extra threads are well scoped. look for browsers immune to threads that are slow in some places Detect areas we can break the platform, regard to threading (eg. mxr causes slow threading, loading of all the text and linkifying and color coding -- all on the main UI thread) bug 437152 Status: potential risk, still looking hard WhatWG Specification
AwesomeBar	here	stephend	dietrich/mardak	S	Ff	bug 455561 - Tracker for Places Awesomebar Enhancements in Fx 3.1 Awesome++ spec show keywords (bug 392143) edit middle (bug 407888) restrict match (bug 395161) Mardak's blog post
Clear Private Data Enhancements	Test Plan	Marcia (related to PB)	Johnathan Nightingale	S	Ff	Flowchart bug 453440 - Extend "Clear Private Data" dialog with time period option bug 463607 - Interaction of Clear Recent History dialog and the private browsing mode
Ctrl-Tab	Ctrl Tab Test Plan	marcia	Dao	S	Ff	UPDATE - feature cut for 1.9.1 Boriss's blog post bug 395980 - Ctrl-Tab bug and dependencies Beta 2: bug 436304 - Implement "All Tabs" panel with previews
Session Restore	Session Restore Test Plan	marcia	Dietrich	S	Ff	UI Mockups bug 339445 - Session store should save/restore sessionStorage data (from DOMStorage) bug 448976 - turn the Session Restore prompt into an error page (and dependent bugs) Relevant bugs fixed in the timeframe Unit Tests
Exiting the Browser Experience	?	Tomcat	?	S	Ff	? Need owner
Search Text Widget	Search Widget Testplan	whimboo	Dao	S	Ff	bug 388811 – provide a search textbox widget Dependency bugs blocked bug 406095 – add emptyText property to textbox
SMIL	SMIL testplan	Martijn	dholbert	S	Pf	UPDATED - Feature is not landing in 3.1 bug 216462 patch queue SVG Spec Animation Section SMIL Animation spec (more general)
Element Traversal	here	Joelm	Jonas Sicking	S	Pf	bug 447917 - Implement ElementTraversal Element Traversal Spec Needs tests. (landed)
Script Defer	here	Joelm	Jonas Sicking	S	Pf	bug 28293 - (defer) scripts with defer attribute not deferred Comment 24 & 25, 39 has simplified testcases (landed)
ACID 3	Testplan	Tomcat	Roc or dbaron	M	Pf	bug acid3
Add-on Update	Addons Testplan	Tchung/Tomcat	?	M	Ff	Beta2: bug 324121 – Extensions with available updates for the next version of the app should not be listed in app update extension will be disabled warning Resolved Fixed, needs testing Beta 2: bug 455906 - Support severities for blocklist entries Resolved Fixed, Need to finish hard block testing bug 421470 – Software update does not download/update even it is set to automatic No fix in site Beta 2: bug 407875 - Unprivileged users are not notified of security updates Flagged for beta 2, but no patch in sight yet. bug 434155 – Help -> Check for Updates does not verify add-on compatibility Resolved Fixed, needs testing bug 434465 – Software update's background add-on update check ignores updated compatibility Fixed by bug 324121. Test that bug, and this bug is good. CUT bug 392180 – Provide force variable to use in the updateURL Beta2: Plugin blocklist - bug 391714, bug 449027, bug 391728
Memory Leaks	Memory Leaks Test Plan	Tomcat	Tomcat	L	Pf/Ff	Checking for memory leaks and asserts on debug builds
Offline/Local Storage	here	juanb	dcamp		Pf	Spec: http://www.whatwg.org/specs/web-apps/current-work/multipage/offline.html CUT for 3.1 bug 422526 - implement localStorage bug 442806 - Use separate, versioned caches for offline apps (and dependencies) bug 442813 - Implement fallback entries (and dependencies)
Password Manager to mozStorage	here	harthur	zpao	?	Pf	bug 288040 not sure it needs QA work
Video Decoders	here	Tchung, ctalbert, mikeal	roc/cdouble?		Pf	bug 422538 – Ogg Theora backend for HTML5 video element (tracking bug) Core element patch has landed --enable-media on trunk? Ogg Vorbis/Theora backend could get review, but legal issues may block Need to push GStreamer bug 422540 and Quicktime bug 435298 backends to patch submission DirectShow backend playing some movies, needs more work bug 435339 Primary bug: bug 382267 add on & related bugs: bug 448674, bug 448603 bug 448909 - Need more controls WHATWG Video tag (video front end work) Video Scrubber Blog bug 462113 - Implement progress bar / scrubber for video controls bug 470596 - css border breaks video controls
JavaScript 1.8.1		bc	brendan		Pf	bug 429508 - JavaScript 1.9/1.8.1 tracking bug bug 380236 - JavaScript 1.8 tracking bug Currently testing 1.9.1 mozilla-central and 1.9.1 tracemonkey branches. JavaScript Tests Needed At Risk bug 445178 - ES 3.1 Decimal Support
CSS Transitions	?	mw22 and/or ctalbert	Zwol, Jonathon Jongsma		Pf	CUT bug 435441 - main bug The spec

Unowned Features

Feature Area and Spec	Test Plan Location	QA Contact	Dev Contact	Size	FF or Platform?	Description
Windows TSF integration	?	??, Gary Kwong (only IME QA)	roc, Masayuki, jim chen	S	Pf	Update - Feature is pushed to post 1.9.1 bug 88831 TSF Spec Security Review
Linkified View Source	here	Curtis Bartley	Curtis Bartley	S	Ff	Blog post Beta 2: bug 17612 - view-source link-browsing bug 455888 - Make View Source not suck (view source meta bug) bug 455948 - The view source window should have a toolbar with toggle-line-wrap and goto-line buttons bug 455945 - View Source should "linkify" links in the source -- i.e. show them as clickable hyperlinks bug 246620 - Add line numbers to View Source for Firefox bug 464314 - view-source link-browsing doesn't decode entities bug 79518 - tracking view source UI
Improved CMS performance	?	?	Bobby Holly	S	Pf	bug 444661 - Decrease Performance Penalty of color management linear interpolations Unclear how to test (landed)
Aero Glass support in chrome on Vista	?	?	Bobby Holly	S	Pf	bug 418454 - Add support for Aero Glass effects to Widget layer bug 367993 - Make Firefox have the Translucent theme like IE7 Need unit tests. Landed

Operating System test plans

• Snow Leopard test plan

Regression Areas

• Top Web accessibility Tests
  • Plugins (eg. Flash, java, AJAX)
  • Extensions (eg. Firebug, NoScript, Fotofox)
  • Themes (eg. Vista, Joga, Simpsons)
  • Top Site Tests (eg. Alexa top 50)
  • Financial Sites (eg. banks, online trading)
• Accessibility Tests
• User Performance Testing
• Major / Minor Update Testing
• L10N Testing (may coincide with Feature area)
• Distribution Testing (may coincide with Feature area)

Link to Schedule and Assignments

Extensions

The following extensions in the list below will be ran through regression tests during selected milestone releases.

See examples here.

Top Crash reports

The main report is the easiest to view.

Banking and Financial Sites

For these types of sites, we’re asking community members and MoCo QA to state which banks and financial institutions they're willing to test. Please add your name to the list.

L10n Testing with Third Parties

For additional string testing, Mozilla will work with the l10n Team for localization and string testing.

Results link to be posted later.

Review Feedback

In this section, we will roll up all common feedback pulled from external sources that appears common and should be tracked by QA. Feedback sources can include:

• Hendrix Feedback system (mozilla.feedback newsgroups)
• Bugzilla
• Breakpad Crash Reporter
• Mozillazine Forums

QA / Support Team Issues

Add any common issues you are finding during your nightly /release testing that you feel would be good for Mozilla support to be aware of.

• See Topcrashers in beta 1
• Private Browsing has landed, but bugs are being found as we speak
• Ctrl-Tab preview pane has landed, but bugs are being found as we speak

Additional References

This section is left to add any additional references relevant to the Firefox 3.1 project