Mozilla QA Community:BonEcho 2.0b2 Community Test Day

From MozillaWiki
Jump to: navigation, search

Welcome to the Bon Echo Community Test Day!

Thank you for participating in this test day. The success of Firefox and Thunderbird is heavily dependent on community members like you. We appreciate your support!

When you are done, please send feedback to testday@mozilla.org or post significant findings to the Mozilla QA blog (http://weblogs.mozillazine.org/qa/). We are very interested to hear about how the test day went, any overall thoughts about Bon Echo, and roll up of issues found and bugs filed.

The official time for the test day is Friday, August 25, 2006, from 7am - 5pm PDT, but anyone can jump on Litmus at any time on any day!

For more information about this release, please check out the Bon Echo wiki. You will find the current schedule, test plan, and specific areas we can use help testing.

FAQ

Where do I start?

Other Options of if you don't use IRC:

  • Use Instant Messenger - http://www.aim.com/. A moderator will be online under the screen name "MozillaTestDay". Please send your screenname to marcia@mozilla.org before the test day so you can be invited to the chatroom, or just message "MozillaTestDay" during the day to figure out what to do.

OK, I am in the channel, now what?

Let the moderators know if you have any questions or encounter any problems. If you are not familiar with our bugzilla bug tracking system, feel free to report your problems to a moderator and they can log any bugs for you.

What build should I test?

We will be testing the Firefox 2.0beta2 release candidate builds:

Firefox (Windows, Linux): http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2.0b2-candidates/rc1/

Firefox (Mac): http://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2.0b2-candidates/rc2/

What specifically will we be testing?

We will be testing the new tabbed browsing and session store features that will be in Firefox 2. A quick overview of the new features:

  • Tab overflow - now when a lot of tabs are open, scroll buttons appear on the left and right side of the tab bar allowing you to scroll through tabs.
  • All Tabs button - a button that shows on the right side of the tab bar that lists all open tabs.
  • Close button on tabs - instead of having the close button on the right side of the tab bar, now close buttons are on each individual tabs.
  • Warn if closing too many tabs - now if you try to open too many tabs at once (more than 15), you will be asked if you want to open those tabs.
  • Tab focusing behavior - when you open a new tab from a link on a page and then close that tab, you will go back to the page the tab was opened from


  • Tab history and clearing private data - make sure tab history is cleared
  • Restore Session and tab history - make sure tab history is restored
  • Restore Session with form data - make sure entered form data is restored
  • Restore Session with multiple windows - make sure all windows are restored and that their tabs, size, position and order are the same; including other windows, like Downloads, Extensions, ...

Existing open bugs:

Testing Instructions

Note: The release candidate builds will be branded Firefox.

Follow these instructions:

  1. Download a Bon Echo build from the nightly directory: ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2.0b2-candidates/rc1 (Windows, Linux) ftp://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/2.0b2-candidates/rc2 (Mac)
  2. Go to Litmus and run the FF2.0 Full Functional Tests for Tabbed browsing. (link to more on litmus)

Some other ways you can help:

  • Think of some test cases that aren't already included in Litmus (we love suggestions!).
  • Do some exploratory testing with tabbed browsing. Do a combination of things to try to explore the feature fully.

Where do I find Litmus?

  • Go to the Litmus start page.
  • Log in or create an account if you don't already have one and then select run tests.
  • Enter the appropriate information and click Submit.
  • Choose a test group and then a subgroup and then click Submit to start testing!

Tips:

  • Make sure you enter the correct information for your platform, version of Firefox, etc.
  • Feel free to ask on IRC or AIM if you need help or have any questions.

Who will be moderating the #testday channel?

The moderator's nicks for this test day will be:

  • tracy
  • bc
  • marcia
  • juanb
  • coop
  • alice
  • mw22
  • ispiked

How do I file bugs?

First of all, let one of the moderators know so they can check if it is already a known problem. Then they can guide you to either provide more details, update an existing bug or file a new bug.

If filing a new bug, please do a quick check to see if the bug already exists. This can easily be done by searching for the error message, window name, or some over identifing feature of the bug.

In the case of any testing endeavor, it is important to include the list of extensions that you have installed since they can often cause strange issues to crop up.

How do I send feedback?

Please use the testday@mozilla.org to send feedback about how the testday went, any overall thoughts about Bon Echo, and roll up of issues found and bugs filed. We really appreciate your help today and your feedback is very valuable.

Rewards for this Test Day

The top 5 testers who have not previously won $25 gift certificates to amazon.com will be rewarded for this test day. If you reside outside of the U.S, you will be awarded a $25 gift certificate to the Mozilla Store.

Results

Total Test Results By Locale

Locale # Results
en-US 676

Total Test Results By Platform

Platform # Results
Windows 441
Linux 47
Mac 188

Total Test Results By Result Status

Result Status # Results
Pass 656
Fail 10
Test unclear/broken 10

Total Test Results By Subgroup

Subgroup # Results # Testcases in Subgroup
Firefox:1.5 Full Functional Tests (FFTs):XMLHttpRequest 2 3
Firefox:1.5 Full Functional Tests (FFTs):Bookmarks 2 34
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Installation 16 4
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Migration 2 3
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Menu Bar 28 10
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Help 1 2
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Options (Preferences) 4 5
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - General 14 10
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Location Bar & Autocomplete 2 2
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Tabbed Browsing 111 18
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Search 11 11
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Popup and annoyance blocking 9 9
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Find in Page 12 6
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Bookmarks 23 14
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - RSS 7 6
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Add-ons Manager 18 12
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Downloading 11 9
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Toolbar Customization 9 5
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Printing 4 4
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - History 4 5
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Password Manager 5 5
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Form Manager 4 4
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Cookies 6 6
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Security and Certs 4 4
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Anti-phishing 12 3
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Import 1 1
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Session Store 2 7
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Technical Tools 20 5
Firefox:2.0 Basic Functional Tests (BFTs):FX 2.0 - Uninstall 9 3
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Installation 24 8
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Migration 3 5
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Menu Bar 28 10
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Help 6 8
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Options (Preferences) 4 13
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - General 5 8
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Location Bar & Autocomplete 4 4
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Tabbed Browsing 265 41
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Search 11 18
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Popup and annoyance blocking 12 12
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Find in Page 21 11
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Bookmarks 59 34
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - RSS 26 23
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Add-ons Manager 21 19
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Downloading 18 16
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Toolbar Customization 14 10
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Printing 5 5
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - History 8 8
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Password Manager 11 11
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Form Manager 6 6
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Cookies 18 18
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Security and Certs 9 9
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Anti-phishing 37 10
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Import 3 4
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Plug Ins 2 4
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Session Store 3 20
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Top Sites 14 8
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - XMLHttpRequest 9 3
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Technical Tools 20 5
Firefox:2.0 Full Functional Tests (FFTs):FX 2.0 - Uninstall 9 3
Firefox:2.0 Smoketests:FX 2.0 - Install and Startup 16 5
Firefox:2.0 Smoketests:FX 2.0 - Functionality 35 27
Firefox:2.0 Surf's Up - A Fun Smoketest!:Common Operations 11 4

Top Testers

Tester # Results
Tomcat 258
DJ_Peng 64
Bobby 62
LonelyBob 62
mkairys 44
robcee 44
juanb 35
marcia 35
Damon 23
dex_sf 23
technolalia 10
DNA300 9
Greenknight 4
stu42j 3

Result Status Breakdown By Tester

Tester Pass Fail Unclear
Bobby 57 0 5
DJ_Peng 61 2 1
DNA300 9 0 0
Damon 23 0 0
Greenknight 4 0 0
LonelyBob 60 1 1
Tomcat 258 0 0
dex_sf 23 0 0
juanb 33 2 0
marcia 33 1 1
mkairys 44 0 0
robcee 42 0 2
stu42j 3 0 0
technolalia 6 4 0