QA/Fennec/Milestones/2.0/ReleaseTest/Beta1

From MozillaWiki
< QA‎ | Fennec‎ | Milestones‎ | 2.0
Revision as of 17:07, 5 October 2010 by Ashah (talk | contribs) (→‎Nexus One [ashah])
Jump to navigation Jump to search

Fennec 2.0 Beta 1 Test Results

The covered testing for this release will focus on smoketests, verifications and updates. For a list of new features coming in for Fennec 2.0, take a look at the Mobile Planning Page and for test coverage, look at our test plan.

Project contacts

PM: Stuart Parmenter (stuart)
Build: John O'duinn (oduinn)
QA: Aakash Desai (aakashd)

Schedule

Code Freeze
-- 10/1
Builds hand to QA
-- 10/1
QA test signoff
--

Build Location and IDs

  1. Go to the candidates folder in our ftp folders
  2. Go to the "4.0b1-candidates" folder
  3. Download the candidate:
    • For the n900, install the application catalog and update to the candidate build via your device App Manager.
    • For Android, download and install the actual build over any other instance

Test Coverage

Smoketests

Lite verification tests (or smoketests) will be first run when the builds are ready. Tests are available on litmus under the Fennec 2.0 Test Suite. In any case of failure in any Smoketest, blockers need to be raised ASAP before proceeding to the BFTs.

Please record your detailed results in the "Detailed Test Results" section below.

BFTs for 2.0 Features

BFTs are a more comprehensive set of litmus tests that cover functional testing of features that have landed on Fennec 2.0. Tests are available on litmus under the Fennec 2.0 Test Suite.

Please record your detailed results in the "Detailed Test Results" section below.

Here are the BFT's that need to be run:

  • AwesomeBar (Location Bar & Autocomplete)
  • Feedback
  • RTL Support
  • Form Assistant
  • Panning & Zooming (Screen Navigation)
  • Sharing
  • Sync

OS Integration

The Operating System Integration testsuite is a set of tests that target how Fennec works with the platform (i.e. Android, Maemo, etc.) and features of the device itself (i.e. Orientation, hard keyboard, soft keyboard, etc.). Tests are available on litmus under the Fennec 2.0 Test Suite under the "OS Integration Tests" testgroup.

Please record your detailed results in the "Detailed Test Results" section below.

Fixed Bugs Verifications

The following link contains a list of bugs that are fixed for beta 1 and need verifications:

Updates

Fennec 2.0 Beta 1 can be downloaded from the Mozilla Beta Catalog.


Updates tests to perform for each build

Test Results

Types of Tests Maemo Android
Smoketests
  • ahoza - PASS
  • AaronMT - PASS
BFTs
  • aakashd - PASS
  • mw22 - result
  • ioana - result
  • nhirata - result
OS Integration
  • nhirata - PASS
  • ashah - result
Updates
  • ahoza/vlad/ioana - result
  • aakashd - PASS
  • ahoza/vlad/ioana - result
  • ashah - result

Detailed Test Results

Smoketests Results

Droid (Milestone)

  • Build: Mozilla/5.0 (Android; Linux armv7l; rv:2.0b7pre) Gecko/20101001 Firefox/4.0b7pre Fennec/4.0b1
  • Results: Pass w/1 smoketest failure
  • Notes:
    • Installing 4.0b1 candidate over my October 2nd nightly (Android indicates that it would replace the application) produces an Android prompt message w/ the Firefox branding on the top left: "Application is not installed". Upon clicking Ok, Fennec nightly starts up.
    • Smokestests

n900 [ahoza]

  • Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.0b7pre) Gecko/20101001 Firefox/4.0b7pre Fennec/4.0b1
  • Results: PASS
  • Notes:
    • build candidate is not multi-language

BFT Results

n900 [aakashd]

  • Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.0b7pre) Gecko/20101001 Firefox/4.0b7pre Fennec/4.0b1
  • Results: PASS
  • Notes:
    • Bugs
      • bug 601619 - page title shows up on urlbar when flipping through awesomescreen panes
      • bug 586288 - Cannot pan XUL, <iframe>s or scrollable
        s
      • bug 599222 - Unresponsive script error in Sync.js during initial sync in Fennec

n900 [mw22]

  • Build:
  • Results: result
  • Notes:
    • Non-reproducible issues
      • After closing Fennec, then restarting, Fennec sometimes seems to crash, because it refuses to start the first time. Starting a second time then works. I've had this twice now, when closing Fennec with multiple tabs open, then restarting. It seems to be related to the fact that closing Fennec, then quickly starting Fennec up again doesn't work either.
      • I got rendering errors while sliding to the left/right in about:config with four other tabs open. Type 'tab' in the filter field, then slide right/left to reproduce.
      • Installing add-ons locally with browser.tabs.remote=true doesn't work.
      • Ctrl-W works in the url bar, which is totally confusing
      • Opening a tab, then quickly start typing eats the first few characters (don't appear in the url bar)
      • I had a case where tapping on the close button didn't work at all. I had to type qtrl-q to close Fennec. I have no idea how I got into that state.
      • Pages open very small when letting them load in a background tab.
      • For some reason, Fennec refuses to remember the tab opened with http://www.kantjils.nl/moz/
      • On the "Start Page" row entry, when "blank page" is selected and restarted with the blank page, visiting a website opens it in a new tab, which doesn't make sense with a blank tab.
      • I had a situation where Fennec seemed to be stuck. It didn't react to any input. After 10s, it disappeared.
      • At one time, I was unable to restart Fennec at all. A restart of the N900 fixed it.

HTC Desire [ioana]

  • Build: Mozilla/5.0 (Android; Linux armv7l; en-US; rv:2.0b7pre) Gecko/20101001 Firefox/4.0b7pre Fennec/4.0b1
  • Results: result
  • Notes:
    • AwesomeBar
      • bug 599550 If I type too fast characters get inserted in the wrong order
    • Feedback
      • bug 597041 [Feedback] Feedback is missing in the control panel on the Android platform
    • RTL Support
      • no support for Arabic Languages on HTC Desire
    • Form Assistant
      • bug 601581 Form Fill appears only at a second tap in maps.google.com
    • Panning & Zooming
      • bug 599781 [regression] Browser shouldn't change scale before pageload
      • bug 600005 Tabs opened in the background are drawn at the wrong scale
      • bug 579628 HTML iframe can't be panned
    • Sharing - no issues
    • Sync
      • Unresponsive script when trying to login for first time

Motorola Droid 2 [nhirata]

  • Build: Mozilla/5.0 (Android; Linux armv7l; en-US; rv:2.0b7pre) Gecko/20101001 Firefox/4.0b7pre Fennec/4.0b1
  • OS: Android 2.2
  • Results: result
  • Notes:
    • RTL
      • bug 601822[RTL] a color filled webpage can move 1 pixel to the left when bringing up the tab panel in RTL
    • Panning/Zooming
      • bug 601792graphic defect can occur when panning with the cursor and focus in a text box.
      • bug 601826Occasionally Droid 2 does not zoom when repeating zoom in/out by double tap
    • Sharing
      • No Issues
    • Misc:
      • bug 601832Android Only : MSNBC webpage does not load correctly; menu is split
  • Known Issues:
    • Sync:
    • Feedback:
      • bug 597041Feedback for android | was pushed for postb1 patch, cannot test for android

OS Integration Results

n900 [nhirata]

  • Build: Mozilla/5.0 (Maemo; Linux armv71; rv:2.0b7pre) Gecko/201001001 Firefox/4.0b7pre Fennec/4.0b1
  • Results: PASS
    • Test unclear/broken: 7
      • Test cases for the n810 need to be updated for the n900
    • Fail: 6
    • Pass: 14
  • Notes:
    • No major blockers.
      • Concern: about not being able to log into Flickr.com
      • Concern: icon missing until reboot of Nokia
    • New bugs found during run:
      • bug 601751 Please consider redesigning the Favorite Panel to have the chic look
      • bug 601729 Fennec does not attempt to autoconnect to the internet.
      • bug 601691 Cannot log into Flickr.com
      • bug 601673 Hardware zooming with the N900 will cause the screen to go to the bottom right
      • bug 601661 [VKB]Bringing up the hardware keyboard should not deselect the Virtual Keyboard selected text
      • bug 601650 icon is missing for Fennec/Firefox after installation.
    • Could not update from trunk nightlies since they've already passed beta 1's product version

Nexus One [ashah]

  • Build: Mozilla/5.0 (Android; Linux armv7l; en-US; rv:2.0b7pre) Gecko/20101001 Firefox/4.0b7pre Fennec/4.0b1
  • Notes:
    • the update notifications did not appear on the nightly builds
    • This is not an OS integration test but something that I encountered while playing with sync. When I connect to my sync account and click on sync now, it shows Last update - in progress... But the Sync Now button remains enabled sometimes. This is not 100% reproducible. This generally happens when you sync it once and try to sync again as soon as the previous sync is complete. Here is the screenshot
    • Known bugs that caused the test to fail:-
    • New bugs filed:
      • bug 601799 Google.com orientation incorrect
      • bug 601805 Soft keyboard goes away after opening on google.com

Updates Results

n900 [aakashd]

  • Build: Mozilla/5.0 (Maemo; Linux armv7l; rv:2.0b7pre) Gecko/20101001 Firefox/4.0b7pre Fennec/4.0b1
  • Results: PASS
  • Notes:
    • Could not update from trunk nightlies since they've already passed beta 1's product version

HTC Desire [ioana]

  • Build: Mozilla/5.0 (Android; Linux armv7l; en-US; rv:2.0b7pre) Gecko/20101001 Firefox/4.0b7pre Fennec/4.0b1
  • Results: FAIL
  • Notes:
    • the notifications did not appear neither on the candidates nor the nightly builds
Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Fennec/Milestones/2.0/ReleaseTest/Beta1&oldid=258537"