MozillaWiki

QA/Firefox Updates/General

< QA‎ | Firefox Updates

Firefox Update Mechanism

The following is an ongoing test plan detailing QA's strategy for ensuring quality of the Firefox update experience in general.

Desktop

Compatibility

The following is a list of P1 environment configurations which need to be tested.

Locales

  • Top 10 locales

Add-ons

Anti-virus Software

Here is a list of anti-virus software to test.

P1 (test these first)
P2
P3
  • Internet security software provided by your ISP

Plug-ins

Owned by Virgil Dicu (Softvision)
Status - On track for draft tests in Litmus by 2011-11-25
  • Based on SUMO article
    • Adobe Flash
    • Adobe Reader
    • Adobe Shockwave
    • Apple Quicktime
    • Gecko Media Player
    • Java
    • Microsoft Silverlight
    • RealPlayer
    • VLC Media Player
    • Windows Media Player
  • Sample Test Matrix: link

Use Cases

Porting of the following usecases to testcases is being tracked in this spreadsheet

Update Paths
  • partial
  • partial + fallback
  • complete
  • complete + fallback
  • pave-over to upgrade
  • pave-over to downgrade
  • major update unadvertised
  • major update advertised
  • pause, resume, hide, cancel
  • timeout and notifications
  • antivirus interaction (ZoneAlarm Forcefield, Symantec, McAfee, Microsoft, Kapersky, AVG, TripWire, Mac/Linux)
  • forced failure at all stages of update process
  • fallback from "silent" to "noisy"
Add-ons
  • Third party add-ons opt-out UI
  • Incompatible add-ons
    • force compatible
    • outdated without an update downloaded
    • outdated with an update downloaded
    • disabled/enabled
  • Compatible add-ons
    • outdated without an update downloaded
    • outdated with an update downloaded
    • updated already
    • disabled/enabled
  • Personas and Themes
Session Store
  • App tabs
  • Panorama
  • Private Browsing
  • Recently closed windows
  • Recently closed tabs
  • Form data
    • composing an email in gmail
    • filling out user account forms
  • Passwords & authenticated sessions
  • Video and Audio streaming
  • Shopping cart data
  • Games
  • User settings (e.g. Clear history)
  • Upgrade right after a session restore
Places
  • Bookmarks
  • History

Litmus Tests

The following testcases are already in Litmus, QA's manual testing suite

Test ID Summary
3.6 -> 8.0 Major Update
31850 Check for updates and verify a major update is offered
31854 Selecting "Details" in "Downloading Update" opens new window on update details page
31858 Selecting "Get the new version" in Software Update dialog takes user to "Downloading Update"
31859 Selecting "Never" in Major Update details dialog dismisses it forever
31857 Selecting "Later" in Major Update details dialog dismisses it
31851 Major Update details are in locale language
31861 Selecting "Restart" in "Downloading Update" after download is complete applies update
32145 Verify navigation buttons appear correctly
33512 Upgrade Firefox while having add-ons pending for update installation
31860 Selecting "Pause" in "Downloading Update" pauses update downloading
31848 After update, compatible extensions still work
31849 After update, compatible themes still work
31852 On first run after install, users are warned about incompatible extensions and can update them
31853 On first run after install, users are warned about incompatible themes and can update them
31862 Selecting "Resume" in "Downloading Update" of paused update resumes download
31863 Users are warned about incompatible extensions
31864 Users are warned about incompatible extensions and can cancel updating them
31865 Users are warned about incompatible extensions and ignore the warning
31866 Users are warned about incompatible themes
31867 Users are warned about incompatible themes and can cancel updating them
31868 Users are warned about incompatible themes and ignore the warning
31855 Selecting "Hide" in "Downloading Update" hides dialog
31856 Selecting "Hide" in "Downloading Update" with paused download prompts user
32505 Users are not warned about disabled incompatible extensions
32853 Verify extensions present on downgrade to previous Firefox version
33512 Upgrade Firefox while having add-ons pending for update installation
Software Update
31792 Apply software update should display Restart Confirmation
31803 Software update dialog wait for idle before prompting
31287 No Updates Found
31492 Make sure we don't offer the same update again
31648 Make sure users that are not Member of the Administrator group don`t receive Update Notifications
32856 Make sure that auto-detect proxy settings do not interfere with software updates

Mozmill Tests

The following tests are already in Mozmill, QA's automated testing repository

  • testDirectUpdate
    • test1.js: enables update logging
    • test2.js: checks for and downloads update via the About dialog
    • test3.js: checks no more updates are offered, the downloaded update was applied, and the About dialog references the correct version
  • testFallbackUpdate
    • test1.js: enables update logging
    • test2.js: checks for and downloads the update via the About dialog
    • test3.js: forces a failure
      • if a complete update: checks for the error page, downloads the update again via the About dialog
      • if a partial update: uses the failure wizard to download the complete update
    • test4.js: checks no more updates are offered, the downloaded update was applied, and the About dialog references the correct version

Mobile

Retrieved from "https://wiki.mozilla.org/index.php?title=QA/Firefox_Updates/General&oldid=375431"