Auto-tools/Projects/Firefox UI Tests

From MozillaWiki
Jump to: navigation, search

About

The Firefox UI tests are a collection of integration tests mainly for testing the user interface of Firefox. The tests and its harness are based on the Marionette testing framework, and enhances it for Firefox Desktop specific behavior, e.g. it adds a page object model aka Firefox-Puppeteer.

Team

This project is maintained by the Automation Crew, and the primary contact is Henrik Skupin.

Goals

The goal of this project is to increase the test coverage for new features in Firefox Desktop. Especially for those features which cannot be covered by any other existing test suite. To achieve that we will also have to continuously improve our harness and firefox-puppeteer page object model.

We want that test development will become as easy as writing manual tests. Therefore we only want to make use of synchronous test steps, and don't force the user to understand the DOM of the browsers interface.

Dependencies / Who will use this

For every feature enhancement which cannot be directly done in our own harness, we have to work together with the developers for Marionette.

The users will be:

  • Firefox QE

Contribution

We welcome contributions to Firefox UI Tests! We track issues inside the Firefox UI Tests component on Bugzilla, see below for a list of open bugs.

If you have a question, jump on #automation on irc.mozilla.org and look for whimboo, or maja_zf.

Source and docs

More information about the tests and how to run those can be found in the Mozilla Developer Network.

Active and Planned Projects

The currently most important projects for the firefox-ui-tests are the following:

  • Bug 1272236 - Let Mozmill-ci execute functional and update tests for Nightly builds on Linux64 via TaskCluster
  • Bug 1272228 - Make Firefox-ui-tests results from TaskCluster Tier-1 or Tier-2 on Treeherder
  • Bug 1272145 - Move Firefox-ui-tests from testing/firefox-ui-tests/tests to individual browser/toolkit components

Bug Table

Full Query
ID Priority Component Summary Assigned to
1573410 P1 Firefox UI Tests Remove firefox ui harness and port remaining tests to Marionette Henrik Skupin (:whimboo) [⌚️UTC+2]
1516519 P5 Firefox UI Tests Intermittent en-US Automation Error: mozprocess timed out after 1000 seconds running ['Z:\\<random>\\build\\venv\\Scripts\\python', 'Z:\\<random>\\build\\venv\\Lib\\site-packages\\firefox_ui_harness\\cli_functional.py' after "WebDriver:DeleteSession"
1542470 P5 Firefox UI Tests Intermittent testing/firefox-ui/tests/functional/safebrowsing/test_notification.py TestSafeBrowsingNotificationBar.test_notification_bar | TimeoutException: Timed out after 300.1 seconds with message: Expected target element "#main-feature"
1578240 P5 Firefox UI Tests Intermittent Mn MacOS Automation Error: mozprocess timed out after 1000 seconds running ['python', ...] after firefox[1651:12232] Persistent UI failed to open file window_1.data: No such file or directory (2)
1643336 P5 Firefox UI Tests Intermittent testing/firefox-ui/tests/functional/sessionstore/test_restore_windows_after_windows_shutdown.py TestWindowsShutdownForcedSessionRestore.test_manual_restart | InvalidSessionIdException: Please start a session
1657875 P5 Firefox UI Tests Intermittent testing/firefox-ui/tests/functional/sessionstore/test_restore_windows_after_restart_and_quit.py TestSessionStoreDisabled.test_no_restore_with_quit | IOError: Process killed after 120s because no connection to Marionette server could be
1672757 P5 Firefox UI Tests Intermittent testing/firefox-ui/tests/functional/sessionstore/test_restore_windows_after_restart_and_quit.py TestSessionStoreEnabledNoPrivateWindows.test_with_variety | InvalidSessionIdException: Please start a session
1673122 P5 Firefox UI Tests Intermittent test_restore_windows_after_restart_and_quit.py TestSessionStoreDisabled.test_restore_with_restart | IOError: Process has been unexpectedly closed (Exit code: 0) (Reason: Process unexpectedly quit without restarting (exit code: 0))
1613386 -- Firefox UI Tests [meta] Remove Firefox-UI tests
1638960 -- Firefox UI Tests Convert 'mach firefox-ui-functional' to run with Python 3

10 Total; 10 Open (100%); 0 Resolved (0%); 0 Verified (0%);