Auto-tools/Projects/Firefox UI Tests

From MozillaWiki
Jump to: navigation, search

About

The Firefox UI tests are a collection of integration tests mainly for testing the user interface of Firefox. The tests and its harness are based on the Marionette testing framework, and enhances it for Firefox Desktop specific behavior, e.g. it adds a page object model aka Firefox-Puppeteer.

Team

This project is maintained by the Automation Crew, and the primary contact is Henrik Skupin.

Goals

The goal of this project is to increase the test coverage for new features in Firefox Desktop. Especially for those features which cannot be covered by any other existing test suite. To achieve that we will also have to continuously improve our harness and firefox-puppeteer page object model.

We want that test development will become as easy as writing manual tests. Therefore we only want to make use of synchronous test steps, and don't force the user to understand the DOM of the browsers interface.

Dependencies / Who will use this

For every feature enhancement which cannot be directly done in our own harness, we have to work together with the developers for Marionette.

The users will be:

  • Firefox QE

Contribution

We welcome contributions to Firefox UI Tests! We track issues inside the Firefox UI Tests component on Bugzilla, see below for a list of open bugs.

If you have a question, jump on #automation on irc.mozilla.org and look for whimboo, or maja_zf.

Source and docs

More information about the tests and how to run those can be found in the Mozilla Developer Network.

Active and Planned Projects

The currently most important projects for the firefox-ui-tests are the following:

  • Bug 1272236 - Let Mozmill-ci execute functional and update tests for Nightly builds on Linux64 via TaskCluster
  • Bug 1272228 - Make Firefox-ui-tests results from TaskCluster Tier-1 or Tier-2 on Treeherder
  • Bug 1272145 - Move Firefox-ui-tests from testing/firefox-ui-tests/tests to individual browser/toolkit components

Bug Table

Full Query
ID Priority Component Summary Assigned to
1201789 P3 Firefox UI Tests Add a test for shutting down Fx during a WebRTC call
1312359 P3 Firefox UI Tests [tracking bug] Port Firefox UI functional tests to plain Marionette tests to reduce maintenance and complexity
1325047 P3 Firefox UI Tests Test test_ssl_status_after_restart.py TestSSLStatusAfterRestart.test_ssl_status_after_restart fails in e10s mode
1206003 P5 Firefox UI Tests Intermittent failure: test_safe_browsing_notification.py TestSafeBrowsingNotificationBar.test_notification_bar | NoSuchElementException: Unable to locate element: seeDetails
1277935 P5 Firefox UI Tests TEST-UNEXPECTED-ERROR | testing/firefox-ui/tests/puppeteer/test_places.py TestPlaces.test_bookmarks | ScriptTimeoutException: ScriptTimeoutException: Timed out
1289115 P5 Firefox UI Tests Fix timeout issues in Places.remove_all_history()
1400780 P5 Firefox UI Tests Intermittent test_restore_windows_after_restart_and_quit.py TestSessionStoreDisabled.test_restore_with_restart | AssertionError: Non private browsing windows should have
1516519 P5 Firefox UI Tests Intermittent en-US Automation Error: mozprocess timed out after 1000 seconds running ['Z:\\<random>\\build\\venv\\Scripts\\python', 'Z:\\<random>\\build\\venv\\Lib\\site-packages\\firefox_ui_harness\\cli_functional.py' after "WebDriver:DeleteSession"
1542470 P5 Firefox UI Tests Intermittent testing/firefox-ui/tests/functional/safebrowsing/test_notification.py TestSafeBrowsingNotificationBar.test_notification_bar | TimeoutException: Timed out after 300.1 seconds with message: Expected target element "#main-feature"
1578240 P5 Firefox UI Tests Intermittent Mn MacOS Automation Error: mozprocess timed out after 1000 seconds running ['python', ...] after firefox[1651:12232] Persistent UI failed to open file window_1.data: No such file or directory (2)
1587709 P5 Firefox UI Tests Intermittent testing/firefox-ui/tests/functional/security/test_mixed_content_page.py TestMixedContentPage.test_mixed_content | AssertionError: u'verifiedDomain' != 'unknownIdentity mixedDisplayContent'
1588399 P5 Firefox UI Tests Intermittent testing\firefox-ui\tests\functional\sessionstore\test_restore_windows_after_restart_and_quit.py TestSessionStoreDisabled.test_restore_with_restart | IOError: Process has been unexpectedly closed (Exit code: 0) (Reason: [Errno 10053] An establ
1093599 -- Firefox UI Tests Add automated test for customized toolbar after restart
1136684 -- Firefox UI Tests Add automated tests for session restore after crash
1148101 -- Firefox UI Tests Create a mozmill test for tracking protection
1176422 -- Firefox UI Tests AutocompleteResults.get_matching_text can return non-matching text for results not of type bookmark
1237766 -- Firefox UI Tests Write a Marionette test that downloads malware and verifies that it's blocked.
1239158 -- Firefox UI Tests Write a Marionette test that checks that visiting a website with a tracker in Private Browsing blocks the tracker
1258194 -- Firefox UI Tests Add tests for download history across restarts
1284119 -- Firefox UI Tests Intermittent test_places.py TestPlaces.test_plugins | ScriptTimeoutException: ScriptTimeoutException: Timed out
1373561 -- Firefox UI Tests Intermittent test_about_private_browsing.py TestAboutPrivateBrowsing.testCheckAboutPrivateBrowsing | NoSuchElementException: Unable to locate element: learnMore
1573410 -- Firefox UI Tests Remove firefox ui harness and port remaining tests to Marionette
1573641 -- Firefox UI Tests Simplify test_ssl_status_after_restart.py

23 Total; 23 Open (100%); 0 Resolved (0%); 0 Verified (0%);