QA/Mozmill Test Automation/Local Test Data

From MozillaWiki
Jump to: navigation, search

Overview

Lead: Anthony Hughes
Co-workers:
Dates: No end date set
Status: On hold
Completion: Tests: 43 / 94 (46%)
Repository Location: mozmill-tests and litmus-data
Tracking Bug / Bug List: bug 604707

Summary

The goal of this project is to have all tests in the mozmill-tests repository using local test pages, not remote test pages.

Details

As stated in the summary, the purpose of this project is to have all tests using local test pages instead of remote ones. The benefits of this are three-fold:

1. It gives us control over the test data, not having to rely on flaky third-party websites.

2. Test data needs to be local for all tests which are incorporated into Mozilla's buildbot system integration.

3. Local tests tend to load far quicker than their remote counterparts.

The work for this project has been broken down into phases. Each of these phases will be detailed below.

Phase I

Goal Create the simplest set of test pages and refactor all tests which can use them
Status Complete
Test Pages
  • Total Pages: 22 / 22 (100%)
  • [DONE] cookie_single.html: install a single cookie
  • [DONE] form.html: a simple webform
  • [DONE] position.html: a simple geolocation testpage
  • [DONE] mozilla.html: a clone of mozilla.org/index.html
  • [DONE] mozilla_community.html: a clone of mozilla.org/community.html
  • [DONE] mozilla_contribute.html: a clone of mozilla.org/contribute.html
  • [DONE] mozilla_governance.html: a clone of mozilla.org/governance.html
  • [DONE] mozilla_grants.html: a clone of mozilla.org/grants.html
  • [DONE] mozilla_mission.html: a clone of mozilla.org/mission.html
  • [DONE] mozilla_organizations.html: a clone of mozilla.org/organizations.html
  • [DONE] mozilla_projects.html: a clone of mozilla.org/projects.html
  • [DONE] login_form.html: a simple login form
  • [DONE] default_plugin.html: a simple page for testing the default plugin
  • [DONE] popups_2.html: a page for triggering popups
  • [DONE] popup.html: a sample popup
  • [DONE] mozsearch.html: a sample Mozilla Search page
  • [DONE] mozsearch.xml: a sample Mozilla Search plugin
  • [DONE] opensearch.html: a sample Open Search page
  • [DONE] opensearch.xml: a sample Open Search plugin
  • [DONE] searchresults.html: a page for displaying search results
  • [DONE] openinnewtab.html: a page for opening a new tab
  • [DONE] openinnewtab_target.html: a target page for a new tab
Tests
  • Total tests: 43 / 43 (100%)
  • Restart
    • [DONE] testMasterPassword/test1.js
  • Addons
    • [DONE] testDisableEnablePlugin.js
  • Awesomebar
    • [DONE] testAccessLocationBar.js
    • [DONE] testCheckItemHighlight.js
    • [DONE] testEscapeAutocomplete.js
    • [DONE] testFaviconInAutocomplete.js
    • [DONE] testGoButton.js
    • [DONE] testLocationBarSearches.js
    • [DONE] testPasteLocationBar.js
    • [DONE] testSuggestHistoryBookmarks.js
    • [DONE] testVisibleItemsMax.js
  • Bookmarks
    • [DONE] testAddBookmarkToMenu.js
  • Cookies
    • [DONE] testDisableCookies.js
    • [DONE] testEnableCookies.js
    • [DONE] testRemoveCookies.js
  • Find in Page
    • [DONE] testFindInPage.js
  • Forms
    • [DONE] testBasicFormCompletion.js
    • [DONE] testClearFormHistory.js
    • [DONE] testDisableFormManager.js
  • Passwords
    • [DONE] testPasswordNotSaved.js
    • [DONE] testPasswordSavedAndDeleted.js
  • Popups
    • [DONE] testPopupsAllowed.js
    • [DONE] testPopupsBlocked.js
  • Preferences
    • [DONE] testRestoreHomepageToDefault.js
    • [DONE] testSetToCurrentPage.js
  • Private Browsing
    • [DONE] testCloseWindow.js
    • [DONE] testDisabledPermissions.js
    • [DONE] testDownloadManagerClosed.js
    • [DONE] testGeolocation.js
    • [DONE] testStartStopPBMode.js
    • [DONE] testTabRestoration.js
    • [DONE] testTabsDismissedOnStop.js
  • Search
    • [DONE] testAddMozSearchProvider.js
    • [DONE] testOpenSearchAutodiscovery.js
    • [DONE] testSearchSelection.js
  • Security
    • [DONE] testIdentityPopupOpenClose.js
  • Session Store
    • [DONE] testUndoTabFromContextMenu.js
  • Tabbed Browsing
    • [DONE] testBackgroundTabScrolling.js
    • [DONE] testNewTab.js
    • [DONE] testOpenInBackground.js
    • [DONE] testOpenInForeground.js
  • Technical Tools
    • [DONE] testAccessPageInfoDialog.js
  • Toolbar
    • [DONE] testBackForwardButtons.js

Phase II

Goal Create remaining set of tests which are doable in Mozmill 1.5.x
Status On Hold - Open to community contributions
Bug bug 604708
Test Pages
  • Total: 0 / 2 (0%)
  • [unassigned] google_suggestions.html: a page for displaying Google Search suggestions
  • [unassigned] delayed.html: a page which takes > 5 seconds to load
Tests
  • Total: 0 / 4 (0%)
  • Forms
    • [unassigned] testAutoCompleteOff.js
  • General
    • [unassigned] testGoogleSuggestions.js
    • [unassigned] testStopAndReloadButtons.js
    • Note: Both should be moved to testLayout
  • Preferences
    • [unassigned] testPreferredLanguage.js

Phase III

Goal All remaining set of tests local
Status Not Started - Pending completion of Phase II, Mozmill-2.0 features, Infrastructure
Bug bug 604716
Test Pages
  • Local page for serving add-ons
  • Local pages for serving various certificates and security conditions
  • Local page for serving cookies from various domains
Tests
  • Total: 0 / 27 (0%)
  • Restart
    • testDefaultBookmarks/test1.js
    • testExtensionInstallGetAddons/test1.js
    • testExtensionInstallGetAddons/test2.js
    • testExtensionInstallUninstall/test1.js
    • testExtensionInstallUninstall/test2.js
    • testExtensionInstallUninstall/test3.js
    • testMultipleExtensionInstallation/test1.js
    • testMultipleExtensionInstallation/test2.js
    • testSoftwareUpdateAutoProxy/test1.js
    • testSoftwareUpdateAutoProxy/test2.js
    • testThemeInstallUninstall/test1.js
    • testThemeInstallUninstall/test2.js
    • testThemeInstallUninstall/test3.js
  • Cookies
    • testRemoveAllCookies.js
  • Search
    • testGetMoreSearchEngines.js
  • Security
    • testBlueLarry.js
    • testEncryptedPageWarning.js
    • testGreenLarry.js
    • testGreyLarry.js
    • testSafeBrowsingNotificationBar.js
    • testSafeBrowsingWarningPages.js
    • testSecurityInfoViaPadlock.js (1.9.1-only)
    • testSecurityNotification.js
    • testSSLDisabledErrorPage.js
    • testSubmitUnencryptedInfoWarning.js
    • testUnknownIssuer.js
    • testUntrustedConnectionErrorPage.js