QA/Desktop Firefox/CC OSX

From MozillaWiki
Jump to: navigation, search

Cross-Compile OSX Testing

Lead Contributor: Mihaela Velimiroviciu

Goals

  • Ensure cross-compiled OSX builds work across all supported versions of OSX to the same standard as Firefox Nightly

Methodology

  • Run Mozmill Functional tests on Mac OSX 10.6/10.7/10.8/10.9 and compare results to the latest mozilla-central results
  • Run Mozmill Endurance tests on Mac OSX 10.6/10.7/10.8/10.9 and compare results to the latest mozilla-central results

Results

Functional Tests

Reports
Failures
  • Initial attempts to run Mozmill tests failed due to missing line in application.ini
    SourceRepository=http://hg.mozilla.org/mozilla-central
  • /testGeolocation/testShareLocation.js :: testVerifyDisplayGeolocationNotification
    Geolocation position is: Position acquisition timed out
  • /testInstallation/testBreakpadInstalled.js :: setupModule
    Cc['@mozilla.org/toolkit/crash-reporter;1'] is undefined

Endurance Tests

Reports
Resident Memory
OSX Version Cross-compile Latest Nightly
10.6 131 / 493 / 188 137 / 226 / 177
10.7 135 / 773 / 221 152 / 225 / 187
10.8 141 / 733 / 222 153 / 237 / 193
10.9 135 / 159 / 144 149 / 216 / 165
Failures
  • /testBookmarks_OpenAndClose/test1.js - test1.js::testOpenAndCloseAllBookmarks - New tab has been opened
    • TimeoutError@resource://mozmill/modules/utils.js:449 waitFor@resource://mozmill/modules/utils.js:487 Assert_waitFor@resource://mozmill/stdlib/securable-module.js -> file:///var/folders/c8/tp1nf4sx1tlf5sx1smgmv4rc0000gn/T/tmphEDUA1.mozmill-tests/lib/assertions.js:555 tabBrowser_openTab@resource://mozmill/stdlib/securable-module.js -> file:///var/folders/c8/tp1nf4sx1tlf5sx1smgmv4rc0000gn/T/tmphEDUA1.mozmill-tests/lib/tabs.js:504 testOpenAndCloseAllBookmarks/ file:///var/folders/c8/tp1nf4sx1tlf5sx1smgmv4rc0000gn/T/tmphEDUA1.mozmill-tests/tests/endurance/testBookmarks_OpenAndClose/test1.js:49 endurance_loop@resource://mozmill/stdlib/securable-module.js -> file:///var/folders/c8/tp1nf4sx1tlf5sx1smgmv4rc0000gn/T/tmphEDUA1.mozmill-tests/lib/endurance.js:127 testOpenAndCloseAllBookmarks/<@resource://mozmill/modules/frame.js -> file:///var/folders/c8/tp1nf4sx1tlf5sx1smgmv4rc0000gn/T/tmphEDUA1.mozmill-tests/tests/endurance/testBookmarks_OpenAndClose/test1.js:47 endurance_run@resource://mozmill/stdlib/securable-module.js -> file:///var/folders/c8/tp1nf4sx1tlf5sx1smgmv4rc0000gn/T/tmphEDUA1.mozmill-tests/lib/endurance.js:104 testOpenAndCloseAllBookmarks@resource://mozmill/modules/frame.js -> file:///var/folders/c8/tp1nf4sx1tlf5sx1smgmv4rc0000gn/T/tmphEDUA1.mozmill-tests/tests/endurance/testBookmarks_OpenAndClose/test1.js:42 Runner.prototype.wrapper@resource://mozmill/modules/frame.js:597 Runner.prototype._runTestModule@resource://mozmill/modules/frame.js:667 Runner.prototype.runTestModule@resource://mozmill/modules/frame.js:713 Runner.prototype.runTestFile@resource://mozmill/modules/frame.js:546 runTestFile@resource://mozmill/modules/frame.js:725 Bridge.prototype._execFunction@resource://jsbridge/modules/server.js:179 Bridge.prototype.execFunction@resource://jsbridge/modules/server.js:183 @resource://jsbridge/modules/server.js:283
  • /testBookmarks_OpenAndClose/test1.js - test1.js::teardownModule - this.controller.window.gBrowser.browsers is undefined
    • @resource://mozmill/modules/controller.js:1302 tabBrowser_closeAllTabs@resource://mozmill/stdlib/securable-module.js -> file:///var/folders/c8/tp1nf4sx1tlf5sx1smgmv4rc0000gn/T/tmphEDUA1.mozmill-tests/lib/tabs.js:165 teardownModule@resource://mozmill/modules/frame.js -> file:///var/folders/c8/tp1nf4sx1tlf5sx1smgmv4rc0000gn/T/tmphEDUA1.mozmill-tests/tests/endurance/testBookmarks_OpenAndClose/test1.js:34 Runner.prototype.wrapper@resource://mozmill/modules/frame.js:592 Runner.prototype._runTestModule@resource://mozmill/modules/frame.js:689 Runner.prototype.runTestModule@resource://mozmill/modules/frame.js:713 Runner.prototype.runTestFile@resource://mozmill/modules/frame.js:546 runTestFile@resource://mozmill/modules/frame.js:725 Bridge.prototype._execFunction@resource://jsbridge/modules/server.js:179 Bridge.prototype.execFunction@resource://jsbridge/modules/server.js:183 @resource://jsbridge/modules/server.js:283
  • /testBookmarks_OpenAndClose/test1.js - teardownModule - Disconnect Error: Application unexpectedly closed