Confirmed users
1,759
edits
Mobile/Fennec/Android/Testing: Difference between revisions
→Advanced setup
| (5 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
This page has instructions for running Firefox tests locally on a device (Android phone, tablet, or emulator) of your choice. | This page has instructions for running Firefox tests locally on a device (Android phone, tablet, or emulator) of your choice. | ||
Having trouble? Ping :gbrown on #mobile | Having trouble? Ping :gbrown on #mobile. | ||
== For the impatient... == | == For the impatient... == | ||
| Line 14: | Line 14: | ||
mach cppunittest | mach cppunittest | ||
mach geckoview-junit | mach geckoview-junit | ||
mach web-platform-test | |||
mach marionette-test | |||
mach test | |||
They all run against a connected Android device using your Firefox for Android build. Don't have a device? These commands will offer to start an emulator. | They all run against a connected Android device using your Firefox for Android build. Don't have a device? These commands will offer to start an emulator. | ||
| Line 142: | Line 145: | ||
mach robocop <test-name> | mach robocop <test-name> | ||
Notes: | Notes: | ||
* To run one test at a time, find the test name (like "testLoad") in <tt>mobile/android/tests/browser/robocop/robocop.ini</tt> and pass it as an argument, like: <tt>mach robocop testLoad</tt>. | * To run one test at a time, find the test name (like "testLoad") in <tt>mobile/android/tests/browser/robocop/robocop.ini</tt> and pass it as an argument, like: <tt>mach robocop testLoad</tt>. | ||
* A rooted device is | * A rooted device is required. Test harnesses may need to kill processes, copy and delete files, or perform other operations which may require special permissions on some devices. | ||
*Additional tips at [[Auto-tools/Projects/Robocop#Frequently_found_errors]] | * Additional tips at [[Auto-tools/Projects/Robocop#Frequently_found_errors]] | ||
=== Static analysis === | === Static analysis === | ||
| Line 228: | Line 223: | ||
Notes: | Notes: | ||
* A rooted device is | * A rooted device is required. Test harnesses may need to kill processes, copy and delete files, or perform other operations which may require special permissions on some devices. | ||
* Setup can take several minutes! Setup is faster if unzip is available on the remote device; if your device does not have unzip, try installing busybox. | * Setup can take several minutes! Setup is faster if unzip is available on the remote device; if your device does not have unzip, try installing busybox. | ||
| Line 286: | Line 281: | ||
== Running tests on the Android emulator == | == Running tests on the Android emulator == | ||
The "Android 4.3 API16+ opt" and "Android 4.3 API16+ debug" tests on treeherder run in an Android ARM emulator. "Android | The "Android 4.3 API16+ opt" and "Android 4.3 API16+ debug" tests on treeherder run in an Android ARM emulator. "Android 7.0 x86_64 opt/debug" tests run in an Android x86 emulator. For best results reproducing test failures, try server is recommended: '''Running the same tests on the same emulator on different host hardware may produce different results'''. | ||
Still, if you want to run the emulator locally, using the same Android image used for tests on treeherder, it is simple: | Still, if you want to run the emulator locally, using the same Android image used for tests on treeherder, it is simple: | ||
| Line 294: | Line 289: | ||
That 'android-emulator' command will download the Android 4.3 API16+ Android image from tooltool, install it, and launch the Android emulator using all the same parameters used for tests on treeherder. (The Android SDK must be installed locally. mach will try to find the emulator binary in your $PATH environment variable, via the $ANDROID_SDK_ROOT environment variable, through your Android build configuration, and finally in the default location used by 'mach bootstrap'.) | That 'android-emulator' command will download the Android 4.3 API16+ Android image from tooltool, install it, and launch the Android emulator using all the same parameters used for tests on treeherder. (The Android SDK must be installed locally. mach will try to find the emulator binary in your $PATH environment variable, via the $ANDROID_SDK_ROOT environment variable, through your Android build configuration, and finally in the default location used by 'mach bootstrap'.) | ||
To use the Android | To use the Android 7.0 x86_64 image: | ||
./mach android-emulator --version x86 | ./mach android-emulator --version x86-7.0 | ||
(On Linux, the x86 emulator requires that kvm is installed. The resulting emulator is much faster than the arm emulator.) | (On Linux, the x86 emulator requires that kvm is installed. The resulting emulator is '''much''' faster than the arm emulator.) | ||
The first time you run an emulator with any particular version, it may take several minutes to download and install the image; subsequent runs will be '''much''' faster. | The first time you run an emulator with any particular version, it may take several minutes to download and install the image; subsequent runs will be '''much''' faster. | ||
| Line 312: | Line 307: | ||
./mach android-emulator && ./mach install && ./mach mochitest | ./mach android-emulator && ./mach install && ./mach mochitest | ||
For | For your convenience, most mach test commands check that an Android device is connected; if not, they suggest running an emulator. Similarly, if tests are requested on a device that doesn't have Firefox installed, mach will offer to install it. So if you just run "mach mochitest" without a phone connected and without an emulator running, you might get: | ||
$ ./mach mochitest testing/mochitest/tests/Harness_sanity | $ ./mach mochitest testing/mochitest/tests/Harness_sanity | ||
| Line 331: | Line 326: | ||
... | ... | ||
=== | === Multiple emulators/devices === | ||
It is possible to test with multiple emulators or devices using the --deviceSerial argument: | |||
$ adb devices | $ adb devices | ||
List of devices attached | List of devices attached | ||
| Line 361: | Line 346: | ||
=== Advanced setup === | === Advanced setup === | ||
Alternatively, you can build desktop Firefox with a mozconfig which might be as simple as: | Alternatively, you can build desktop Firefox with a mozconfig which might be as simple as: | ||
| Line 395: | Line 374: | ||
--remoteTestRoot=<remote-directory> | --remoteTestRoot=<remote-directory> | ||
== Trouble-shooting testing problems == | == Trouble-shooting testing problems == | ||
| Line 431: | Line 390: | ||
** Make sure _SERVER_ADDR in the test output is the same as your machine's IP address. | ** Make sure _SERVER_ADDR in the test output is the same as your machine's IP address. | ||
* If using MOZ_HOST_BIN, ensure the binaries in your MOZ_HOST_BIN folder are executable, in case you pull them down from the FTP site. You might see "OSError: [Errno 13] Permission denied" if they are not executable. Use chmod to fix them. Check certutil, pk12util and ssltunnel. | * If using MOZ_HOST_BIN, ensure the binaries in your MOZ_HOST_BIN folder are executable, in case you pull them down from the FTP site. You might see "OSError: [Errno 13] Permission denied" if they are not executable. Use chmod to fix them. Check certutil, pk12util and ssltunnel. | ||
* Is your device rooted? Many test suites require root permissions. Don't want to root your device? Consider using an emulator with 'mach android-emulator'. | |||