Packaging Android host utilities: Difference between revisions

update mac os x build directions to handle artifact not being produced by the automated systems
(wildcard not needed for fuzzy try syntax)
(update mac os x build directions to handle artifact not being produced by the automated systems)
 
(10 intermediate revisions by 3 users not shown)
Line 1: Line 1:
== Packaging Android host utilities ==
== Packaging Android host utilities ==


Host utilities are executable files that run on a *host* machine. These utilities provide services to an Android *target* device, such as running a web server or a certificate authority. Host utilities must be periodically updated with a good build from mozilla-central or mozilla-inbound due to addition of new tests targeted at Android and/or changing of component behavior.
Host utilities are executable files that run on a *host* machine. These utilities provide services to an Android *target* device, such as running a web server or a certificate authority. Host utilities must be periodically updated with a good build from mozilla-central or autoland due to addition of new tests targeted at Android and/or changing of component behavior.


This page will provide instruction on how to package new versions of the host utilities.
This page will provide instruction on how to package new versions of the host utilities.
Line 7: Line 7:
=== Generate new archive ===
=== Generate new archive ===


On Treeherder, identify a target build, preferably from mozilla-central or mozilla-inbound. Check the test output to ensure a reasonably good build (one without too many failures).
On Treeherder, identify a target build, preferably from mozilla-central or autoland. Check the test output to ensure a reasonably good build (one without too many failures).


example: for the version 66.0 update, [https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=c9fa642e9e3aecaa8852f5c647904eecddaa4450&selectedJob=216676591 this build] was selected. For more details, please see [https://bugzilla.mozilla.org/show_bug.cgi?id=1514075 bug 1514075].
example: for the version 66.0 update, [https://treeherder.mozilla.org/#/jobs?repo=mozilla-central&revision=c9fa642e9e3aecaa8852f5c647904eecddaa4450&selectedJob=216676591 this build] was selected. For more details, please see [https://bugzilla.mozilla.org/show_bug.cgi?id=1514075 bug 1514075].
Line 39: Line 39:


Preparing for macOS is not quite so simple. Due to requirements of macOS, it's best to do this on a Mac OS X machine so that you can codesign the host utility binaries. If binaries are unsigned, users will be prompted at every invocation to "allow network connections". For more information on signing macOS binaries, see [http://apple.stackexchange.com/a/150711 here].
Preparing for macOS is not quite so simple. Due to requirements of macOS, it's best to do this on a Mac OS X machine so that you can codesign the host utility binaries. If binaries are unsigned, users will be prompted at every invocation to "allow network connections". For more information on signing macOS binaries, see [http://apple.stackexchange.com/a/150711 here].
# From the selected treeherder build, identify '''macOS opt'''.
# Click on the green B icon, which will bring up a pane covering bottom third of the window.
# On the bottom left pane is a header named '''Task'''. Click on the hash.
# A new tab will open and load Taskcluster. Click on tab named "Run Artifacts".
# Download target.common.tests.tar.zst
# Contact Releng (required to get a non-self-signed set of binaries), and have them follow the contents of the script below:
<pre>
tar xvf firefox-66.0a1.en-US.mac.common.tests.tar.gz 'bin/*'
open firefox-66.0a1.en-US.mac.dmg
cp -R /Volumes/Firefox\ Nightly/Firefox\ Nightly.app/Contents/MacOS/* bin
cp -R /Volumes/Firefox\ Nightly/Firefox\ Nightly.app/Contents/Resources/* bin
find bin -type f -perm +111 -print | grep -v \\. | xargs sudo codesign --force --deep --sign
mv bin host-utils-66.0a1.en-US.mac
tar cvf host-utils-66.0a1.en-US.mac.tar host-utils-66.0a1.en-US.mac/*
gzip host-utils-66.0a1.en-US.mac.tar
</pre>
===== prior process  =====
This isn't working now because we no longer publish the artifact.


# Navigate to [https://ftp.mozilla.org/pub/firefox/nightly/ Mozilla FTP].
# Navigate to [https://ftp.mozilla.org/pub/firefox/nightly/ Mozilla FTP].
Line 56: Line 78:
tar cvf host-utils-66.0a1.en-US.mac.tar host-utils-66.0a1.en-US.mac/*
tar cvf host-utils-66.0a1.en-US.mac.tar host-utils-66.0a1.en-US.mac/*
gzip host-utils-66.0a1.en-US.mac.tar
gzip host-utils-66.0a1.en-US.mac.tar
</pre>
==== Windows (experimental) ====
Similar to Linux. Follow these steps to locate and download the necessary files
# From the selected treeherder build, identify '''Windows 2012 debug''' ('''Windows 2012 opt''' isn't being built any longer).
# Click on the green B icon, which will bring up a pane covering bottom third of the window.
# On the bottom left pane is a header named '''Task'''. Click on the hash.
# A new tab will open and load Taskcluster. Click on tab named "Run Artifacts".
# Download target.common.tests.tar.gz.
# Download target.zip
# Follow the contents of the script below:
<pre>
unzip target.zip
mkdir temp_common
tar xvf target.common.tests.tar.gz -C 'temp_common'
rm firefox/firefox*
rm -r firefox/browser
mv 'temp_common'/bin/* firefox
mv firefox host-utils-66.0a1.en-US.win32
tar cvf host-utils-66.0a1.en-US.win32.tar host-utils-66.0a1.en-US.win32
gzip host-utils-66.0a1.en-US.win32.tar
</pre>
</pre>


Line 70: Line 115:
python tooltool.py upload --authentication-file=[token_location] --message [commit_message]
python tooltool.py upload --authentication-file=[token_location] --message [commit_message]
</pre>
</pre>
# Update the manifest in testing/config/tooltool-manifests/linux64/hostutils.manifest.
# Update the manifest in testing/config/tooltool-manifests/macosx64/hostutils.manifest.


If updating host utilities for Linux, repeat using 32bit/x86 archives.
If updating host utilities for Linux, repeat using 32bit/x86 archives.
Do the same for Windows. testing/config/tooltool-manifests/win32/hostutils.manifest


=== Verification ===
=== Verification ===
Line 84: Line 131:
# Update to current tip by hg pull and hg update -r tip -C
# Update to current tip by hg pull and hg update -r tip -C
# Copy and paste the updated manifest to the hostutils.manifest file for Linux 64 and Linux 32.
# Copy and paste the updated manifest to the hostutils.manifest file for Linux 64 and Linux 32.
# Commit change and run moz-phab.
# Initiate a try run of all Android tests:
# Initiate a try run of all Android tests:


<pre>
<pre>
./mach try fuzzy -q=android-em-
# all (use this by default)
./mach try fuzzy --no-artifact -q='test-android-'
 
# only hardware
# ./mach try fuzzy --no-artifact -q=test-android-hw-
# only emulator
# ./mach try fuzzy --no-artifact -q=test-android-em-
</pre>
</pre>


Line 94: Line 148:
Changes to host utilities for macOS cannot be tested on try server, therefore the following steps are taken:
Changes to host utilities for macOS cannot be tested on try server, therefore the following steps are taken:


# Update to current tip by hg pull and hg update -r tip -C
# Copy and paste the updated manifest to the hostutils.manifest file for Mac.
# Commit change and run moz-phab.
# Ensure the currently selected MOZCONFIG designates an Android build target.  
# Ensure the currently selected MOZCONFIG designates an Android build target.  
# Sanitize and rebuild with the following:
# Sanitize and rebuild with the following:
<pre>
<pre>
hg pull
hg update -r tip -C
./mach clobber
./mach clobber
./mach build
./mach build
Line 104: Line 159:
# Once build is complete, package the apk using ./mach package
# Once build is complete, package the apk using ./mach package
# Install on device with ./mach install
# Install on device with ./mach install
# Run a short test with ./mach robocop testAboutPage
# Run a short test with ./mach mochitest testing/mochitest/tests/Harness_sanity
# Accept offer to update host utilities.
# Accept offer to update host utilities.


21

edits