B2G/Updating

Note: The latest version of this document can be found on MDN at Creating and applying Firefox OS update packages. Further updates should be made there.

This page intends to help engineers set up an environment for updating B2G devices. It focuses on low-level details of the updating tools and omits many higher-level topics. The higher-level issues are covered in XXX other doc.

Updating is the process of (i) on a build host, building incremental update packages from old version(s) to a new version; (ii) on the client, finding the right update to download; (iii) downloading the update; (iv) applying the update to existing files on the device. Each of these items are covered below.

Types of updates

B2G phones can update the entire system through FOTA updates. The technology behind FOTA updates is shared with the Android project. The locations on the phone's hard drives that can be changed using FOTA updates include the system partition, kernel, modem baseband, recovery image used for updating, or any other file on the device.

B2G can also update just the Gecko and Gaia files on device, through a mechanism we call Gecko/Gaia OTA updates. All of the Gecko and Gaia files are in the /system/b2g directory on the device. They comprise the core Gecko runtime and the device's user interface.

FOTA updates

B2G does not depend on a particular FOTA client; the interface is abstracted through an API we call "librecovery". However, we recommend using the "GOTA" recovery client, and the discussion here assumes GOTA is being used.

FOTA update packages mainly consist of a file called update.zip. This package consists of

• a set of binary diffs and new files required to update the client to the newer software version
• an "update script" that controls how the diffs and new files are loaded onto the client
• an internal signature used to verify the update package

Again, this format and set of files are the same as those used in normal updates of Android OSes.

B2G additionally wraps this update.zip package in a mar wrapper. This mar wrapper allows an additional level of verification that's explained below.

Gecko/Gaia OTA updates

Gecko/Gaia OTA updates use the same technology that's used to update the Firefox desktop web browser. These update packages are called "MARs", which stands for "Mozilla ARchive". Much like the update.zip packages discussed above, MARs consist of a set of binary diffs and new files needed to update the client to a newer software version.

The Gecko client verifies the integrity of MARs that it downloads, and MARs can be signed by multiple parties.

Again, on B2G clients, Gecko/Gaia OTA updates can only change files in the /system/b2g directory.

Why have two update technologies?

This question is not covered in detail in this document; please refer to XXX other doc. The short answer is that

• Gecko/Gaia OTA updates can be applied "in the background", while B2G continues to run normally. This provides a much better user experience because users don't need to reboot their phone and wait while an update is applied. Instead, the update is applied while the user continues to use the phone, and when the update is finished the user only needs to agree to restart the main b2g process. This takes a matter of seconds, instead of the minutes that are usually required to apply FOTA updates. Therefore, since most updates will only need to change files in /system/b2g, we trade off the full generality of FOTA updates for a better user experience with Gecko/Gaia OTA updates in the common case.
• Gecko/Gaia OTA update packages can sometimes be smaller than FOTA update packages, though not always; they should never be larger. This means that users can sometimes have less data to download.

Building updates for multiple software versions

Building updates is the process of generating the files needed to update B2G clients from version X of the software to a newer version Y.

The update package that's needed to update the client depends on what files have changed between version X and Y.

• If only files in /system/b2g have changed, we will generate a Gecko/Gaia OTA update
• If any file in a location outside of /system/b2g changed, we will generate a FOTA update

To generate an incremental update package (for both FOTA and Gecko/Gaia OTA updates), our tools require full builds for both version X and version Y. "Full build" means that the package includes all the files that are needed to image a client with version X or Y. When we produce a full build for version X, we don't know which future versions Y etc. we will be updating to from version X. Because of that, we build both full FOTA packages and Gecko/Gaia packages for each version. This allows us to generate either a Gecko/Gaia OTA incremental update, or a FOTA incremental update if needed, between version X and all future versions Y.

So at a high level, the process of building an update looks like

1. With software version X
   • Generate a complete Gecko/Gaia OTA MAR for the contents of /system/b2g
   • Generate a complete FOTA update.zip and target files zip for the device partitions
2. With software version Y
   • Generate a complete Gecko/Gaia OTA MAR for the contents of /system/b2g
   • Generate a complete FOTA update.zip and target files zip for the device partitions
3. If only files in /system/b2g have changed, generate an incremental Gecko/Gaia update MAR from X to Y
4. Otherwise, generate an incremental FOTA update.zip from X to Y. Wrap the incremental FOTA update.zip in a MAR for delivery to the B2G client.
5. Sign the packages as required by delivery agreements

The remainder of this section describes how to use B2G's tools to implement each of these steps.

Note: the steps below assume that you have already set up a b2g build environment at the location $b2g. The commands below reference the $b2g/build.sh helper script, but make can also be used.

Generating a complete Gecko/Gaia OTA update MAR

Invoke the gecko-update-full target to generate a complete update MAR for the last successful b2g build. This will place the MAR in $b2g/objdir-gecko/dist/b2g-update/b2g-gecko-update.mar:

$ cd $b2g
$ ./build.sh gecko-update-full
$ cp objdir-gecko/dist/b2g-update/b2g-gecko-update.mar <destination>

Generating a complete FOTA update zip and target files zip

The default target in the B2G build system will generate a FOTA update.zip / target files zip when the kernel binary has been copied to the appropriate location under vendor/. This enables boot image, recovery image, and update.zip generation.

The complete FOTA update.zip is generated to the location out/target/product/$DEVICE/$DEVICE-ota-$VARIANT.$USER.zip
The target files zip is generated to the location out/target/product/$DEVICE/obj/PACKAGING/target_files_intermediates/$DEVICE-target_files-$VARIANT.$USER.zip

$ cd $b2g
$ ./build.sh gecko-update-fota
$ cp out/target/product/$DEVICE/obj/PACKAGING/target_files_intermediates/$DEVICE-target_files-$VARIANT.$USER.zip <destination>

Variable values:

Generating an incremental OTA update MAR

In this example, we're assuming that we're generating an update from software version X to version Y. The location of the full Gecko/Gaia OTA MAR built from software version X using the instructions above will be called $MAR_X below. This might be a path on a build server like /home/build/b2g/versions/X/update.mar. Similarly, the location of the full MAR built from version Y will be called $MAR_Y.

The tool build-gecko-mar.py will generate an incremental Gecko/Gaia OTA update MAR using $MAR_X and $MAR_Y. We'll call the destination of the generated file $GENERATED_INCREMENTAL_MAR_X_Y.

$ cd $b2g
$ ./tools/update-tools/build-gecko-mar.py --from $MAR_X --to $MAR_Y $GENERATED_INCREMENTAL_MAR_X_Y

Generating an incremental FOTA update zip

In this example, we're assuming that we're generating an update from software version X to version Y. The location of the full FOTA target files zip build from software version X using the instructions above will be called $TARGET_FILES_X below. This might be a path on a build server like /home/build/b2g/versions/X/target_files.zip. Similarly, the location of the full FOTA target files built from version Y will be called $TARGET_FILES_Y.

The tool build/tools/releasetools/ota_from_target_files will generate an incremental FOTA update.zip using $TARGET_FILES_X and $TARGET_FILES_Y. We'll call the destination of this intermediate file $INTERMEDIATE_FOTA_UPDATE_FOTA_X_Y.

After this update.zip is generated, the last step is to wrap it in a MAR for delivery to the B2G client. The tool tools/update-tools/build-fota-mar.py does this. We'll call the destination of this generated file $GENERATED_INCREMENTAL_FOTA_X_Y

$ cd $b2g
$ ./build/tools/releasetools/ota_from_target_files -v \
    -i $TARGET_FILES_X \
    -p out/host/$HOST_ARCH \
    -k $FOTA_SIGNING_KEY \
    $TARGET_FILES_Y \
    $INTERMEDIATE_FOTA_UPDATE_FOTA_X_Y
$ ./tools/update-tools/build-fota-mar.py $INTERMEDIATE_FOTA_UPDATE_FOTA_X_Y --output=$GENERATED_INCREMENTAL_FOTA_X_Y

Variable values:

Hosting updates (respectively, polling for updates on the client side)

B2G clients poll for updates by fetching and parsing an update manifest. This file is called update.xml.

B2G clients are configured to poll for updates on specific servers. They query a specially-constructed path on the server. HTTPS is the recommended protocol that the client uses to query the server; however HTTP is also supported. The server and path polled by clients can be changed by shipping an update to existing clients that changes the polling code.

In the examples below, let's assume we're hosting updates on the server updates.b2g.com.

The URL polled by the client is mainly parametrized by

However, there are more values that can be used to construct the queried update URL.

The B2G client then uses the value of its configured update host and these values to construct a URL to poll at runtime. An example of such a URL is

https://updates.b2g.com/release/unagi1/18.0/20121203123456/update.xml

If the server returns a "404 Not Found" to the client's request, then there is no update available. If the server returns a "200" and a manifest file, then there may be an update available. The manifest describes the newly available build; that is, the build the client would update to. An example manifest is

<?xml version="1.0"?>
<updates>
  <update type="major" appVersion="19.0" version="19.0" extensionVersion="19.0" buildID="20121210123456"
          licenseURL="http://www.mozilla.com/test/sample-eula.html"
          detailsURL="http://www.mozilla.com/test/sample-details.html">
    <patch type="partial" URL="https://updates.b2g.com/release/unagi1/18.0/20121203123456/update.mar"
           hashFunction="SHA512" hashValue="5111e033875752b7d9b32b4795152dea5ef954cb8a9d4a602dd19a923b464c43521287dcb5781faf3af76e6dc5e8a3dd9c13edea18c1f2c8f3bd89e17d103d6f"
           size="41901319"/>
  </update>
</updates>

The fields in the manifest describe

• metadata used to show a user interface on the client
• metadata about the newly-available version
• the location of the update package
• metadata used to verify the download of the update package

The client device or the user may wish to decline an update.

Using these mechanisms, servers can host updates packages to update any old client version to the newest version. Or they may host only a "linear update history" in which clients must upgrade through a single path.

The details of the interaction between build servers and the update host is currently beyond the scope of this document. It is highly dependent on the production environment.

Verifying and applying updates

After a B2G has successfully polled for an update, downloaded it, and verified the integrity of the downloaded update package, the final step is to apply the update.

The first step in applying an update is to verify the signatures embedded in the MAR packages. This is done by the B2G client itself after checking the integrity of the downloaded package. The code used for this is the same for both FOTA and Gecko/Gaia OTA updates.

After signatures are verified, the process of applying an update diverges between Gecko/Gaia OTA updates and FOTA updates.

Applying Gecko/Gaia OTA updates

The B2G client applies these using the updater binary. This is part of the Gecko distribution and is the same code used to apply updates for desktop Firefox. As described above, the update is applied while the B2G client continues to run normally. Users are able to make and receive calls, run apps, browse the web etc. while updates are being applied.

The specific details of updater binary are beyond the scope of this document, but it works approximately by performing the following steps

• making a copy of the /system/b2g files
• applying binary patches, removing old files, and adding new ones as specified by the MAR file
• restarting the main b2g process so that it uses the new files

After the b2g process finishes restarting, the user is running the new version of the B2G client software.

Applying FOTA updates

The FOTA client applies these. The Gecko client "hands off" the update to be applied by calling into the librecovery API. What happens after this step is specific to each FOTA client.

In the implementation of librecovery used for the GOTA client, the downloaded update package is staged to be applied and special commands are enqueued for the recovery client. librecovery then reboots the device into the recovery mode. The recovery client then runs the update script in the update.zip to update files and partitions as needed. The recovery client may need to reboot multiple times to update all files.

After the final reboot, the use is running the new version of the B2G client software.