User:NThomas:Mar Generation
The aim of this page is to document changes in format of the mar container, and the files within it.
Contents
Big picture
A 'mar file' is an update to a Mozilla application, an archive of files to add/patch/replace as well as a 'manifest' of instructions to be performed. It is delivered by a query to an update server, which is out of scope for this page, we only want to talk about how the file is structured and created.
MAR archive
Relevant directories: modules/libmar
Software_Update:MAR is the main documentation for the MAR format. To date the only major revision is ...
Signing
From Version: Firefox/Gecko 12.0; bug 481815.
A maintenance service was added on Windows, which can install updates without user intervention (no password or UAC prompts). It landed in mozilla-central during the Firefox 12.0 cycle (see comment 481), and rode the trains to release. Initially it was for administrator accounts, but bug 711475 extended it to limited users accounts in Firefox 26.0.
To secure the update process a digital signature is included in the mar file, as well as a product information block. The latter labels the mar with a
- ProductVersion - the application of the bits contained by mar file
- ChannelID - the combination of the product name and code branch, eg firefox-mozilla-central, firefox-mozilla-beta.
These are set when the mar utility creates a mar file. Default values are set at compile time, but can be overridden by -V and -H arguments when mar is called. A signature can be added to an existing mar using the signmar utility; Release Engineering set up automated signing in bug 509158.
When it's time to apply an update, the updater checks the signature of the mar file using a builtin certificate, and uses the product information block to ensure that the version doesn't decrease, and that ChannelID is allowed in <install_dir>/update-settings.ini. More information about the process flow is available at Windows_Service_Silent_Update.
Older copies of the updater (ie before v12.0) ignore the signature and product information block, so we can serve 'modern' mar files to old applications.
Signing of all platforms started with bug 974570 for Firefox 30.0, in preparation for verifying mar files on all platforms bug 973933.
MAR contents
Relevant directories
- tools/update-packaging/ - scripts to generate mar files
- other-licenses/bsdiff/ - mbsdiff, which creates patch files
- toolkit/mozapps/update/updater - the updater executable
The mar contains a set of files belonging to the application being updated, and one (or more) manifests which describe how to apply them. When stored in the mar, the files are all bzip2 compressed.
Application files
There are two types of files - patches created by mbsdiff, and full copies. The patches encode a CRC to ensure the local file is as expected.
Update manifests
The manifest supports various actions, eg ADD (aka overwrite), PATCH, REMOVE. The updater processes the manifest in three passes - PREPARE, EXECUTE, and FINISH.
update.manifest
From Version: Gecko 1.8.0, Firefox/Thunderbird 1.5
Last Version: Gecko/Firefox 29.0, Thunderbird 24.x.y (End of 24 ESR); bug 896224|, bug 896223
This initial format dates back to the introduction of the current update system at Firefox 1.5 (aka Mozilla 1.8.0 branch). It supports the following actions:
ADD <file> # replace <file> ADD-IF <testfile> <file> # replace <file> if <testfile> exists PATCH <patchfile> <filetopatch> # apply binary diff <patchfile> to <filetopatch> PATCH-IF <testfile> <patchfile> <filetopatch> # apply <patchfile> to <filetopatch> if <testfile> exists REMOVE <file> # delete <file>
Support for v1 was removed in Gecko/Firefox 30.0 by bug 896224 (updater) and bug 896223 (mar generation).
updatev2.manifest
From Version: Gecko/Firefox/Thunderbird 5.0; bug 386760
Modified Version: Gecko/Firefox/Thunderbird 12.0; bug 660038 Removal of ADD-CC and REMOVE-CC;
A new manifest, updatev2.manifest, was introduced to add support for directory removal and channel changing. The new actions were
RMDIR <deaddir>/ # remove an empty directory <deaddir> RMRFDIR <deaddir>/ # recursively empty and remove <deaddir>ADD-CC <file> # replace <file> if the channel is changingfunctionality removed by bug 660038 TYPE <type> # 'partial' or 'complete', must be first line of manifest
The updater looked first for updatev2.manifest, and if not present looked for update.manifest.
In addition, a precomplete file was added, which describes how to uninstall the current application using a subset of instructions:
REMOVE <file> # delete <file>REMOVE-CC <file> # remove <file> if channel is changingfunctionality removed by bug 660038 RMDIR <deaddir>/ # remove an empty directory <deaddir>
The precomplete is used when a complete update is to be applied, removing the existing install before laying down the new set of files. In a partial update, deprecated files are deleted by REMOVE isntructions, and precomplete gets a PATCH. This change meant we no longer needed to maintain a list of files to remove on update (ie with browser/installer/removed-files.in), which greatly simplified update testing for rapid releases.
bug 386760 was also where the mar generation changes happened, with channel change removals in {{bug|660038}.
updatev3.manifest
From Version: Gecko/Firefox 29.0, Thunderbird 31.0; bug 759469
This new manifest adds support for a new instruction
ADD-IF-NOT <testfile> <newfile> # add <newfile> if <testfile> exists
The updater tries to read updatev3.manifest, and falls back to updatev2.manifest if needed. Support for update.manifest was removed at a very similar time (see above).
This change was primarily in support of bug 908134 to offer give release builds to users on the beta channel. We needed to exclude update-settings.ini from mar files, otherwise beta users would be querying on the beta channel but only able to install mar files with the release channel-id. ADD-IF-NOT lets us replace the file if it had gone missing on the user side, although not properly in the beta user case.
bug 900251 handled the mar generation changes.
