- 1 Quick Links For You
- 2 Quick Links For Us
- 3 About Us
- 4 Contacting Release Engineering
- 5 Contributing
- 6 Team
- 7 Releng Offerings
- 8 Development
- 9 Operations
- 10 Releng Applications
- 11 Mozilla Applications
- 12 External Third Party Dependencies
- 13 Deprecated
- 14 Subpages
We ship Firefox to the world!
Release Engineering does this by:
- Developing world-class, scalable, open, secure, and maintainable infrastructure for application builds and tests, on a diverse set of desktop and mobile platforms
- Building fast and reliable pipelines to transform source code into shipped products
Contacting Release Engineering
Our team is located on 3 continents and in 5 time zones. This allows us to provide better service, especially if you contact us using one of the following methods:
- To request a new service, please file a bug
- To ask a general question, ping in #ci on IRC, or email firstname.lastname@example.org
- To request status on an existing request, set the "needinfo" flag in the bug.
Our team works normal business hours for our respective time zones. If you need to escalate an issue that is both urgent and important, use the following methods:
- Use #ci in IRC or email email@example.com, starting your message with 'URGENT:'. If no response in 15 minutes, then
- Be sure the bug severity is set to "blocker"
- Call one or more of the following people (nagios page preferred if you have access):
If you get voice mail, please leave your callback number.
If you are interested in getting involved with Release Engineering, please see our contributing page! We'd love to hear from you.
Mozilla's Release Engineering:
- twitter: @MozRelEng
- blogs: Planet Release Engineering
- calendar: Google Calendar
- mailing list: firstname.lastname@example.org
|Ben Hearsum||bhearsum||@bhearsum||Ben's blog|
|Justin Wood||Callek||"Callek's Blog"|
|Chris AtLee||catlee||@chrisatlee||chris' random ramblings|
|Jordan Lund||jlund||ghost blog|
|Nick Thomas||nthomas||@nthomasftang||F'tang F'tang Release wrangling at the bottom of the world|
|Rail Aliiev||rail||@railaliiev||Rail's blog|
|Rok Garbas||@garbas||Rok's blog|
|Johan Lorenzo||jlorenzo||Johan's blog|
|Simon Fraser||sfraser||@simonfr||Simon's blog|
As mentioned above, Releng offer pipelines to transform source code into shipped products. The following sections break down how that happens. Each section can be understood as both isolated and combined parts.
Initiating a Release
Configuring & Scheduling
Ship It then creates Taskcluster tasks that are needed to take a builds and release them to users. These tasks make up the release graph that are derived from templates within taskgraph and are submitted via action tasks.
The signing of build artifacts and binaries. Some features of this:
- signing is done through a client/server model
- integrated with Taskcluster thanks to signingscript in conjunction with scriptworker, a securely managed Taskcluster worker type. signingscript tasks are part of the task graph that is generated from Ship It
- keys are generated with modern standards
- keys are secured by our infra and rotated regularly
- coming soon: self served signing through exposed API
As important release artifacts are created, they are then uploaded to archive.m.o where they can served from and live permanently for historical or longstanding support reasons.
Like signing, uploading is integrated with Taskcluster thanks to beetmoverscript in conjunction with scriptworker, a securely managed Taskcluster worker type. signingscript tasks are part of the task graph that is generated from Ship It
Install & Update Tests
Like product tests, releases likewise need tests. This ensures that initial downloads install correctly on all supported platforms and updates work as expected, all before actually releasing. Right now this is done through our Update and Final Verify tests. Which, like all other sections are part of the release graph.
TODO: link to documentation and source of these
Shipping & Updates
Finally, we release the product via both initial downloads and as new updates.
For Desktop Firefox:
- Initial installs are served via updating the download page, mozilla.com (Bedrock) and updating Bouncer, Mozilla's mirroring and redirection server.
- Updates are handled by our own application, Balrog. Balrog is our server side component of the update system
For Android Products, including Firefox:
- releases are submitted to the Play Store
- Python Standards - Our Python coding standards
- Development Best Practices - read this when you're starting something new
- Deployment - How we deploy our tools, and how do we update them.
- Wiki Guidelines - how to wiki garden for fun and profit
Release Engineering has historically been involved with the Build and Release pipeline of a product. As part of that, we have some responsibility to help ensure that the build (continuous integration) and release (deployment) pipeline infrastructure is in good running health. We have two functional groups that monitor and respond to any operational issues that come up.
Buildduty provides support for Firefox's build and test infrastructure
The role of Buildduty has grown beyond Release Engineering's scope. There is a dedicated team of 6 people, providing 24/7 support of Mozilla's build and test infrastructure.
Releaseduty provides support for Firefox's release infrastructure and is currently handled by Releng
See the ReleaseEngineering/Releaseduty home page for more information.
- Applications - Various applications and services that RelEng provides
- TODO update application list with: Services, Scriptworker, ChainOfTrust
- Taskcluster - our task execution tool for providing continuous integration and release pipelines
- Treeherder - aggregates all the build, test, and release tasks as a reporting dashboard
- PuppetAgain - Release Operations tool and puppet configs for configurating build and release machines. Both for workers and hosting applications
- TODO - add Release Operations replacement for Slaveapi: Controller
External Third Party Dependencies
below pages should be deleted or be given a "warning" outdated box where appropriate.
- Release Checklist and Build Notes
- Releng Post Mortem Meetings
- What is releaseduty?
- Release Promotion
- Update Generation