Oxidation2

From MozillaWiki
Revision as of 02:58, 5 December 2017 by Nnethercote (talk | contribs)
Jump to navigation Jump to search

Oxidation is a project to integrate Rust code in and around Firefox.

Rust support has been required on all platforms since Firefox 54, and the first major Rust components were shipped in Firefox 56 (encoding_rs) and 57 (Stylo). Moving forward, the goal of Oxidation is to make it easier and more pleasant to use Rust in Firefox, and correspondingly to increase the amount of Rust code in Firefox.

This page is intended to serve as the starting point for all matters relating to Rust code in Firefox: the what, the why, and the how.

Guidelines

The goal of this section is to provide some high-level guidelines about when Rust should be used.

In summary, Rust should be used in the following situations.

  • For new components and completely rewritten components there should be a strong bias towards using Rust, especially for code around Firefox but not within Firefox.
  • For existing components it's more complicated!

The following sections have more detail. Ultimately, choice of language for a code component is an engineering decision, with corresponding trade-offs, and is best decided by individual teams.

Rust Strengths

Rust has the following strengths.

  • Memory safety, which prevents crashes and security vulnerabilities.
  • Thread safety, which enables improved performance via parallelism.
  • High expressiveness.
  • A great community.

Rust Weaknesses

One major issue with Rust relates to personnel. There is a wide variety of experience levels within Mozilla, for both coding and reviewing. Also, Rust's learning curve is steep at the start, which can be intimidating.

There are also technical challenges.

  • Compile times are currently slow.
  • Crossing the C++/Rust boundary layer can be difficult, and is relatively slow due to no inlining.
    • Bindgen can be tricky, especially working around clang bugs in different versions and on different platforms
  • Android is currently not a Tier 1 platform.
    • rillian: "As I understand it, the problem is no one has set up tests on a cloud service, so there's no way for upstream to gate work on Android regressions. Qemu emulation is too slow. There aren't many services with arm support, but there are a couple of new options. If someone does the work of setting something up we should be able to get the Rust team to adopt it."
  • Integration with Gecko often requires adding wrapper/glue code. Such code is typically valuable and reusable, however.
  • Sharing components between Gecko and Servo is difficult.
    • The workflow is annoying and inefficient. Furthermore, permission to land directly on autoland is sometimes needed to fix problems, but few people have that.
      • vcs-sync/Phabricator/Lando will hopefully fix this.
    • There are continuous integration mismatches.
    • Servo CI is potentially under-resourced.
    • This is not relevant for Gecko-only Rust components, however.

Recommendations

Therefore, Rust is most suitable in the following situations.

  • For components that are relatively standalone, with small and simple APIs.
    • This minimizes the C++/Rust boundary layer issues.
    • Infrastructure tools that are standalone programs are ideal.
    • Note that it's good software engineering practice to write loosely-coupled components anyway.
  • For components that process untrusted input, e.g. parsers.
  • For components where parallelism can provide big performance wins.
  • For components where Servo has demonstrated success.

In terms of where to keep Rust crates, there are three options, from most-coupled to least-coupled.

  • Put the crate in mozilla-central or in Servo's repository.
    • For binding code, the decision to put it into Gecko or Servo can be difficult. The best choice depend on the details of the binding code in question.
  • Put the crate somewhere else (e.g. a separate GitHub repository), and regularly vendor it into mozilla-central.
    • This works well if relatively few people are working on the crate, and the API isn't too complex.
    • Can be useful if useful development can be done using just that crate, without the rest of Gecko or Servo.
    • This also makes sense for pre-existing third-party crates that we choose to import.
    • This is done by WebRender, for example.
  • Put the crate somewhere else (e.g. crates.io), and use Cargo to access it at build-time.
    • This is only suitable for highly general-purpose crates.

In general, erring on the side of tighter coupling is advisable. For example, the heapsize crate used in memory reporting was moved to crates.io, and then other crates came to depend on it. Later on it needed major API changes, and we ended up replacing it with a new crate called malloc_size_of (stored in Servo's repository) because that was easier than modifying heapsize.

Documentation

Rust in general

Rust in Firefox

Supported Rust versions for Firefox builds

Below are our expectations if everything goes smoothly; we may need newer toolchains to address specific issues. Our ultimate policy is just that release Firefox will not require unstable or beta Rust to build.

  • Firefox 56 (released 2017 September 26) requires Rust 1.17.0 or later.
    • Official builds will use Rust 1.19.0.
  • Firefox 57 (released 2017 November 14) requires Rust 1.20.0.
    • Mozilla-central can start using Rust 1.19.0 language features 2017 August 3.
    • Mozilla-central can start using Rust 1.20.0 features 2017 September 14.
  • Firefox 58 (released 2018 January 16) requires Rust 1.21.0.
    • Mozilla-central can use Rust 1.21.0 language features 2017 October 26.
  • Firefox 59 (released 2018 March 6) requires Rust 1.22.0.
    • Mozilla-central can use Rust 1.22.0 features 2017 December 7.
    • Firefox 59 ESR is expected to continue to work with Rust 1.22 throughout its support lifetime.
  • Firefox 60 (released 2018 May 1) requires Rust 1.23.0.
    • Mozilla-central can use Rust 1.23.0 features 2018 January 18.
  • Firefox 61 (released 2018 June 26) requires Rust 1.25.0.
    • Mozilla-central can use Rust 1.24.0 features 2018 March 6.
    • Mozilla-central can use Rust 1.25.0 features 2018 April 12.

Rust Components

Within Firefox

Completed

  • MP4 metadata parser: bug 1161350 (shipped for desktop in Firefox 48)
    • Why Rust? Parses untrusted input, replaces libstagefright, a 3rd-party library with a history of security vulnerabilities.
  • Replace uconv with encoding-rs: bug 1261841 (shipped in Firefox 56)
  • CSS style calculation (from Servo): bug stylo (shipped for desktop in Firefox 57)
    • Why Rust? Code taken from Servo, uses parallel algorithms.
  • U2F HID backend: bug 1388843 (shipped in Firefox 57)

In progress

  • URL parser: bug url-oxidation
  • WebM demuxer: bug 1267492
  • WebRender: bug webrender
    • Why Rust? Code taken from Servo, has high performance; Rust's memory and thread safety provides protection against complexity.
  • Audio remoting for Linux: bug 1362220
  • SDP parsing in WebRTC: bug 1365792 (aiming for Firefox 59)
  • Linebreaking with xi-unicode: bug 1290022
  • Optimizing WebVM compiler backend: cretonne
    • Why Rust? Well-separated component with a clear interface.

Proposed

  • Parallel layout
    • Code from Servo.
  • Replace the XML parser
    • Why Rust? Parses untrusted input, replaces expat, a 3rd-party library with a history of security vulnerabilities.
  • WebMIDI: bug 1201593, bug 1201596, bug 1201598
  • Gamepad code: bug 1286699
  • Replace the telemetry module(?)
    • Why Rust? The existing C++ code has a history of threading problems.
  • Share media playback stack with Servo(?)
  • Replace DOM serializers (XML, HTML for Save As.., plain text)
  • Image decoders?
    • Why Rust? Parsing untrusted input, some history of security vulnerabilities.

Outside Firefox

Completed

Blockers and obstacles

This section lists areas where Rust integration could be improved.

  • Tracking bug: Make the developer experience for Firefox + Rust great: bug rust-great
  • Bindings/interop
    • No IPDL binding generator
    • No WebIDL binding generator for DOM components (Servo must have something here?)
    • No XPIDL binding generator (bug 1293362)
    • Immature rust-bindgen and cheddar/cbindgen tools for general cross-language support.
  • Compile speed
  • Management (landing, merging, etc.) of Shared Gecko/Servo components
  • Remaining minor crash report issues bug 1348896
  • Debugging: improve gdb and lldb support for Rust. The first step is to establish Rust language support in DWARF distinct from the existing C++ support.
  • IDE/symbol lookup support?
  • Code coverage?
  • Profiling improvements? Especially for parallel code
  • Test integration?

Meetings

  • Mozlando [1]
  • Oxidation Nov. 2015 [2]
  • Mozlandia [3]
  • Whistler [4]


Retrieved from "https://wiki.mozilla.org/index.php?title=Oxidation2&oldid=1185294"