Rust in Large Organizations Meeting
At RustConf, employees of several large organizations (Mozilla, Microsoft, Google, Facebook, and more) met with some members of the Rust core team (Niko, Manish, and Alex) to talk about how Rust is currently being used in those organizations.
If you're interested in diving straight into the specifics right away, you can find them in the notes from the meeting here.
Large software engineering organizations often have different requirements than other users of Rust, and so it's interesting to get together to specifically identify these requirements and brainstorm solutions. The goals of the meeting were for members of these large organizations to:
- share how they are solving problems that may be common to all large scale deployments of Rust.
- discuss with members of the Rust project how we can collectively make using Rust in large organizations easier while still ensuring that other users' experience at the very least stays the same if not improves.
There were three main topics of conversation that occurred in this meeting:
- building Rust inside of large, existing build systems
- FFI and unsafe: technical and process concerns
- supporting the Rust project
Rust in Existing Build Systems
Many large organizations have existing build systems that build much more than Rust code and require careful control over how artifacts are built for various reasons including compliance. These requirements often make using the de facto Rust tooling like Cargo difficult. This is because as Cargo exists today, it generally behaves as if it is the only build system being used.
While rustc has similar characteristics to C and C++ compilers making it relatively easy to fit it into existing build systems, there are huge benefits to using Cargo. For example, most Rust tooling and third-party crates assume Cargo. Eschewing Cargo in favor of rustc essentially removes large organizations from the rest of the wider Rust community - a rift that none of us is interested in seeing.
An Example: build.rs
Cargo has many features that, while convenient for most users of Rust, are not acceptable in large existing build systems. For example, build.rs files are currently too powerful and don't have clearly marked inputs and outputs. This makes auditing their usage and caching their results extremely difficult.
Possible ways to fix this are developing ways to constrain build.rs files to have clearly delineated inputs and outputs that make auditing and caching straightforward. Such a feature could potentially make some use cases of build scripts more straight forward (i.e., one liners in a cargo.toml file) which would benefit the entire community.
An overview of build system topics
Many other topics were discussed including:
- bootstrapping rustc in a deterministic way: this is often needed for compliance reasons.
- custom toolchains: some companies might be working on new or proprietary targets.
- more control over the linker: this allows certain optimizations and deterministic outputs.
- controlling dependency resolution and building: aids in the goal of reproducibility, caching and facilitating auditing
- IDE and other tooling support: most Rust tooling assumes a single Cargo project or workspace. What if there is much, much more?
- distributed builds: distributing builds has many advantages including performance.
FFI and Unsafe
While Rust is already very capable when it comes to C FFI (and will continue to get better through the work of the FFI working group), many large organizations have the need for better C++ interoperability. It is possible that all the work that needs to be done can happen inside of the bindgen/cbingen libraries that are already being extensively used, but this remains to be seen.
One thing related to FFI though not exclusively so is the use of unsafe. Many companies have ad-hoc policies on how to control unsafe, and there doesn't seem to be agreement on what the right way to do so would be. Usually these methods including ensuring that certain "privileged individuals" review code if any unsafe code blocks are added or edited.
Beyond this, Facebook is working on verification tooling to ensure safe usage of unsafe as part of the Libra project. Another possible improvement is labeling third party crates on crates.io that have been formally verified or audited by a trusted auditor which could be an easy win towards making the entire ecosystem more safe.
Governance and Support
Lastly, it's clear that many of us in these large organizations are huge fans of Rust. We seen a bright future in the language and want to do our part to help ensure that Rust has a bright future.
We identified several ways we can help that we're already doing including: hiring individuals to work on aspects of the Rust project and sponsoring Rust related events and donating resources (e.g, CI servers). An obvious area where companies are not currently contributing is donating money directly to the Rust project. This is largely because there is no set of entities that represent Rust. For example, if a company wants to support crates.io, there is currently no straight-forward and transparent way to do so.
The conversation has begun around whether there needs to be a set of entities that can represent the Rust project in various capacities and allow organizations or individuals who want to contribute monetarily to the project to do so. But of course, there are obviously lots of questions around governance that first need to be solved.
The responsibility for this lies with the Rust Governance working group who will work with everyone in the community to come to a conclusion that works best. If this is something you'd like to get involved in make sure to join the working group!
Next Steps
There's a lot of work that needs to happen, and it's not quite clear what the priorities of that work will be. Many of us will split up and discuss with relevant parts of the Rust project (e.g., cargo, governance WG, FFI WG, etc.) how we can help to solve some of these issues.
We want to keep the conversations open to the entire community through users.rust-lang.org, the Discord and Zulip, and we'll probably meet again at future Rust events like RustFest in November.
Here's to making the Rust project even better than it is in the future for everyone!