Static linking for rust without glibc - scratch image

Alright,

a few considerations about MUSL & static linking.

The standard way you are describing works fine for single threaded applications that are not performance sensitive. However, the moment you use anything async, concurrency, or even remotely performance sensitive you will measure between a 10x and 40x drop in performance on all metrics, for example nr requests, latency, whatever.

The TWEAG Blog explains the details quite well. Also, I added more links to background information and benchmarks in the repo linked at the end of this post:
SUPERCHARGING YOUR RUST STATIC EXECUTABLES WITH MIMALLOC

The root-cause isn't MUSL per se, but the default memory allocator used by MUSL because it suffers severely from thread congestion. The new ng allocator isn's solving anything yet, unfortunately.

What can you do to have a single static binary in a scratch image while still persevering good performance?

In a nutshell, you have to patch the memory allocator in MUSL. I know this sounds crazy, but I actually did that and stuffed everything in an easy to use multi-arch build container.

The way you use this is very similar to any other build container:

  1. Build the build container and publish it in the registry of your choice. I've added an example for GCP cloud build and a GH action for that.

  2. Use that build container to build your Rust code as single static binary.

  3. Stuff this single static binary in a scratch image and your done.

This is identical to any other multi-stage build with the exception that the patched memory allocator makes sure your static binary runs as fast and in some cases even faster than the glibc linked version.

The thing with scratch images is, you have to add SSL certificates and a user / group manually to ensure your can handle secure networking and run in unprivileged mode. Again, I made a simple example with Dockerfile for that.

Hope that helps.

Links:

Build container with patched memory allocator. Feel free to fork & customize:

Rust scratch image with SSL certificates and unprivileged user.

Sample Dockerfile to build a static binary with patched memory allocator and stuff it into a custom scratch image with SSL certs.

2 Likes