Rust and Docker

Hi,
I have recently tried dockerizing a rust application and it is by far not as simple as I would have imagined. There is the official rust docker image, but the images produced by it are very large since they ship with Ubuntu and (depending on the exact dockerfile) with all the rust tooling.
If you just want to have small images (for instance based on Alpine linux) things get complicated very fast. Is there any plan to change these circumstances? I could imagine that getting docker running for Rust is a relatively low effort for the relatively high effect that we can deploy rust applications in the cloud. I would really love to have it as simple as with golang.

1 Like

What was complicated in particular? You should be able to just copy the built executable and run it, since Rust links statically by default.

Well I am having the issue described here:
https://stackoverflow.com/questions/72424759/rust-in-docker-image-exec-no-such-file-or-directory
However building with the suggested solution results in another error:

cargo build --target x86_64-unknown-linux-musl

results in a giant list of errors such as

use of undeclared crate or module `mem`

Probably this is since memory management is not included in that target?
Anyways I feel that there should be a dead simple approach describing how to release rust applications to docker with less than ten lines of code, :slight_smile: similar to this:

You should install targets that you want to use:

rustup target add x86_64-unknown-linux-musl

mem is not about "memory management", if what you mean by that is dynamic allocation. That'd be alloc.

This isn't related to MUSL, or Docker, or Rust version. It seems to be just a regular bug in the code you're trying to compile. You can get this error if you write something like this:

#[cfg(windows)]
use std::mem;

fn foo() {
    let s = mem::size_of::<i32>();
}

and then compile it on a platform that doesn't match the cfg.

Rust can make all of std optional, but that would give you a very different error message. There is no Rust configuration in which just the mem module would disappear, so the error message is specific to the code you're compiling.

1 Like

This sounds strange since for x86 it compiles fine...

Thank you, I forgot that!
But now it fails like this:

RUST_BACKTRACE=1 SQLX_OFFLINE=true cargo build --target x86_64-unknown-linux-musl
   Compiling regex v1.5.5
   Compiling libc v0.2.126
   Compiling futures-core v0.3.21
   Compiling ring v0.16.20
   Compiling typenum v1.15.0
   Compiling indexmap v1.8.1
   Compiling num-traits v0.2.15
   Compiling unicase v2.6.0
   Compiling num-integer v0.1.45
   Compiling erasable v1.2.1
   Compiling futures-task v0.3.21
   Compiling openssl-sys v0.9.73
   Compiling unicode-normalization v0.1.19
   Compiling http-body v0.4.4
   Compiling httparse v1.7.1
   Compiling crossbeam-utils v0.8.8
error: failed to run custom build command for `ring v0.16.20`

Caused by:
  process didn't exit successfully: `/home/olep/Documents/software/Huddle/notifier/target/debug/build/ring-f696b55488b2d439/build-script-build` (exit status: 101)
  --- stdout
  OPT_LEVEL = Some("0")
  TARGET = Some("x86_64-unknown-linux-musl")
  HOST = Some("x86_64-unknown-linux-gnu")
  CC_x86_64-unknown-linux-musl = None
  CC_x86_64_unknown_linux_musl = None
  TARGET_CC = None
  CC = None
  CROSS_COMPILE = None
  CFLAGS_x86_64-unknown-linux-musl = None
  CFLAGS_x86_64_unknown_linux_musl = None
  TARGET_CFLAGS = None
  CFLAGS = None
  CRATE_CC_NO_DEFAULTS = None
  DEBUG = Some("true")
  CARGO_CFG_TARGET_FEATURE = Some("fxsr,sse,sse2")

  --- stderr
  running "musl-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-U_FORTIFY_SOURCE" "-DNDEBUG" "-c" "-o/home/olep/Documents/software/Huddle/notifier/target/x86_64-unknown-linux-musl/debug/build/ring-3cd0dd4557c022ec/out/aesni-x86_64-elf.o" "/home/olep/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesni-x86_64-elf.S"
  thread 'main' panicked at 'failed to execute ["musl-gcc" "-O0" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-I" "include" "-Wall" "-Wextra" "-pedantic" "-pedantic-errors" "-Wall" "-Wextra" "-Wcast-align" "-Wcast-qual" "-Wconversion" "-Wenum-compare" "-Wfloat-equal" "-Wformat=2" "-Winline" "-Winvalid-pch" "-Wmissing-field-initializers" "-Wmissing-include-dirs" "-Wredundant-decls" "-Wshadow" "-Wsign-compare" "-Wsign-conversion" "-Wundef" "-Wuninitialized" "-Wwrite-strings" "-fno-strict-aliasing" "-fvisibility=hidden" "-fstack-protector" "-g3" "-U_FORTIFY_SOURCE" "-DNDEBUG" "-c" "-o/home/olep/Documents/software/Huddle/notifier/target/x86_64-unknown-linux-musl/debug/build/ring-3cd0dd4557c022ec/out/aesni-x86_64-elf.o" "/home/olep/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/pregenerated/aesni-x86_64-elf.S"]: No such file or directory (os error 2)', /home/olep/.cargo/registry/src/github.com-1ecc6299db9ec823/ring-0.16.20/build.rs:653:9
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/std/src/panicking.rs:584:5
     1: core::panicking::panic_fmt
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/panicking.rs:143:14
     2: build_script_build::run_command::{{closure}}
               at ./build.rs:653:9
     3: core::result::Result<T,E>::unwrap_or_else
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/result.rs:1428:23
     4: build_script_build::run_command
               at ./build.rs:652:18
     5: build_script_build::compile
               at ./build.rs:512:13
     6: build_script_build::build_library::{{closure}}
               at ./build.rs:447:18
     7: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:280:13
     8: core::option::Option<T>::map
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/option.rs:906:29
     9: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::next
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/iter/adapters/map.rs:103:9
    10: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter_nested::SpecFromIterNested<T,I>>::from_iter
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/vec/spec_from_iter_nested.rs:26:32
    11: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/vec/spec_from_iter.rs:33:9
    12: <alloc::vec::Vec<T> as core::iter::traits::collect::FromIterator<T>>::from_iter
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/alloc/src/vec/mod.rs:2552:9
    13: core::iter::traits::iterator::Iterator::collect
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/iter/traits/iterator.rs:1778:9
    14: build_script_build::build_library
               at ./build.rs:443:16
    15: build_script_build::build_c_code::{{closure}}
               at ./build.rs:416:9
    16: <core::slice::iter::Iter<T> as core::iter::traits::iterator::Iterator>::for_each
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/slice/iter/macros.rs:211:21
    17: build_script_build::build_c_code
               at ./build.rs:415:5
    18: build_script_build::ring_build_rs_main
               at ./build.rs:279:5
    19: build_script_build::main
               at ./build.rs:240:13
    20: core::ops::function::FnOnce::call_once
               at /rustc/7737e0b5c4103216d6fd8cf941b7ab9bdbaace7c/library/core/src/ops/function.rs:227:5
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...

Update:
installing openssl using homebrew and setting

export OPENSSL_DIR="/home/linuxbrew/.linuxbrew/opt/openssl@3"

lets me compile the app successfully. However during that journey for my feeling I still had to do much too many hacky steps with really unclear error messages, not at all feeling like a first-class cloud-native experience which I believe Rust deserves to deliver!
Maybe I will create a GitHub pages entry at least describing the issues I ran into, but it would be nice to improve the error messages at the following points, too:

  1. Running the binary compiled for the "normal" linux target in alpine linux without the correct dependencies installed should give a more useful message than exec /my_binary: no such file or directory
  2. Trying to compile to the x86_64-unknown-linux-musl-target without adding it in rustup should instantly terminate saying the target is not added instead of leaving a giant list of errors
  3. Compiling without the correct OpenSSL-Libraries linked should clearly state which environment variable needs to be set (export OPENSSL_DIR="/home/linuxbrew/.linuxbrew/opt/openssl@3") in order to get ist working. Here it confuses me that the include and the lib dir which either have to be set both or the parent directory of those needs to be set.

Maybe it's my lack of experience with Rust, but I really feel that dockerizing a Rust application should be as simple as with Go.

That doesn't depend on Rust at all. That's how the OS/shell/dynamic loader handles the error. Rust can't do basically anything about that.

This is also something that doesn't depend on the language in any way. At best, it could be documented in the crate that depends on OpenSSL, but this is not a valid criticism about the language or the official tooling.

Now knowing that many factors of cross-compilation and FFI interop simply aren't fixable at the language level, how do you propose Rust should fix these issues? And what does this have to do with the cloud?

Alpine Linux is too huge. Why do you bundle entire linux distribution if all you need is the single statically linked binary? Google's distroless repository can be a good starting point, not just for Rust but for other languages.

2 Likes

Well, to be honest you make me feel pretty not-so-well-informed about how the language works and I am sorry for the criticism without knowing the technical details.

Nontheless I still believe we need to do something to smoothen the experience for new users wishing to containerize their app.

In my job the virtually only way we use to deploy apps to the cloud is by building a docker image from the app and I would not underestimate how important it is that this workflow just works.

Not knowing enough about the technical details the only way I can come up with is to clearly document the steps that you need to take on a well-reachable website.

And this still remains, right?

1 Like

It certainly is important. However, the very point of containerization is that you basically ship the system where the software was verified to build and work correctly. I would say that for this exact reason, one should not attempt to cross-compile for a foreign Docker image/container. One should build in and for the image one plans to run the application in.

It is reasonable that you don't want all the Rust build tools in an image, but you could just perform a multi-stage build and leave only the build artifacts in the last (deployed) stage.

It would be nice, although I'm not sure how a target not being installed can be detected reliably, in a robust, and most importantly, non-misleading manner. Saying that "if there is no unknown-linux-musl-gcc, then the target is not installed" would be naïve, and telling the user to install the target in this case even though it might already be installed would simply be misleading and actively counter-productive.

I'd recommend starting with a simpler example.

Compile a simple hello world for the correct target and build your container using a scratch dockerfile in the directory your 'hello' binary is located:

FROM scratch
ADD hello /
CMD ["/hello"]

Once you get that working, move on to more complicated applications.

Walk before you run.

2 Likes

Wow this actually worked right out of the box, just as I imagined it! In the docs they say it directly ships with libc so this is why it works and alpine does not. So as far as I can see it the standard way that should be advertised for dockerizing a rust app should be via this image, right?

By not thinking the developer experience ends at some "language proper" level and the rest is not Rust's problem. Rust has fantastic diagnostics for rustc's compilation phase, but stops caring at all about what happens from the linking phase onwards. I know for rustc implementors that's a meaningful barrier, but for end users that's an arbitrary cliff.

Zig didn't say "it's not our problem". They've made it their problem. All of that mess. They went all the way and shipped their own libc, system headers, linker, and clang. Instead of throwing hands in the air "welp, that's an OS issue", they've done it the hard way and did the ugly tedious OS-specific parts too.

Golang also avoids this whole problem, but in their case by leaning very heavily on NIH. Nevertheless, they ship all the necessary bits, and avoid all the problematic ones, so the end result for developers is that cross-compilation just works.

Rust's cross-compilation is only better than C's, but this is a very low bar.

2 Likes

I'm hopeful that someday we'll get "pure Rust" targets for the major platforms, which would help a lot with cross-compilation.

1 Like

Other reasons for using distroless (instead scratch/alpine) is that it comes with certificates and time zone data. It's about the bare minimal distro for anything HTTP related.

I have to say, that is not a reasonable approach.

Why is it not?

Rust already deals with a lot of OS-specific ugly problems (e.g. in libstd abstracts over many poor system-specific APIs, has workarounds for things like buggy timers, deals with quirks of Windows filesystems, vcvarsall.bat, and its runtime situation, implemented locks on Linux "the hard way"). Drawing the line at the cross-linker seems arbitrary, especially that Rust already puts some effort to abstract away differences between MSVC and gcc-style linkers, and is trying to ship lld.

Many things that Rust already does used to be considered unreasonable or a necessary evil in C. For example Rust made building on Windows its problem, where in C it's considered each project's problem, or even Microsoft's problem. The cc crate is also a collection of platform-specific behaviors and workarounds for getting compilers working, which "traditionally" were end user's responsibility.

So it seems arbitrary and somewhat unfinished that Rust can tolerate hackiness of digging around Windows registry to make MSVC linker just work without specifying a path to VS installation, but won't have a peek at Debian's sysroots to make cross-compilation work without Cargo config changes.

1 Like

Here is a template with fast and slim Dockerfile based Rust app container, here is a post about perfect Rust container images using Nix