Linking (of native libs) error occuring only with `cargo test`, build works

I recently encountered strange compilation error, it demonstrates itself only in certain environment, so I'm kind of lost and looking for any ideas how to fix.

Code works fine in local environment (Ubuntu 20.04, rust 1.62), however when I try it in alpine linux 3.14 (same version of stable rust installed via rustup) in docker container I can compile with cargo build, but it fails to compile cargo test with following error:

error: linking with `cc` failed: exit status: 1
  |Weird linking erro
  = note: "cc"  <shortened here>
  = note: /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lavformat
          /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lavutil
          /usr/lib/gcc/x86_64-alpine-linux-musl/10.3.1/../../../../x86_64-alpine-linux-musl/bin/ld: cannot find -lavcodec
          collect2: error: ld returned 1 exit status        

error: could not compile `media_info` due to previous error

All three libraries are there (on pretty standard place /usr/lib) and normal build links just OK (these dynamic libraries are used in normal code).

So I'm kind of confused why cargo test is not compiling and need some hints where to look for source of this problem.

The library code, is here audioserve/crates/media_info at master · izderadicka/audioserve · GitHub

Dockerfile is here (cargo test is now commented) - audioserve/Dockerfile at master · izderadicka/audioserve · GitHub

Thanks

I don’t have any suggestions for what the underlying problem is, but you may be able to get closer to figuring it out by adding -v to cargo build and cargo test, to see how it is running rustc, and setting the environment variable RUSTC_LOG to rustc_codegen_ssa::back::link=info to see how rustc is running the linker.

Do you have the same error building on a Debian container image, for example, rather than Alpine?

On Debian (Ubuntu to be precise) everything works fine.

I have done bit more experiments and found:

If this flag is used export RUSTFLAGS="-C link-self-contained=no" even normal compilation fails on cannot find -lunwind, if opposite is set export RUSTFLAGS="-C link-self-contained=yes then both cargo build and cargo test compiles, but binaries crashes with SegFault. Also binary build without link-self-contained (which should use some heuristic, which library to link) crashes with SegFault.

My assumption is that on Alpine 3.14 new version of Rust (installed via rustup) is not compatible with provided by system native libraries (maybe different version of lib musl?). I'm not sure about exact mechanism - I needed newer compiler because of edition2021 dependency, but on the other hand I need some older versions of native libraries like libavformat, openssl ...
For now I resolved by downgrading some dependencies, so everything compiles fine with rust 1.52, which is provided with Alpine 3.14. But I'm really curious what exactly is going on, cause I'll have to solve it in near future.

If anybody has some input on this, it'll be very welcomed, thanks.

Also target on Alpine packed rust is different - it's x86_64-alpine-linux-musl, while for rust loaded via rustup it's x86_64-unknown-linux-musl - There probably is some reason for different targets?

Doesn't Alpine Linux want to link everything statically?

If the compiler is expecting to do a statically build (i.e. compiling for x86_64-unknown-linux-musl) and there is no avformat.a on the system, only avformat.so, then the linker will complain about missing libraries.

Alpine uses dynamically linked MUSL as its platform libc, which breaks so many assumptions. TBH it's more suited for languages like Java or Go which doesn't care much about libc and system libs.

Linkinking of native libs is dynamic AFAIK. I'm able todo fully static binary with target x86_64-unknown-linux-musl Problem is with dynamic linking and different behaviour of packed rust and rust provided by rustup (different targets)

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.