Statically compile (musl) a dynamic library (cdylib)

help
1

Hi all,

is a week I am trying to compile a dynamic rust library (cdylib) without dynamically linking any glibc pieces.

This seems an almost impossible beast to tackle, but I am wondering if any of you have ever succeeded.

What I tried so far is something like:

$ cargo build --target=x86_64-unknown-linux-musl               
error: cannot compile `rediSQL v0.5.0 (file:///home/simo/rediSQL)` package, because target `x86_64-unknown-linux-musl` does not support the `cdylib` crate type
# Nope :(
$ RUSTFLAGS="-C target-feature=+crt-static" cargo build --target=x86_64-unknown-linux-musl                                                       
error: cannot compile `rediSQL v0.5.0 (file:///home/simo/rediSQL)` package, because target `x86_64-unknown-linux-musl` does not support the `cdylib` crate type
# Nope :(
$ RUSTFLAGS="-C target-feature=-crt-static" cargo build --target=x86_64-unknown-linux-musl      
    Finished dev [unoptimized + debuginfo] target(s) in 173.60 secs
# Almost
$ ldd target/x86_64-unknown-linux-musl/debug/libredis_sql.so 
        linux-vdso.so.1 =>  (0x00007fff795a5000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007ff61d54d000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff61d183000)
        /lib64/ld-linux-x86-64.so.2 (0x00007ff61dc3e000)
# Nope :(

Do you guys have any alternative to this?

It is a quite complex build and I would prefer to keep it on my own machine (it requires bindgen that needs libclang).

2 Likes
2

It is possible1 to create a freestanding cydlib using musl target, however, it maybe not quite useful as one might expect. rust libstd (musl+cdylib) uses thread local storage (TLS) extensively (even in stdio), even without threads involved explicitly.TLS is implemented in user space in Linux, it's totally depends on libc implementation, so if your application was built with glibc, and the cdylib is a freestanding dynamic shared object (DSO) (statically) linked with musl, then there're two threads implementation, and by no means they're compatible with each other, as a result, if your (freestanding) dso use any TLS (again, this is super common with rust libstd), it would most certainly immediately crash your program.

If you don't use any thread/TLS in the freestanding DSO, it might be fine, then again this is pretty much the same like no_std, with extra complexity, added code size, and less idiomatic.

The catch is it is hard to have multiple different libc implementation live in the same (application's) address space, even with freestanding DSOs, as they're not as isolated as one might have thought. The DSOs have to be loaded by a dynamic linker, part of dynamic linker's responsibly is to setup/initialize TLS for the DSO, and TLS depends on thread (pthread) implementation completely. The dynamic linker also exports symbols (again there's no specification which symbols are exported), __tls_get_addr specifically for TLS, musl and glibc have totally different implementation, as you could have guessed. Even worse, __tls_get_addr from the linker always assume libc is from the exact C code base, it moves data for %fs:<offset>, where <offset> is also implementation dependent, and it is not a simple function (hard to be patched), it accesses thread TCB's in its slow path.

2 Likes