Jemalloc, pthread_getname_np, and musl

wspeirs · February 5, 2023, 8:01pm

TL;DR

Building my code using the x86_64-unknown-linux-musl target works on one computer, but not on another with the same Cargo.lock, and same versions of rustc. I believe it to be different versions of musl... how can I tell what version of musl is being used?

Full Story

I'm trying to build a static binary w/musl that uses jemalloc for the allocator. I get the following error when I build:

  = note: /usr/bin/ld: target/x86_64-unknown-linux-musl/debug/deps/libjemalloc_sys-0ea9152d514c746d.rlib(prof_sys.pic.o): in function `prof_sys_thread_name_read_impl':
          target/x86_64-unknown-linux-musl/debug/build/jemalloc-sys-05a180e44afaaef0/out/build/src/prof_sys.c:308: undefined reference to `pthread_getname_np'
          collect2: error: ld returned 1 exit status
          
  = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
  = note: use the `-l` flag to specify native libraries to link
  = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#cargorustc-link-libkindname)

The weird thing is that the same code builds on a different machine w/out any issues. As I started to debug why, I noticed the deps seem to be different:

non-working machine

sha1sum ../target/x86_64-unknown-linux-musl/debug/deps/*jemalloc*
51dd24da63e456ff4dc2ba0a5cb1bde45be4f78a  ../target/x86_64-unknown-linux-musl/debug/deps/jemallocator-f23099e6266abcc6.d
b9b8af877654ff8cb8fdcbc5dba3e07ea77b66b2  ../target/x86_64-unknown-linux-musl/debug/deps/jemalloc_sys-0ea9152d514c746d.d
871580a3508083aef522ac6935a0ecd68c96922d  ../target/x86_64-unknown-linux-musl/debug/deps/libjemallocator-f23099e6266abcc6.rlib
383c9622e91fe0280a5b0886f91c35b131bae332  ../target/x86_64-unknown-linux-musl/debug/deps/libjemallocator-f23099e6266abcc6.rmeta
052dc61a820dcaefb7ee37dbf0d67a4d127b605c  ../target/x86_64-unknown-linux-musl/debug/deps/libjemalloc_sys-0ea9152d514c746d.rlib
7e975d0669c965223d097eaf458c18faddc30cb5  ../target/x86_64-unknown-linux-musl/debug/deps/libjemalloc_sys-0ea9152d514c746d.rmeta

working machine

sha1sum ../target/x86_64-unknown-linux-musl/debug/deps/*jemalloc*
48a00695412202e4a2742fe29837ae5095b461c3  ../target/x86_64-unknown-linux-musl/debug/deps/jemallocator-ac4ef0434b9e9d61.d
cece3611044a5ced4ac248cbd71e9680f62c99bf  ../target/x86_64-unknown-linux-musl/debug/deps/jemalloc_sys-60220cfee7b2b8e5.d
83d6a46cadfb8807e0c9e45ef17215adf782a7df  ../target/x86_64-unknown-linux-musl/debug/deps/libjemallocator-ac4ef0434b9e9d61.rlib
699cdd9c9d9d07548215c6ee7f88f6f8432f79a1  ../target/x86_64-unknown-linux-musl/debug/deps/libjemallocator-ac4ef0434b9e9d61.rmeta
e70df2d617fa061921d46546c99109f4714f7150  ../target/x86_64-unknown-linux-musl/debug/deps/libjemalloc_sys-60220cfee7b2b8e5.rlib
8abfc5834746b35287b721839b3224cc4df1b01d  ../target/x86_64-unknown-linux-musl/debug/deps/libjemalloc_sys-60220cfee7b2b8e5.rmeta

I am using pyoxidizer to include pyo3, but all of the pyoxidizer deps are the same on both machines. The Cargo.lock files and of course the version of rustc are the same on both machines as well.

So ultimately I have 3 questions:

What determines the contents of the deps files in target/x86_64-unknown-linux-musl/debug/deps/ different for *jemalloc*?
How can I tell which version of musl is being used when I compile? My issue seems similar to this (pprof 0.9.1: Databend cannot be built on musl targets · Issue #138 · tikv/pprof-rs · GitHub), which they solve by upgrading the version of musl to 1.2.3 used to compile databend.
Is there something else that could be causing these builds to be different besides the version of rustc used, and the versions of crates in Cargo.lock?

bjorn3 · February 5, 2023, 9:07pm

The musl target of rustc ships with musl 1.1.24: rust/musl-toolchain.sh at a67649675014546ce454d65bc8fe3ebd18e6a319 · rust-lang/rust · GitHub

wspeirs · February 6, 2023, 5:02pm

Interesting... looks like pthread_getname_np wasn't added until 1.2.3

Not sure how this is working on my laptop then... has to be something that is pulling in a different version of jemalloc-sys.

So 2 more/same questions:

Where do those deps files come from, if not Cargo.lock?
Are there plans to upgrade the rust toolchain to use a newer/latest version of musl?

cuviper · February 6, 2023, 5:52pm

The musl upgrade is complicated by the time64 change, but it is in progress:

github.com/rust-lang/rust

Update the version of musl used on `*-linux-musl` targets to 1.2.3

rust-lang:master ← wesleywiser:musl_1.2_upgrade

opened 09:04PM - 20 Jan 23 UTC

wesleywiser

+7 -16

Update the version of musl used on our Linux musl targets from 1.1.24 to 1.2.3 a…s proposed in rust-lang/compiler-team#572. musl 1.2.3 is the latest version of musl and supports the same range of Linux kernels as the 1.1 series. As such, it does not affect the minimum supported version of Linux for any of the musl targets. One of the major musl 1.2 features is support for [time64](https://musl.libc.org/time64.html). This support is both source and ABI compatible with programs built against musl 1.1 and so updating the musl version for these targets should not cause Rust programs to fail to run or compile (a [crater run](https://github.com/rust-lang/rust/pull/107129#issuecomment-1407196104) has been completed which demonstrates this for the `i686-unknown-linux-musl` target). Once this change reaches stable, the `libc` crate will then be able to [update their definitions to support 64-bit time](https://github.com/rust-lang/libc/pull/3068), matching the default musl 1.2 APIs exactly.

github.com/rust-lang/libc

Upgrade musl supported version to 1.2.3

rust-lang:master ← wesleywiser:musl-1.2

opened 10:47PM - 12 Jan 23 UTC

wesleywiser

+296 -374

Rebase of #2088. This PR changes the supported version of musl libc to 1.2.3.… The bulk of the changes in this PR are a result of `time_t` changes to support 64-bit time on all platforms (the [time64 change](https://musl.libc.org/time64.html)). This is an API breaking change because of an additional private padding field which is added [on some platforms](https://github.com/rust-lang/libc/commit/e5a7a26272bdfb9aeeade65cd612e48bfceb46b3). This change has been run through crater [several](https://github.com/rust-lang/rust/pull/102891#issuecomment-1317145761) [times](https://github.com/rust-lang/rust/pull/102891#issuecomment-1343528016). The [last run](https://github.com/rust-lang/rust/pull/102891#issuecomment-1351810652) reported 959 regressions, of which upgrading to a newer major or minor version of a dependency would resolve the regression for >700 crates. A breakdown of which targets contain this breaking change and what Target Tier they are is available in the [compiler team MCP to upgrade the `*-linux-musl` targets to musl 1.2](https://github.com/rust-lang/compiler-team/issues/572). Thanks to @kaniini, @richfelker, @danielframpton and @Amanieu for their work on this! Fixes #1848 Closes #2088

system · May 7, 2023, 5:52pm

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.