Mac M1 release build fails for libc and/or proc2_macro

I've been facing this issue for over a month where the release build for my project is failing for libc and/or proc2_macro. It's unclear which one because the build is being killed with a signal. I'm on a Mac M1 system. The build works for my Linux system.

active toolchain
----------------

stable-aarch64-apple-darwin
rustc 1.69.0 (84c898d65 2023-04-16)
Homebrew clang version 16.0.2
Target: arm64-apple-darwin22.4.0
Thread model: posix
InstalledDir: /opt/homebrew/opt/llvm/bin

I have XCode command line tools installed.

pkgutil --pkg-info=com.apple.pkg.CLTools_Executables | grep version
version: 14.3.0.0.1.1679647830

I've successfully built and cloned these crates separately. But the build for my project fails. Log for cargo build -vv --release -j 1 attached below.

   Compiling libc v0.2.144
     Running `CARGO=/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/Users/twitu/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.144 CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_DESCRIPTION='Raw FFI bindings to platform libraries like libc.
' CARGO_PKG_HOMEPAGE='https://github.com/rust-lang/libc' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=libc CARGO_PKG_REPOSITORY='https://github.com/rust-lang/libc' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.2.144 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=144 CARGO_PKG_VERSION_PRE='' DYLD_FALLBACK_LIBRARY_PATH='/Users/twitu/Code/nautilus_trader/nautilus_core/target/release/deps:/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/lib:/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/lib:/Users/twitu/lib:/usr/local/lib:/usr/lib' rustc --crate-name build_script_build /Users/twitu/.cargo/registry/src/github.com-1ecc6299db9ec823/libc-0.2.144/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=83 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="extra_traits"' --cfg 'feature="std"' -C metadata=699e1cda262548ee -C extra-filename=-699e1cda262548ee --out-dir /Users/twitu/Code/nautilus_trader/nautilus_core/target/release/build/libc-699e1cda262548ee -C strip=symbols -L dependency=/Users/twitu/Code/nautilus_trader/nautilus_core/target/release/deps --cap-lints warn -C link-arg=-undefined -C link-arg=dynamic_lookup '-Aclippy::drop_non_drop'`
   Compiling autocfg v1.1.0
     Running `CARGO=/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/bin/cargo CARGO_CRATE_NAME=autocfg CARGO_MANIFEST_DIR=/Users/twitu/.cargo/registry/src/github.com-1ecc6299db9ec823/autocfg-1.1.0 CARGO_PKG_AUTHORS='Josh Stone <cuviper@gmail.com>' CARGO_PKG_DESCRIPTION='Automatic cfg for Rust compiler features' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='Apache-2.0 OR MIT' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=autocfg CARGO_PKG_REPOSITORY='https://github.com/cuviper/autocfg' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=1.1.0 CARGO_PKG_VERSION_MAJOR=1 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' DYLD_FALLBACK_LIBRARY_PATH='/Users/twitu/Code/nautilus_trader/nautilus_core/target/release/deps:/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/lib:/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/lib:/Users/twitu/lib:/usr/local/lib:/usr/lib' rustc --crate-name autocfg /Users/twitu/.cargo/registry/src/github.com-1ecc6299db9ec823/autocfg-1.1.0/src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=83 --crate-type lib --emit=dep-info,metadata,link -C embed-bitcode=no -C debug-assertions=off -C metadata=9f3f8b7265b4cb5f -C extra-filename=-9f3f8b7265b4cb5f --out-dir /Users/twitu/Code/nautilus_trader/nautilus_core/target/release/deps -C strip=symbols -L dependency=/Users/twitu/Code/nautilus_trader/nautilus_core/target/release/deps --cap-lints warn -C link-arg=-undefined -C link-arg=dynamic_lookup '-Aclippy::drop_non_drop'`
   Compiling proc-macro2 v1.0.56
     Running `CARGO=/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/Users/twitu/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-1.0.56 CARGO_PKG_AUTHORS='David Tolnay <dtolnay@gmail.com>:Alex Crichton <alex@alexcrichton.com>' CARGO_PKG_DESCRIPTION='A substitute implementation of the compiler'\''s `proc_macro` API to decouple token-based libraries from the procedural macro use case.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=proc-macro2 CARGO_PKG_REPOSITORY='https://github.com/dtolnay/proc-macro2' CARGO_PKG_RUST_VERSION=1.31 CARGO_PKG_VERSION=1.0.56 CARGO_PKG_VERSION_MAJOR=1 CARGO_PKG_VERSION_MINOR=0 CARGO_PKG_VERSION_PATCH=56 CARGO_PKG_VERSION_PRE='' DYLD_FALLBACK_LIBRARY_PATH='/Users/twitu/Code/nautilus_trader/nautilus_core/target/release/deps:/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/lib:/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/lib:/Users/twitu/lib:/usr/local/lib:/usr/lib' rustc --crate-name build_script_build --edition=2018 /Users/twitu/.cargo/registry/src/github.com-1ecc6299db9ec823/proc-macro2-1.0.56/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=83 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="proc-macro"' --cfg 'feature="span-locations"' -C metadata=4bcfedfaa435c0ed -C extra-filename=-4bcfedfaa435c0ed --out-dir /Users/twitu/Code/nautilus_trader/nautilus_core/target/release/build/proc-macro2-4bcfedfaa435c0ed -C strip=symbols -L dependency=/Users/twitu/Code/nautilus_trader/nautilus_core/target/release/deps --cap-lints warn -C link-arg=-undefined -C link-arg=dynamic_lookup '-Aclippy::drop_non_drop'`
     Running `/Users/twitu/Code/nautilus_trader/nautilus_core/target/release/build/proc-macro2-4bcfedfaa435c0ed/build-script-build`
error: failed to run custom build command for `proc-macro2 v1.0.56`

Caused by:
  process didn't exit successfully: `/Users/twitu/Code/nautilus_trader/nautilus_core/target/release/build/proc-macro2-4bcfedfaa435c0ed/build-script-build` (signal: 9, SIGKILL: kill)

I've asked about this issue on SO but did not find any answers.

Do you have a working C toolchain? Usually, on a Mac, you'll need the "Xcode Command Line Tools" (which have exactly nothing to do with Xcode, they are the standard LLVM+Clang toolchain).

I believe I don't need XCode itself just the CommandLine Tools themselves. They are installed.

pkgutil --pkg-info=com.apple.pkg.CLTools_Executables | grep version
version: 14.3.0.0.1.1679647830

LLVM and clang themselves have been installed via brew.

Have you migrated your macOS installation from an Intel Mac? Does file $(which rustup) show the right architecture?

1 Like

I have the architecture set correctly.

active toolchain
----------------

stable-aarch64-apple-darwin
rustc 1.69.0 (84c898d65 2023-04-16)

This is different than the toolchain. You could have x86-64 rustup version emulated under rosetta which is set to use stable-aarch64-apple-darwin toolchain.

1 Like
file $(which rustup)
/Users/twitu/.cargo/bin/rustup: Mach-O 64-bit executable arm64

It appears to be the correct architecture for me.

2 Likes

Hnmmmmm, that's a weird behavior. If such kill happened on Linux, it'd be very likely the OOM killer, but AFAIK macOS never does that. I presume you don't have any silly overeager antivirus either.

Maybe try running the build using a different shell?

Or the standard "have you tried turning it off an on again": rustup self uninstall and install it again.

I tried both it gives the same result. The weird part is that other projects build without failing. It's only this project. One main difference :thinking: perhaps is the crate-type. This failing project has the below config while for other projects it's the default which is rlib I believe.

crate-type = ["rlib", "staticlib", "cdylib" 

I removed these crate types and built release for the default rlib but it still failed. It's very weird. I'm failing to understand how a project can cause failure in building dependencies that too fundamental ones like libc and proc-macro2.

what happens if you run the build script binary manually, does it crash? can you use gdb to get a stack trace? the build script from proc-macro2 looks like there's nothing should cause a crash to my eye

So when I clone the proc_macro2 repo and do a cargo build --release it completes successfully. I even tried adding crate-type = ["rlib", "staticlib", "cdylib"] and to lib section and it still succeeded.

Next I tried doing the same for libc and here it failed for both staticlib and cdylib. It fails even for debug build.

   Compiling libc v0.2.144 (/Users/twitu/Code/libc)
error: `#[panic_handler]` function required, but not found

error: language item required, but not found: `eh_personality`
  |
  = note: this can occur when a binary crate with `#![no_std]` is compiled for a target where `eh_personality` is defined in the standard library
  = help: you may be able to compile for a target that doesn't need `eh_personality`, specify a target with `--target` or in `.cargo/config`

error: could not compile `libc` due to 2 previous errors

eh_personality :thinking: :neutral_face:

I understand what these crate types are at a high level and I need them for my project. But I'm not sure why they behave interact weirdly with libc crate.

Detailed log for cargo build -vv below.

Compiling libc v0.2.144 (/Users/twitu/Code/libc)
     Running `CARGO=/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/bin/cargo CARGO_CRATE_NAME=libc CARGO_MANIFEST_DIR=/Users/twitu/Code/libc CARGO_PKG_AUTHORS='The Rust Project Developers' CARGO_PKG_DESCRIPTION='Raw FFI bindings to platform libraries like libc.
' CARGO_PKG_HOMEPAGE='https://github.com/rust-lang/libc' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=libc CARGO_PKG_REPOSITORY='https://github.com/rust-lang/libc' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.2.144 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=2 CARGO_PKG_VERSION_PATCH=144 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 DYLD_FALLBACK_LIBRARY_PATH='/Users/twitu/Code/libc/target/debug/deps:/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/lib:/Users/twitu/.rustup/toolchains/stable-aarch64-apple-darwin/lib:/Users/twitu/lib:/usr/local/lib:/usr/lib' OUT_DIR=/Users/twitu/Code/libc/target/debug/build/libc-0cf5d1ecce4d6701/out rustc --crate-name libc src/lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=150 --crate-type staticlib --emit=dep-info,link -C embed-bitcode=no -C split-debuginfo=unpacked -C debuginfo=2 --cfg 'feature="default"' --cfg 'feature="std"' -C metadata=9ff6a6a69c719ecb -C extra-filename=-9ff6a6a69c719ecb --out-dir /Users/twitu/Code/libc/target/debug/deps -C incremental=/Users/twitu/Code/libc/target/debug/incremental -L dependency=/Users/twitu/Code/libc/target/debug/deps --cfg freebsd11 --cfg libc_priv_mod_use --cfg libc_union --cfg libc_const_size_of --cfg libc_align --cfg libc_int128 --cfg libc_core_cvoid --cfg libc_packedN --cfg libc_cfg_target_vendor --cfg libc_non_exhaustive --cfg libc_long_array --cfg libc_ptr_addr_of --cfg libc_underscore_const_names --cfg libc_const_extern_fn`
error: `#[panic_handler]` function required, but not found

error: language item required, but not found: `eh_personality`
  |
  = note: this can occur when a binary crate with `#![no_std]` is compiled for a target where `eh_personality` is defined in the standard library
  = help: you may be able to compile for a target that doesn't need `eh_personality`, specify a target with `--target` or in `.cargo/config`

error: could not compile `libc` due to 2 previous errors

that's a different kind of error, and it's not related to your original problem. your original problem is that the build script of the proc-macro2 crate crashed. the libc error is irrelevant, libc is special, and the error simply means you are building it in a unsupported manner.

don't just blindly try everything and hope the original problem will be fixed. you need to a systematic method to diagnostic the problem. as shown by cargo, the original problem is caused by the build script of proc-macro2 crate, so what you should do is to figure out how and why the build script crashed in the first place.

also, cargo set the build environment differently depending on whether a crate is built as a dependency or is built as root crate. so build the proc-macro2 separately might not reproduce the original crash. what you should do instead, is add a [patch] section in your project and override the proc-macro2 dependency with a local path.

reading the code of build.rs from proc-macro2, it mostly just check some conditions and println!() cargo configurations. besides the println!() calls, it only calls std for two things: retrieve environment variables, and spawn a rustc subprocess to do version detection.

but environment variables (I believe) are all user space and don't need system calls at all, so I can only imagine it's the operation of spawning a subprocess which triggered some system guard mechanism.

but that's just my speculation, I don't have an apple system to test out. but you can take it as a hint and do some digging yourself. for instance, what happens if you change this line to hard coded full path to rustc? will it still crash? what if you change it to a non-existing path, will it crash?

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.