RUST_LOG, cargo build, and cross compiling

Hello everyone!

[TLDR] Customized cargo+rust+rust-std-static 1.72 build seems to ignore the target architecture, always dropping back to i686. The issue seems to be in cargo build which emits wrong target and linker parameters for all the rustc commands it crafts and runs. The tooling seems to be fine, as running the fixed rustc command by hand is able to compile and link the crate. [/TLDR]

I'm compiling Rust 1.72.1 for Sailfish OS, and I've managed to build rustc and cargo for the i686 tooling, and rust-std-static packages for the target architectures (i686-unknown-linux-gnu (kinda), armv7-unknown-linux-gnueabihf and aarch64-unknown-linux-gnu). I can then install the packages to the build environment (it's a bit of a mess, don't worry about it!) and successfully get to building. We use patched rustc and cargo (0003 and 0006) to use SB2_RUST_TARGET_TRIPLE instead. Those patches used to work fine with 1.52 and presumably with 1.61 too - with 1.72 it seemingly does not.

I am going to check if we could just use HOST_TRIPLE and TARGET_TRIPLE instead and possibly even drop the two patches mentioned - I will post back the results at some point (holidays etc.)

The only target that works is i686 (as it's not really cross-compiling). It produces working, runnable binaries (tested in the i486 device emulator that comes with Sailfish SDK), all is well there. But whenever I try to compile for armv7hl or aarch64, the cargo build command never emits the correct parameters for the plethora of rustc commands, but always defaults/falls back to i686.

The following builds happen in a Docker container, there's one for each Sailfish OS target (Sailfish OS version + architecture combination):

sfdk build for i686 (ok)
+ export HOST_TRIPLE=i686-unknown-linux-gnu
+ export SB2_RUST_TARGET_TRIPLE=i686-unknown-linux-gnu
+ export TARGET_TRIPLE=i686-unknown-linux-gnu
[...]
+ cargo build -j 1 -vv --release --no-default-features --bin harbour-whisperfish --features sailfish --manifest-path /home/matti/SFOS/whisperfish/rpm/../Cargo.toml
   Compiling proc-macro2 v1.0.70
     Running `CARGO=/usr/bin/cargo [...] LD_LIBRARY_PATH='/home/matti/SFOS/whisperfish/target/release/deps:/usr/lib' rustc [...] --target i686-unknown-linux-gnu -L dependency=/home/matti/SFOS/whisperfish/target/release/deps --cap-lints warn`
(the project builds to completion)
sfdk build for aarch64 (not ok)
+ export HOST_TRIPLE=i686-unknown-linux-gnu
+ export SB2_RUST_TARGET_TRIPLE=aarch64-unknown-linux-gnu
+ export TARGET_TRIPLE=aarch64-unknown-linux-gnu
[...]
+ cargo build -j 1 -vv --release --no-default-features --bin harbour-whisperfish --features sailfish --manifest-path /home/matti/SFOS/whisperfish/rpm/../Cargo.toml
   Compiling proc-macro2 v1.0.70
     Running `CARGO=/usr/bin/cargo [...] LD_LIBRARY_PATH='/home/matti/SFOS/whisperfish/target/release/deps:/usr/lib' rustc [...] --target i686-unknown-linux-gnu -C linker=i686-unknown-linux-gnu-gcc -L dependency=/home/matti/SFOS/whisperfish/target/release/deps --cap-lints warn`
error: linking with `i686-unknown-linux-gnu-gcc` failed: exit status: 1

Here's the proof that the build side of things works: If I take the rustc command from the aarch64 block, fix up the target and linker by hand and run the command, the crate (proc_macro) builds (and links?) successfully:

manual rustc for aarch64 (ok!)
$ CARGO=/usr/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/mersdk/.cargo/registry/src/index.crates.io-1cd66030c949c28d/proc-macro2-1.0.70 CARGO_PKG_AUTHORS='David Tolnay <dtolnay@gmail.com>:Alex Crichton <alex@alexcrichton.com>' CARGO_PKG_DESCRIPTION='A substitute implementation of the compiler'\''s `proc_macro` API to decouple token-based libraries from the procedural macro use case.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=proc-macro2 CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/dtolnay/proc-macro2' CARGO_PKG_RUST_VERSION=1.56 CARGO_PKG_VERSION=1.0.70 CARGO_PKG_VERSION_MAJOR=1 CARGO_PKG_VERSION_MINOR=0 CARGO_PKG_VERSION_PATCH=70 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/matti/SFOS/whisperfish/target/release/deps:/usr/lib' rustc --crate-name build_script_build --edition=2021 /home/mersdk/.cargo/registry/src/index.crates.io-1cd66030c949c28d/proc-macro2-1.0.70/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=170 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=e5b9f9b0785b6323 -C extra-filename=-e5b9f9b0785b6323 --out-dir /home/matti/SFOS/whisperfish/target/release/build/proc-macro2-e5b9f9b0785b6323 --target aarch64-unknown-linux-gnu -C linker=aarch64-meego-linux-gnu-gcc -L dependency=/home/matti/SFOS/whisperfish/target/release/deps --cap-lints warn
{"artifact":"/home/matti/SFOS/whisperfish/target/release/build/proc-macro2-e5b9f9b0785b6323/build_script_build-e5b9f9b0785b6323.d","emit":"dep-info"}
{"artifact":"/home/matti/SFOS/whisperfish/target/release/build/proc-macro2-e5b9f9b0785b6323/build_script_build-e5b9f9b0785b6323","emit":"link"}

In the effort of fixing this, I have tried the following:

  • manually adding --target aarch64-unknown-linux-gnu to the linked cargo build command
    • note: changing the target and linker for rustc command manually works however
  • having .cargo/config.toml with [target.aarch64-unknown-linux-gnu] linker = "..."
  • having .cargo/config.toml with [build] target = "aarch64-unknown-linux-gnu"`
  • setting HOST_TRIPLE to i686 and TARGET_TRIPLE to aarch64, disabling SB2_* env var

None of those change the behavior - it still falls back to i686 target and default linker. The only way I can get a reaction out of it is by entering an invalid triplet, say, aarch64-unknown-linux-foobar in which case the cargo build command aborts very early - so it does something at least.

The first question: If I enter any of the three target triple values, it fallbacks to i686. Why is that? Is there a check somewhere that a target is correctly installed perhaps?

The second question: I would like to trace the execution cargo build takes to see where the target triple picked up and/or changed back, but it looks like it doesn't play ball with RUST_LOG=trace. How can I get it working to enable some old-fashioned log::trace or eprintln debugging?

Sorry for the long post! And thanks for any tips in advance! Happy holidays!

Right, dropping the patches (0003, 0004 and 0006) didn't change the behaviour one bit - so I'll have to try to get cargo build work with RUST_LOG to narrow it further.

I'm also half reading between the lines and half assuming, but it looks like SB2_RUST_TARGET_TRIPLE is something that's needed elsewhere, for now I'm going with the assumption that I need to keep it around.

Build scripts, proc macros and their transitive dependencies are always built for the host as they actually run on the host. And if --target is passed to cargo they will ignore RUSTFLAGS.

I thought checked the Whisperfish CI with enough care, but it turns out it also emits the i686 target and linker for proc-macro2. So in that sense, everything seems to be correct.

However, another error remains:

+ cargo build -j 1 -vv --release --no-default-features --bin harbour-whisperfish --features sailfish --manifest-path /home/matti/SFOS/whisperfish/rpm/../Cargo.toml
warning: /home/matti/SFOS/whisperfish/whisperfish/Cargo.toml: dependency (symphonia) specified without providing a local path, Git repository, version, or workspace dependency to use. This will be considered an error in future versions
warning: /home/matti/SFOS/whisperfish/whisperfish/Cargo.toml: unused manifest key: dependencies.symphonia.verion
   Compiling proc-macro2 v1.0.70
     Running `CARGO=/usr/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/mersdk/.cargo/registry/src/index.crates.io-1cd66030c949c28d/proc-macro2-1.0.70 CARGO_PKG_AUTHORS='David Tolnay <dtolnay@gmail.com>:Alex Crichton <alex@alexcrichton.com>' CARGO_PKG_DESCRIPTION='A substitute implementation of the compiler'\''s `proc_macro` API to decouple token-based libraries from the procedural macro use case.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=proc-macro2 CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/dtolnay/proc-macro2' CARGO_PKG_RUST_VERSION=1.56 CARGO_PKG_VERSION=1.0.70 CARGO_PKG_VERSION_MAJOR=1 CARGO_PKG_VERSION_MINOR=0 CARGO_PKG_VERSION_PATCH=70 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/matti/SFOS/whisperfish/target/release/deps:/usr/lib' rustc --crate-name build_script_build --edition=2021 /home/mersdk/.cargo/registry/src/index.crates.io-1cd66030c949c28d/proc-macro2-1.0.70/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=170 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=d432f6ac262028f9 -C extra-filename=-d432f6ac262028f9 --out-dir /home/matti/SFOS/whisperfish/target/release/build/proc-macro2-d432f6ac262028f9 --target i686-unknown-linux-gnu -C linker=i686-unknown-linux-gnu-gcc -L dependency=/home/matti/SFOS/whisperfish/target/release/deps --cap-lints warn`
error: linking with `i686-unknown-linux-gnu-gcc` failed: exit status: 1
  |
  = note: LC_ALL="C" PATH="/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/bin:/opt/cross/bin/:/home/mersdk/.mb2/wrappers/whisperfish:/home/mersdk/.mb2/user-wrappers/whisperfish:/usr/local/bin:/bin:/usr/bin" VSLANG="1033" "i686-unknown-linux-gnu-gcc" "-m32" "/tmp/rustcVlglLx/symbols.o" "/home/matti/SFOS/whisperfish/target/release/build/proc-macro2-d432f6ac262028f9/build_script_build-d432f6ac262028f9.build_script_build.dd50f1435413fcc2-cgu.0.rcgu.o" "/home/matti/SFOS/whisperfish/target/release/build/proc-macro2-d432f6ac262028f9/build_script_build-d432f6ac262028f9.build_script_build.dd50f1435413fcc2-cgu.1.rcgu.o" "/home/matti/SFOS/whisperfish/target/release/build/proc-macro2-d432f6ac262028f9/build_script_build-d432f6ac262028f9.2yuy40uilqifz11z.rcgu.o" "-Wl,--as-needed" "-L" "/home/matti/SFOS/whisperfish/target/release/deps" "-L" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib" "-Wl,-Bstatic" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libstd-c007fe971ec9fa45.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libpanic_unwind-c970a419c5fb2257.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libobject-cdb433d94e361c81.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libmemchr-c38c6fb0b635db28.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libaddr2line-5380f775bf94517b.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libgimli-72c7ecf93e2df1a7.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/librustc_demangle-726d98f6d47fcb8b.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libstd_detect-54227e1d164a9562.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libhashbrown-63b50559e08bc911.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/librustc_std_workspace_alloc-400c39d4913e6b01.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libminiz_oxide-001243b3a0933aac.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libadler-e267b8dbee33a529.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libunwind-f57aa4b841d99932.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libcfg_if-312528a27f51c229.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/liblibc-72c73fa2836e7533.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/liballoc-227f99bd3086a5dc.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/librustc_std_workspace_core-d220d53f92ed4bf8.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libcore-9fb215fa5b50ff91.rlib" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib/libcompiler_builtins-593628d088c3eb8e.rlib" "-Wl,-Bdynamic" "-lgcc_s" "-lutil" "-lrt" "-lpthread" "-lm" "-ldl" "-lc" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-L" "/home/matti/SailfishOS/mersdk/targets/SailfishOS-4.5.0.18-aarch64.default/usr/lib/rustlib/i686-unknown-linux-gnu/lib" "-o" "/home/matti/SFOS/whisperfish/target/release/build/proc-macro2-d432f6ac262028f9/build_script_build-d432f6ac262028f9" "-Wl,--gc-sections" "-pie" "-Wl,-z,relro,-z,now" "-nodefaultlibs"
  = note: gcc: error: /tmp/rustcVlglLx/symbols.o: No such file or directory
          

error: could not compile `proc-macro2` (build script) due to previous error

Caused by:
  process didn't exit successfully: `CARGO=/usr/bin/cargo CARGO_CRATE_NAME=build_script_build CARGO_MANIFEST_DIR=/home/mersdk/.cargo/registry/src/index.crates.io-1cd66030c949c28d/proc-macro2-1.0.70 CARGO_PKG_AUTHORS='David Tolnay <dtolnay@gmail.com>:Alex Crichton <alex@alexcrichton.com>' CARGO_PKG_DESCRIPTION='A substitute implementation of the compiler'\''s `proc_macro` API to decouple token-based libraries from the procedural macro use case.' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='MIT OR Apache-2.0' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=proc-macro2 CARGO_PKG_README=README.md CARGO_PKG_REPOSITORY='https://github.com/dtolnay/proc-macro2' CARGO_PKG_RUST_VERSION=1.56 CARGO_PKG_VERSION=1.0.70 CARGO_PKG_VERSION_MAJOR=1 CARGO_PKG_VERSION_MINOR=0 CARGO_PKG_VERSION_PATCH=70 CARGO_PKG_VERSION_PRE='' LD_LIBRARY_PATH='/home/matti/SFOS/whisperfish/target/release/deps:/usr/lib' rustc --crate-name build_script_build --edition=2021 /home/mersdk/.cargo/registry/src/index.crates.io-1cd66030c949c28d/proc-macro2-1.0.70/build.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --diagnostic-width=170 --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C debug-assertions=off --cfg 'feature="default"' --cfg 'feature="proc-macro"' -C metadata=d432f6ac262028f9 -C extra-filename=-d432f6ac262028f9 --out-dir /home/matti/SFOS/whisperfish/target/release/build/proc-macro2-d432f6ac262028f9 --target i686-unknown-linux-gnu -C linker=i686-unknown-linux-gnu-gcc -L dependency=/home/matti/SFOS/whisperfish/target/release/deps --cap-lints warn` (exit status: 1)
error: Bad exit status from /var/tmp/rpm-tmp.8XSyGH (%build)

That is: note: gcc: error: /tmp/rustcVlglLx/symbols.o: No such file or directory (after supposed linking). This is most likely something to do with the nature of Scratchbox and its way of cross-compiling, so I don't expect a great deal of help from here... I searched the Internet for clues as per why that might be, but found basically nothing. It's like rustc doesn't produce symbols.o file at all - this can be verified with inotifywait which only reports checking for symbols.o under /tmp, never creating such file.

I did some more digging and noticed that symbols.o should be created by add_linked_symbol_object function. AFAICT this was added to Rust 1.62 and would explain why Jolla was able to bump Rust to 1.61 only. I'll try negating adding symbols.o to cmd and will report back how that goes.

If anyone has an idea why that's not created or what's happening there, I'm all ears...

I was first able to bypass the issue by returning early from add_linked_symbol_object in compiler/rustc_codegen_ssa/src/back/link.rs and export_symbols in compiler/rustc_codegen_ssa/src/back/linker.rs, but it would have regressed old issues back into existence, so the search continued.

In the end it turned out to be an issue in Scratchbox -- when I manually set $TMPDIR to the sources folder sub-path, the compilation now Just Works^TM. (I'm not that familiar with Scratchbox innards, so I don't know any details...)

So: It's not a bug in Rust but in Scratchbox, and hitting it can be easily avoided by setting $TMPDIR -- as such, I'm marking this post as the solution :slight_smile: