How can I determine missing dependencies in a cargo build script?

Hello, Rustaceans! I have a problem I need help with. The summary version: I can't get a build script to tell me which file it's missing—any ideas how I could find that out or what headers, etc I need?

The Story

I'm trying to package a Rust program using Nix so that I can use it as a dependency in a larger project (the Rust program is a rewrite of a duct-tape-and-baling-wire set of scripts in bash/sqlite/regex.)

Part of Nix's whole thing is that all dependencies have to be explicitly specified. It enforces this in a variety of ways, like not inheriting the environment of the calling shell, not allowing network access during specific stages, etc.

Normally this doesn't cause problems for me—I run the build script, see what's missing, add it as a dependency, repeat until I get a passing build. (And because of the isolation, if I can get it working locally it'll usually work on whatever other machine I build on. Makes CI way easier.)

Anyway, here we get to my problem: I'm trying to compile this binary, which uses tree_sitter and the cc crate. It's got some Cargo pre-build script which I don't fully understand yet (porting this project is an excuse to try out Rust for me.) It's not working when I isolate the build with nix-build, though! Here's the full build log:

The Full Output of cargo build inside nix-build
[naersk] cargo_version (read): 1.45.1
[naersk] cargo_message_format (set): json-diagnostic-rendered-ansi
[naersk] cargo_release: --release
[naersk] cargo_options: -Z unstable-options
[naersk] cargo_build_options: $cargo_release -j "$NIX_BUILD_CORES" --out-dir out --message-format=$cargo_message_format
[naersk] cargo_test_options: $cargo_release -j "$NIX_BUILD_CORES"
[naersk] RUST_TEST_THREADS: 8
[naersk] cargo_bins_jq_filter: .
[naersk] cargo_build_output_json (created): /private/var/folders/1r/_mn2yckn047g2fp6442389580000gn/T/nix-build-magic-deps-0.1.0.drv-0/tmp.YoBhsVPU6E
[naersk] crate_sources: /nix/store/zr98z1bbybmv8k3vpbw9b05a3xac6v5k-crates-io
[naersk] RUSTFLAGS:
[naersk] CARGO_BUILD_RUSTFLAGS:
[naersk] CARGO_BUILD_RUSTFLAGS (updated): --remap-path-prefix /nix/store/zr98z1bbybmv8k3vpbw9b05a3xac6v5k-crates-io=/sources
building
cargo -Z unstable-options build $cargo_release -j "$NIX_BUILD_CORES" --out-dir out --message-format=$cargo_message_format
   Compiling proc-macro2 v1.0.24
   Compiling unicode-xid v0.2.1
   Compiling syn v1.0.44
   Compiling cc v1.0.61
   Compiling memchr v2.3.3
   Compiling serde_derive v1.0.117
   Compiling ryu v1.0.5
   Compiling serde v1.0.117
   Compiling lazy_static v1.4.0
   Compiling serde_json v1.0.59
   Compiling regex-syntax v0.6.20
   Compiling anyhow v1.0.33
   Compiling same-file v1.0.6
   Compiling itoa v0.4.6
   Compiling thread_local v1.0.1
   Compiling walkdir v2.3.1
   Compiling aho-corasick v0.7.14
   Compiling quote v1.0.7
   Compiling regex v1.4.1
   Compiling tree-sitter v0.17.0
   Compiling magic v0.1.0 (/private/var/folders/1r/_mn2yckn047g2fp6442389580000gn/T/nix-build-magic-deps-0.1.0.drv-0/dummy-src)
error: failed to run custom build command for `tree-sitter v0.17.0`

Caused by:
  process didn't exit successfully: `/private/var/folders/1r/_mn2yckn047g2fp6442389580000gn/T/nix-build-magic-deps-0.1.0.drv-0/dummy-src/target/release/build/tree-sitter-500903a431b9bf22/build-script-build` (exit code: 1)
--- stdout
cargo:rerun-if-env-changed=TREE_SITTER_STATIC_ANALYSIS
cargo:rerun-if-env-changed=DEBUG
cargo:rerun-if-changed=src/error_costs.h
cargo:rerun-if-changed=src/lib.c
cargo:rerun-if-changed=src/subtree.h
cargo:rerun-if-changed=src/reusable_node.h
cargo:rerun-if-changed=src/point.h
cargo:rerun-if-changed=src/unicode
cargo:rerun-if-changed=src/stack.c
cargo:rerun-if-changed=src/tree_cursor.h
cargo:rerun-if-changed=src/language.h
cargo:rerun-if-changed=src/lexer.c
cargo:rerun-if-changed=src/tree.h
cargo:rerun-if-changed=src/get_changed_ranges.c
cargo:rerun-if-changed=src/unicode.h
cargo:rerun-if-changed=src/array.h
cargo:rerun-if-changed=src/query.c
cargo:rerun-if-changed=src/stack.h
cargo:rerun-if-changed=src/subtree.c
cargo:rerun-if-changed=src/length.h
cargo:rerun-if-changed=src/lexer.h
cargo:rerun-if-changed=src/language.c
cargo:rerun-if-changed=src/reduce_action.h
cargo:rerun-if-changed=src/tree_cursor.c
cargo:rerun-if-changed=src/parser.c
cargo:rerun-if-changed=src/clock.h
cargo:rerun-if-changed=src/alloc.h
cargo:rerun-if-changed=src/node.c
cargo:rerun-if-changed=src/get_changed_ranges.h
cargo:rerun-if-changed=src/bits.h
cargo:rerun-if-changed=src/tree.c
cargo:rerun-if-changed=src/atomic.h
TARGET = Some("x86_64-apple-darwin")
OPT_LEVEL = Some("3")
HOST = Some("x86_64-apple-darwin")
CC_x86_64-apple-darwin = None
CC_x86_64_apple_darwin = None
HOST_CC = None
CC = Some("clang")
CFLAGS_x86_64-apple-darwin = None
CFLAGS_x86_64_apple_darwin = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("false")
CC_x86_64-apple-darwin = None
CC_x86_64_apple_darwin = None
HOST_CC = None
CC = Some("clang")
CFLAGS_x86_64-apple-darwin = None
CFLAGS_x86_64_apple_darwin = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
CC_x86_64-apple-darwin = None
CC_x86_64_apple_darwin = None
HOST_CC = None
CC = Some("clang")
CFLAGS_x86_64-apple-darwin = None
CFLAGS_x86_64_apple_darwin = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None

--- stderr


error occurred: No such file or directory (os error 2)



warning: build failed, waiting for other jobs to finish...
error: build failed
[naersk] cargo returned with exit code 101, exiting

The most relevant piece (to my eye) is at the end: error occurred: No such file or directory (os error 2). I've tried all my usual tricks to get this to tell me what file wasn't found, but to no avail. I'm adding clang and gcc to the build environment, so I suspect it's some macOS build header I'm missing.

So finally, the core of my question: how do I get this build script to tell me what dependency it's missing?

I think this is not possible without modifing the build script to print the path of the file it is missing. The build script probably does something like fs::read_to_string("file that doesn't exist") or std::process::Command::new("program which is not in $PATH"), and that would fail with that error.

I'd do one of the following:

  • If I can modify the build.rs, modify it to print the path on failure
  • If I can't, I'd strace the build script while it is running, to spy on the latest syscall it makes before failure.

As a fellow NixOSer, I can attest that it is a royal pain when build.rs needs some native library/tool, but it is not mentioned in the readme. Here's how we deal with it in rust-analyzer: ci.yml :smiley:

2 Likes