I recently switched from cargo run <mybin> to cargo install --path path --root /destination the binary and running it after "installing" the binary to a destination.
Now in CI I sometimes get a race during the cargo install step which causes the build to fail.
Compiling proc-macro2 v1.0.51
Compiling unicode-ident v1.0.8
Compiling quote v1.0.23
info: syncing channel updates for '1.54.0-x86_64-apple-darwin'
info: latest update on 2021-07-29, rust version 1.54.0 (a178d0322 2021-07-26)
info: downloading component 'rust-src'
info: installing component 'rust-src'
Compiling syn v1.0.109
Compiling serde_derive v1.0.152
Compiling serde v1.0.152
Compiling semver v1.0.16
Compiling thiserror v1.0.39
info: syncing channel updates for '1.54.0-x86_64-apple-darwin'
info: latest update on 2021-07-29, rust version 1.54.0 (a178d0322 2021-07-26)
info: downloading component 'rust-src'
info: installing component 'rust-src'
Compiling camino v1.1.3
Compiling serde_json v1.0.94
Compiling itoa v1.0.6
Compiling unicode-width v0.1.10
info: rolling back changes
error: failed to install component: 'rust-src', detected conflict: 'lib/rustlib/src/rust/Cargo.lock'
error: could not compile `thiserror`
It seems that in CI, sometimes something triggers rust-src to be installed twice, concurrently, which causes the install command to fail. I'm not really sure why rust-src is even being installed during the build, and am curious why this did not happen with my previous cargo run approach.
The error message mentions thiserror, but thiserror doesn't seem to try to install rust-src in it's build-script.
It has nothing to do with the build script. thiserror has a rust-toolchain.toml file, which specifies rust-src as one of the required components. This means that if you try to compile thiserror, cargo will automatically call rustup to install the required components.
However, AFAIK it should only apply to the currently built crate, not its dependencies (because it's the leaf crate which determines which targets should be built, for what platform, and what rustc components are required for development).
Looking at your repository, you're writing a CMake generator for rustc. I couldn't understand in a reasonable timeframe how exactly you deal with makefile generation and dependencies, but I have a hunch that you're trying to install each dependency individually, rather than offloading them all to cargo. If that's the case, it may be that your individual cargo install commands for each dependency pick up the toolchain details from rust-toolchain.toml and rust-toolchain files of your dependencies.
The race condition likely happens because you try to build all dependencies in parallel. I would expect cargo to lock the relevant resources anyway, but perhaps the devs forgot to do it (why would anyone install toolchains in parallel via rustup anyway?), or perhaps directory locking doesn't work on MacOS.
Ahh, so it's probably multiple of those in my dependencies getting picked up by mistake.
If that's the case, it may be that your individual cargo install commands for each dependency pick up the toolchain details from rust-toolchain.toml and rust-toolchain files of your dependencies.
No, there is only a single cargo install in the project, and it is guaranteed to be sequential. Some background: The cargo install of my CMake generator happens at configure time. The CMake generator internally uses cargo metadata to parse a Cargo.toml manifest and emit CMake code which describes the packages in the Manifest and creates CMake targets for them (essentially writing Makefiles). No other cargo build / cargo install command is even run at configure time.
My next best guess now is that there are actually two dependencies of mine that require rust-src in their toolchain.toml, and somehow cargo install invokes rustup in such a way that these toolchain files are picked up by rustup.
To fix my CI I guess I'll just preinstall rust-src (even though I shouldn't need it). If I have some time I'll also test if this still happens with more recent versions of cargo.
Edit: Interestingly enough cargo install still installs rust-src on nightly. Not sure about the race condition - it's rare enough that I would have to rerun the jobs quite often.
I did some digging and figured out what is happening:
A dependency has a rust-toolchain.toml file.
The same dependency has a buildscript that executes something like rustc --version. If the rustc is a rustup proxy it will respect the rustup-toolchain.toml defined in that crate and start installing. Setting CARGO_BUILD_RUSTC=/path/to/.rustup/.../rustc before invoking cargo prevents installing rust-src.
If you have multiple dependencies fulfilling 1. and 2. at the same time you can get a race since Cargo will compile crates concurrently.
Probably a fix would be to add the rust-toolchain.toml files to be excluded from publishing on crates.io for libraries.