Include_str!() does not work when releasing because of changed pathes?


#1

I just ran into this issue when releasing imag. In one of the bin crates I import a file which lives at / of the repository. When compiling in debug mode (normal cargo build), everything works. When compiling for release (cargo publish) it fails with this error:

   Compiling imag-init v0.6.0 (file:///home/m/archive/development/rust/imag/bin/core/imag-init/target/package/imag-init-0.6.0)
error: couldn't read bin/core/imag-init/target/package/imag-init-0.6.0/src/../../../../imagrc.toml: No such file or directory (os error 2)
  --> bin/core/imag-init/target/package/imag-init-0.6.0/src/main.rs:48:42
   |
48 | const CONFIGURATION_STR : &'static str = include_str!("../../../../imagrc.toml");
   |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: failed to verify package tarball

Caused by:
  Could not compile `imag-init`.

To learn more, run the command again with --verbose.

This is of course because the number of ../ changed from 4 to 7 levels up. I had to change that for the release and revert it afterwards to be able to continue development.

I’m not sure whether to report this. Is it already known? Is there a way to do this right? Propably a compiletime check which path to use?


#2

It sounds like you need a more robust way to do that. If build-time environment variables are not enough, maybe set up a build script to copy the right output files into place.

As used other places, combining include_str with build time environment vars should work:

include_str!(concat!(env!("OUT_DIR"), "/somefile"))

#3

I assume there isn’t a way to #[cfg()] this? Like #[cfg(releasebuild)] and #[cfg(not(releasebuild))]?


#4

I would say don’t use cfgs like that, that’s not robust. The proper way should work for any buildmode.


#5

Okay, I implemented this with a build-script now. But I guess it will fail anyways, won’t it?


#6

Check with “cargo package”.


#7

Jup, failed with No such file or directory.


#8

In your `build.rs, you could find the file with something like that.


#9

When I need to do these sorts of things I’ll normally make the include path relative to my project root (using env!("CARGO_MANIFEST_DIR") and concat!()). That way you’ve got a stable location which everything is relative to and you don’t need to hack around with a build script of #[cfg].

So your include_str!() line could be written something like this:

const CONFIGURATION_STR : &str = include_str!(concat!(env!("CARGO_MANIFEST_DIR"), "../../../imagrc.toml"));

#10

After trying to reproduce the issue locally and understanding the cause of the issue, this what i found:

The stable version of cargo unpack the crates in target/package/ and build from there. This means that using CARGO_MANIFEST_DIR will not solve this problem. This is described here.

For some reason, with the beta and nightly version of cargo, this issue doesn’t appears, I think this a bug related to some recent changes regarding the handling of the workspaces. EDIT: this is not a bug, The latest version of cargo just use workspace-root/target instead of crate/target and this hide the issue because they are now at the same depth.

You use env! in your build.rs. According to the documentation here, this doesn’t work.

Anyway, the file is not included in what is published on crates.io. So if anybody try to build imag-init from crates.io, it doesn’t work. I think that the solution is to include the imagrc.toml file in the root of the imag-init crate via a symlink (like the README.md).


#11

With my latest change it seems to work, although I doubt that building from crates.io will work (because as you just commented, the file is not included in the artifact). When adding include = [ "../../../imagrc.toml" ] to the Cargo.toml of imag-init, the build fails, though:

$ cargo package --manifest-path bin/core/imag-init/Cargo.toml 
   Packaging imag-init v0.6.0 (file:///home/m/archive/development/rust/imag/bin/core/imag-init)
   Verifying imag-init v0.6.0 (file:///home/m/archive/development/rust/imag/bin/core/imag-init)
error: failed to verify package tarball

Caused by:
  failed to read `/home/m/archive/development/rust/imag/bin/core/imag-init/target/package/imag-init-0.6.0/Cargo.toml`

Caused by:
  No such file or directory (os error 2)
 1 │ cargo package --manifest-path bin/core/imag-init/Cargo.toml fg
error: Invalid arguments.

Usage:
    cargo package [options]

#12

It seems you should just arrange it so that the files you need to build are part of the package – add a step that links or copies the file in. Some projects already use symlinks for this, and it seems to work.


#13

Nope, after creating a symlink from the bin/core/imag-init directory with ln -s ../../../imagrc.toml ./imagrc.toml and putting it into includes=[] it does not work either, same error:

error: failed to verify package tarball

Caused by:
  failed to read `/home/m/archive/development/rust/imag/bin/core/imag-init/target/package/imag-init-0.6.0/Cargo.toml`

#14

As a result the released imag-init 0.6.0 does not build :frowning: (using current rustc nightly)

cargo install -f imag-init
error: couldn't read src/../../../../../../../imagrc.toml: No such file or directory (os error 2)
  --> src/main.rs:48:42
   |
48 | const CONFIGURATION_STR : &'static str = include_str!("../../../../../../../imagrc.toml");
   |                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

error: aborting due to previous error

error: failed to compile `imag-init v0.6.0`, intermediate artifacts can be found at `/tmp/cargo-install.q6HpMkN0k414`

#15

If you use includes=[], you must list everything, including the Cargo.toml. It works as a white list.

You should just create the symlink, and modify the include_str! paramter accordingly.


#16

So simply include_str!("../imagrc.toml") and a symlink imagrc.toml next to the Cargo.toml of imag-init?


#17

Yes, I think it should work.


#18

And you should also yank the version that doesn’t work on crates.io.


#19

Ok. Pushed this. cargo build worked and cargo package worked, too.

Will publish imag-init in 0.6.1 shortly with another fix included as well (but not all other crates, as compiling and publishing them all would take hours). Thank you very much for your help!