Sys crate C source files from a different directory

[sidcha]

I'm tying to add rust support for a C library with two notable deviations form common practice:

• there is no -sys crate for the bindgen wrappers (but a module in the same crate)
• the C sources are present one level above the rust crate root

So the simplified project structure is as below:

.
├── rust
│   ├── Cargo.toml
│   ├── build.rs
│   └── src
│       ├── lib.rs              <---- safe wrappers
│       ├── ...
│       └── osdp_sys
│           └── mod.rs          <---- bindgen generated wrappers 
├── include
│   ├── foo.h.                  <---- public API
│   └── ...
└── src
    ├── foo.c                   <----- C soruces
    ├── ...

In build.rs, I include the C source files (using git to identify the project root) and cargo build works. When I try to publish the crate to crates.io, that also seems to work but then when I try to install the crate from crates.io version, crate build fails because it cannot find any of the files in src/ directory.

I've also tried to add include key in [package] to specify those paths without much luck. So the question boils down to: how do I package source files outside of CARGO_MANIFEST_DIR at package time? Thanks for any help.

[tbfleming]

If you're using CMake, makefiles, or something similar, you could add a rule which copies the C sources to a subtree of rust/

[sidcha]

I use cmake and make for the parent directory, so I can add a rule to copy all source files to rust/vendor/ and ignore it in git. Is that what you meant? Will this rule be invoked from build.rs? Also as I understood, cargo would respect gitignore and not publish rust/vendor/.

[tbfleming]

Yes. Since cargo packages need the C sources, you may have to temporarily remove them from .gitignore during publish.

[sidcha]

I tried a variation of this method by copying source files into rust/vendor/ from build.rs, gitignore-ing it, and adding an include in Cargo.toml to add the dir to package but now I get this warning when I run cargo package:

Source directory was modified by build.rs during cargo publish. Build scripts should not modify anything outside of OUT_DIR.

To proceed despite this, pass the `--no-verify` flag.

This is beginning to look really ugly I'd expect there to be a better way to do this.

[tbfleming]

Cargo and supporting tooling supports one of two cases for linking a C library:

• It already exists on each user's system, or
• The sources are within the package's subtree

[sidcha]

IMHO, requirement of being within the subtree of the package is too restrictive. I do have a Cargo.toml in the parent dir marking it a cargo workspace; maybe it should be possible to include non-package files from the same workspace?

[tbfleming]

The published package is a tarball (minus excluded files) of the package (not workspace) subtree plus some additional files.

Maybe you could create a new tool which generates it and posts it to crates.io?

[kpreid]

@sidcha

I tried a variation of this method by copying source files into rust/vendor/ from build.rs,

This is too late. build.rs runs on every compilation including the ones done by users who downloaded the package. Your file copying would have to be done as a step before building or publishing, not in the build script.

There's a trick that might be useful, though: put a symlink to the code directory in your package. Cargo will automatically replace the symlink with a copy of the directory when packaging. Then you don't need any extra build steps at all.

[sidcha]

The symlink method does not work. My build.rs complains about missing files when I run cargo package.

[kornel]

This is not supported.

Crates published to crates.io are not supposed to peek into parent directories. Crates.io does not support publishing of workspaces — every published crate is removed from its workspace and converted to a standalone crate.

Make a sys crate for it that bundles the source code inside of its directory tree.

Rust/Cargo strongly prefers its conventions over configuration and flexibility. You don’t have freedom to structure projects they way you like them. Structure them the way Cargo likes them.

[sidcha]

@kpreid I found why this symbolic link method did not work. The link's target directory had a Cargo.toml (defining it as a workspace) so cargo package did not include it. After removing the Cargo.toml file, it works. Thank you.