Publishing crate that has static C-library as git submodule

I am trying to publish a *-sys crate for a C-library that is usually statically linked. The crate uses bindgen to generate the Rust bindings. My idea was to fetch the library as git submodule and have build instructions for it in build.rs This works nicely, I can even fetch my *-sys crate as cargo dependency using the git dependency specification.

The repo is automatically cloned, the submodule for the C-library is fetched, the C-library is build and so on, everything fine.

Until I try to cargo publish the crate. Then the package verification fails, because the build added the submodule to the source file.

How is this supposed to work?

Adding it as submodule should work. For example openssl-src does this: https://github.com/alexcrichton/openssl-src-rs What is the exact verification error?

Ah, sorry I have misread the error message.

So here is what happened. The C-lib uses the waf tool (python) as building tool. This leads to __pycache__ folders in the source tree of the library . The verification then complained, that these __pycache__ folders are not supposed to be there. Then I put a git clean -fxd to build.rs at the end of the build process. Unfortunately this removes the whole C-library as the packager copies it to target/package/<package>-sys-<version>/<submodule>/. There it is no longer managed by git. So git clean -fxd just removed the whole submodule sources.

Then the verification complains that the files have been removed not added. That's what I misread.

So now the question how I can remove these __pycache__ folders in the source tree in a reliable, ideally platform independent way. Probably easiest by hand. It's just three of them.

cargo publish makes a .tar.gz package, so it "forgets" everything about git. Files can come from a git submodule as long as they are already fetched and present in the file system when you publish the crate, but it won't be a git repository any more.

But don't try to use git in the published crate. Your build.rs script should work offline, and it shouldn't be using git at build time. It must not modify your crate's files — crate once published is supposed to be frozen and read-only.

You can copy vendored code to your build scripts OUT_DIR if you have to build code in a way that creates temporary files in the same location.

Cargo.toml has include/exclude keys that can be used to avoid publishing temporary/irrelevant files.

1 Like

If you run python with the -B argument python will not create the __pycache__ dir. You can also set PYTHONDONTWRITEBYTECODE=1.

1 Like

The -B option does the trick. Then there's still a lockfile that waf leaves to state that the build has been configured. That I just delete by hand in build.rs using std::fs::remove_file()

Thank you very much for your help.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.