Integrating a build helper

I'm writing a code generation helper which I plan to run occasionally. It's not really appropriate to stuff this functionality into

  • I need for other things.
  • The code generation helper functionality will be quite large.
  • I plan to commit the autogeerated code.
  • The helper will be run only every once in a while.

I'm wondering where to put it.

(Gory details: I'm writing Rust bindings against a large C API and the helper script will parse header files and autogenerate tests which validate the behavior of the hand-coded bindings.)

The obvious thing to do would be to code up a Python script and stuff it in a dev/ directory. But this is a Rust project and I'd rather just use Rust. It's just that the mechanism isn't really suited for use cases like this.

I think this might be the sort of thing @matklad's cargo-xtask is intended for?

1 Like

Recently, I’ve also been using #[ignore] #[test] for this. Makes it easy to run, guarantees that it doesn’t stop compiling, and requires zero boilerplate.

1 Like

An example program (examples/ would also work like an ignored test. (It gets compiled but not run by default when you run cargo test.)

1 Like

Thank you for the suggestions!

The #[ignore] #[test] plan will definitely work. It's a little hackish, but the haskishness is isolated and hidden away from users so no big deal.

The idea of using an example also occurred to me, since you can cargo run --example foo. But I have real example code, and it's not ideal to clutter up the examples/ directory with stuff that's not actual example code for the benefit of users.

Actually, this speaks to what's weird about cargo run. From Node.js and npm, I'm used to being able to create script commands in package.json which are invoked via npm run foo. I miss being able to do that in Rust, since I've made heavy use of npm run scripts before.

cargo-xtask seems like it might be the most elegant solution, and the closest in spirit to npm run. But at first glance I was surprised that the cargo-xtask documentation was opinionated enough to recommend organizing the repository as a Cargo workspace. There may be good arguments for workspaces, but it doesn't feel like a tool which provides the npm run foo functionality I'm looking for should care.

xtask isn't a tool — it's a pattern where you write a binary (with the suggested name xtask) that contains your build steps, and it needs to be in the same workspace to be found and compiled by Cargo (sort of bootstrapping).

Yeah, that's very useful indeed. One problem with npm run specific implementation is that it uses system's shell to run stuff. That makes build non-hermetic -- if your's shell is bash, but mine is fish or cmd.exe, npm run might behave differently for us. That's where xtask pattern is less messy -- it runs your binary directly.

That being said, xtask pattern is even more of a hack than #[test] #[ignore] (yes, I have a thing for miss-using Cargo :slight_smile: ), that's why it's very rough around the edges.

You totally can use xtasks without workspaces, but then you'll have two Cargo.locks (one for xtasks, and one for the project proper) and two target directories.

And yeah, the docs are a bit confusing. We probably shouldn't start with teaching how to create a cargo workspace, and go to the meat of the .cargo.config [alias] trick. PRs welcome!


After studying cargo-xtask, what I wound up doing was creating a new package in dev/gen-tests, and then adding the following alias in .cargo/config which allows me to run cargo gen-tests mymodule:

gen-tests = "run --manifest-path dev/gen-tests/Cargo.toml --"

The only complication I encountered was that the CARGO_MANIFEST_DIR pointed at the dev/gen-tests dir rather than the repository root, but I was able to hack that with ../...

For my use case at least, I don't think that's a concern. Since the compilation of the task helper doesn't happen until that helper is invoked, it doesn't slow the main compilation path.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.