Parametric building

1

I'm developing a simulator. I decided, from the very beginning, that most crucial parameters of simulation would be compile-time constants to make code as fast as possible.

However, I need to run the same simulation with different parameters. To achieve this, the compile-time parameters can be set via environment, like:

const foo: usize = match std::option_env("FOO_VAL")
   Some(s) => parse_uint(s),
   None => default_foo,
};

I might use cargo build --target-dir some-hash, but then all the depedencies would be downloaded and rebuilded for each hash. Thus I'm doing this by sequential rebuilding, with just changing the env vars, like:

for foo in `100 200 500`
do
    export FOO_VAL=$foo
    cargo build --release
    cp target/release/my_binary my_binary_foo_$f
done

What I would like to have is 1) resusing already the compiled packages, 2) set different target name, 3) run builds in parallel. Am I missing something in cargo build?

2

You can also try cargo install --path . --root out_dir_$foo. This only changes the output directory, not the binary name. Not sure if this can compile in parallel, but if not you can just copy the project a few times. cargo install - The Cargo Book

1 Like
3

One idea might be to use overlayfs with the source directory as the lower layer, and a separate upper layer for each cargo process. that way the processes can run in parallel and reuse build artifacts without interfering with each other.

4
  1. resusing already the compiled packages

Your "sequential build" solution already does this, no?

  1. set different target name

You can edit the package name in Cargo.toml with sed, for example.

  1. run builds in parallel

I don't think cargo supports that, but can definitely copy the project a few times as already mentioned. I am doing something similar for my Bevy game, I have one copy for desktop build and other copy for web browser build (otherwise all dependencies need to be recompiled every rime I change target).

You can easily copy the source between the copies, the "problem" will be distributing the work between the copies. I guess some sort of "queue" with one worker per copy could work?

5

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.