Composable binaries

1

I would like to build programs, that are composed from two parts: Generators and Consumers.
Lets call them "partial binaries" that are eventually composed into a true, executable binary.

For example two generators:

fn generate() -> u32 {
    13
}

fn generate() -> u32 {
    125
}

And (for brevity just one) consumers:

fn consume(num: u32) {
    if num == 42 {
        println!("My favorite number!");
    } else {
        println!("Not my favorite number: {}", num);        
    }
}

Eventually, there will be many generator and consumers and
I would like to 'magically' execute a pair of them: random_prime + is_favorite results in stdout Not my favorite....

Question: How would you do this?

I will go into further detail below how I would build this. But any suggestion different form my approach below is equally welcome.

My results so far:
By compiling all consumers and generators as libraries, I could eventually assemble the executable by linking together generate and consume.
Throwing in the glue code obviously:

fn main() {
    consume(generate());
}

I have not fully build this out, but I feel like this would come to the desired result.
The big downside is that this requires a large number of crates just to compile them into libraries - one crate for each consumer/generator.

Compiling random_primes to an dedicated partial binary artifact would be much more elegant and cargo allows the definition of many binaries in a single crate.
I just need to rip out the traditional linker step and eventually link them manually with the glue code.

I was able to get this somewhat to work by running

cargo rustc --bin random_prime -- --emit=obj
cargo rustc --bin random_cube -- --emit=obj
cargo rustc --bin is_favorite -- --emit=obj

which yields the object files. These can then be linked with the glue code to an executable (details omitted for brevity).

From my perspective this appears to be conceptually correct, although fairly elaborate.
What do you think? Is there an easier way?

Additional notes:

  • My use case it not at all about generators/consumers, but this was the best minimal example I could come up with.
  • I plan to use this with embedded targets eventually. Hence, dynamic linking/loading is not an option.
2

Your real-world use case still is unclear to me. I don't understand why you would want two separate binaries in the first place if your only goal is to combine them.

You can achieve conditional compilation either by attribute flags (feature or target flags, depending on your actual use case) or just predefine all the needed binaries and let them import the respective distinct code.

2 Likes
3

I don't understand why you would want two separate binaries in the first place if your only goal is to combine them.

My motivation is purely development ergonomics. I want to rapidly iterate through different combinations of generators and consumers.

Say there are 10 generators and 10 consumers, they would combine into 100 binaries.
I would like to avoid defining 100 main functions manually.

I will make an attempt with feature flags: cargo run -F random_primes -F is_favorite would yield the desired result.
I am not sure how much boilerplate is required outside the "partial binary" itself, but it's certainly worth drafting.

4

I see. So you want to do some kind of meta-programming to assemble certain kinds of combinations of functions. Maybe you can automate that with a build.rs script.

5

Is there a strong reason this experimenting needs to switch at build time? Are you using link time optimization for example?

If not, this looks like it could simply switch the generator and consumer implementations based on arguments, for example.

fn main() {
  let mut args = std::env::args();
  args.next();
  let generator = args.next().as_deref().unwrap_or("foo");
  let consumer = args.next().as_deref().unwrap_or("bar");
  let value = match generator {
    "foo" => foo_generator(),
    ...
    _ => panic!("Invalid generator: {generator}"),
  };
  match consumer {
    "bar" => bar_consumer(value),
    ...
    _ => panic!("Invalid consumer: {consumer}"),
  };
}

A bit clumsy, but should be fine for experimenting?

1 Like
6

Another option I'd consider is just whipping up a <your_favorite_scripting_language> [1] script and generate and compile the respective main.rs files with that.

  1. I'd use Python for that ↩ī¸Ž