Tricky conditional build problem with no_std

Hi there!

I'm running into a problem that I believe is caused by some conditional compilations around a no_std feature I'm trying to build. I've described the exact instance of the problem on my PR to capnproto-rust, but I'll outline what I believe the major components are here.

I would like to be able to build capnproto-rust as a no_std dependency in my project, but in order to do that I need to make some no_std substitutions for some std features. Specifically, capnproto-rust makes heavy use of the std::io::{Read, Write} traits, as well as some other std::io utilities. I found rust-core_io which is a patched version of std::io which rewrites many of the std::io implementations to lean on a custom allocator (using the alloc crate). Therefore, I figured I should be able to make conditional compilations where the implementations use std::io in std mode and core_io::io in no_std mode.

Jumping forward a bit, I set up this conditional compilation and worked through all of the renaming that I needed to do, and eliminated all of the compile errors except for this one:

$ cargo build
   Compiling capnpc v0.10.1 (/home/nick/Documents/capnproto-rust/capnpc)
error[E0277]: the trait bound `T: core_io::io::Read` is not satisfied
    --> /home/nick/Documents/capnproto-rust/capnpc/src/codegen.rs:1852:19
     |
1852 |     let message = serialize::read_message(&mut inp, capnp::message::ReaderOptions::new())?;
     |                   ^^^^^^^^^^^^^^^^^^^^^^^ the trait `core_io::io::Read` is not implemented for `T`
     |
     = help: consider adding a `where T: core_io::io::Read` bound
     = note: required by `capnp::serialize::read_message`
error: aborting due to previous error
For more information about this error, try `rustc --explain E0277`.
error: Could not compile `capnpc`.
To learn more, run the command again with --verbose.

Here's some context surrounding the problem code (found here in capnproto):

/// Generates Rust code according to a `schema_capnp::code_generator_request` read from `inp`.
pub fn generate_code<T>(mut inp: T, out_dir: &::std::path::Path) -> ::capnp::Result<()>
    where T: ::std::io::Read
{
    use capnp::serialize;
    use std::io::Write;

    let message = serialize::read_message(&mut inp, capnp::message::ReaderOptions::new())?;
    // snip
}

So here's the tricky part. There are two main crates in capnproto-rust, capnp, the dependency that gets linked into capnproto projects, and capnpc, which is the "compiler" that generates rust bindings at build time according to a given capnproto schema. However, capnpc also depends on capnp. So I would expect the dependencies to look something like this:

# Application dependency | my-nostd-application -> capnp (should be compiled as no_std)
# Build-time dependency  | capnpc -> capnp (should be compiled with std)

If we look back at the error message, we see that the generate_code function in capnpc takes a generic parameter (mut inp: T) which implements std::io::Read, but that serialize::read_message from capnp is asking for a parameter which implements core_io::io::Read. I believe the problem here is that cargo looks at the requirements for building the capnp dependency and sees that my application is depending on capnp with the no_std feature flag enabled, so it compiles capnp accordingly, producing an implementation of serialize::read_message<R>(read: &mut R, ...) where R: core_io::io::Read {...}. However, it fails to notice that capnpc depends on capnp with a different set of feature flags, notably not using the no_std flag. I believe the actual dependency graph produced looks something like this, where both capnpc and my application are depending on the no_std version:

# Application dependency | my-nostd-application -> capnp (compiled as no_std)
# Build-time dependency  | capnpc -------------/

I believe the expected behavior would be for cargo to compile capnp a separate time, producing an implementation of serialize::read_message<R>(read: &mut R, ...) where R: std::io::Read {...}, at which point the types should line up and everything should work.

So I suppose my question is whether my interpretation of the problem seems reasonable, or whether there's a simpler explanation or solution to what I'm seeing? Are there some best-practices for conditional compilation that I'm missing, or is this just an unfortunate edge-case? What might be my best path forward from here? Any advice on this would be greatly appreciated!

Features should generally be additive, so usually you'll see no-default-features being the no_std build, then a std feature as an opt-in or default for the extra functionality that needs it.

But even then you'll still hit the problem that features are unified across all dependencies, build or otherwise:
https://github.com/rust-lang/cargo/issues/2589

I did notice that the trend seems to be use a std feature rather than a no_std feature, but the reason I did it the other way around this time is because I wanted core_io to be an optional dependency that's only included when building with the no_std feature. I had made this change in the Cargo.toml:

[dependencies]
core_io = { version = "0.1.20190701", features = [ "alloc", "collections" ], optional = true }

[features]
no_std = ["core_io"]

Do you know of any way to reverse this optional dependency inclusion? I.e. if I were to use the pattern of making the feature be std, could I include core_io only when the std feature is not present?

Maybe use features for both? Then I guess a pure no-default-features wouldn't work, but each dependant can choose what they need. Note that optional dependencies already become a feature in themselves, so you can have std and core_io as your features. Maybe call it std_io for consistency.

If both features are activated, can you implement them at the same time? Maybe not, if you have APIs like read_message<R> where R requires different Read types. That means these features aren't really additive -- this may be OK as long as you note it as a caveat of your crate.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.