Varying Generic Parameters with features

I came across the same problem which was already asked two other times:

I came up with an Idea of how one might be able to make this work with procedural macros. I already tried implementing first steps but am stuck at one particular point from which I cannot seem to solve the problem.

Let me explain the basic idea:

// This struct has two generic parameters which depend on features
// This code will compile just fine
Struct<
    #[cfg(feature = "first") A,
    #[cfg(feature = "second")] B,
> {...}

// Hopefully this procedural macro can fix our problem.
// The implementation of [Trait] for [Struct] should be doable with
// any combination of features enabled/disabled.
#[feature_generics(
    #[cfg(feature = "first") A,
    #[cfg(feature = "second") B
)
impl<A, B> Trait<A, B> for Struct<A, B> {...}
  1. Create an attribute procedural macro, which will be used on every impl<T1, T2, ...>
  2. The attributes of the macro will tell the compiler which generic parameters are desired with which features enabled.
  3. We parse the impl block and replace every occuring instance of <A, B> with either <A, B>, <A>, <B> or nothing depending on which features are enabled.
  4. The inside of the impl block will still be needed to be guarded by the correct #[cfg(feature = ...)] flags at relevant positions.

The problem with this approach seems that it is not possible to evaluate if a feature is enabled when we parse it as a token from a token stream. The compiler knows about enabled features since #[cfg(feature = ...)] will work in procedural macros.
There are two options how to fix this problem which I tried with no luck.

  1. Evaluate if a feature is defined via the cfg!(feature = ...) macro. This does not seem to be working as it is not possible to unpack a proc_macro::TokenTree as a string inside the macro. Rather, the written variable will be parsed (ie. cfg!(feature = token.as_string()) will be looking for the feature "token.as_string()")
  2. Find the names of all currently enabled features and see if the TokenTree is contained in there. However, I have not found a way to list enabled features, not even with unstable intrinsics of the std crate.

The last option would be to generate the code for each possible implementation and guard it with multiple respective #[cfg(feature = ...)] flags.

#[cfg(feature = "first")]
#[cfg(feature = "second")]
impl<A, B> Trait<A, B> for Struct<A, B> {...}

#[cfg(feature = "first")]
#[cfg(not(feature = "second"))]
impl<A> Trait<A> for Struct<A> {...}

#[cfg(not(feature = "first"))]
#[cfg(feature = "second")]
impl<B> Trait<B> for Struct<B> {...}

#[cfg(not(feature = "first"))]
#[cfg(not(feature = "second"))]
impl Trait for Struct {...}

This last solution was not tried by me so far and seems not very nice since it leads to loads of code being generated before being thrown out again. This intermediate step scales by 2^N_GENERICS.

Macros are not magic bullets. Every Rustacean should be aware of

the one profound insight about Rust macro development: the difference between someone who is competent with macros vs an expert at macros mostly has nothing to do with how good they are "at macros".

90% of what enables people to push the limits of possibility in pursuit of a powerful and user-friendly macro library API is in their mastery of everything else about Rust outside of macros, and their creativity to put together ordinary language features in interesting ways that may not occur in handwritten code.

You may occasionally come across Rust macros that you feel are really advanced or magical. If you ever feel this way, I encourage you to take a closer look and you'll discover that as far as the macro implementation itself is concerned, none of those libraries are doing anything remotely interesting. If it is a procedural macro, they always just parse some input in a boring way, crawl some syntax trees in a boring way to find out about the input, and paste together some output code in a boring way exactly like what you would learn in a few hours by working through any part of my procedural macro workshop. If it is a macro_rules macro, everything is conceptually just as boring but when stretched to its limits it becomes a write-only syntax that poses a challenge for even the author to follow and understand later, let alone someone else not already fluent in the basics of macro_rules.

To the extent that there are any tricks to macro development, all of them revolve around what code the macros emit, not how the macros emit the code. This realization can be surprising to people who entered into macro development with a vague notion of procedural macros as a "compiler plugin" which they imagine must imply all sorts of complicated APIs for how to integrate with the rest of the compiler. That's not how it works. The only thing macros do is emit code that could have been written by hand. If you couldn't have come up with some piece of tricky code from one of those magical macros, learning more "about macros" won't change that; but learning more about every other part of Rust will. Inversely, once you come up with what code you want to generate, writing the macro to generate it is generally the easy part.

from: Dtolnay GitHub - dtolnay/case-studies: Analysis of various tricky Rust code

I would try to write feature gated stuff in separate modules and shared stuff in shared module. Why? Because using feature gated modules takes advantage of module system in Rust and is common in practice.

And deduplicate boilerplate code by type system (i.e. structs/enums/traits) or macros (for things type system can't deal with) if necessary.

use gated::{Struct, Trait};
use shared::Shared;

// ********** Usage **********

fn main() {
    let s = Struct {
        #[cfg(any(feature = "first", feature = "second"))]
        a: (),
        #[cfg(feature = "first")]
        #[cfg(feature = "second")]
        b: (),
        shared: Shared {},
    };
    eprintln!("{s:?}",);
    s.f();
}

// ********** Definition **********

pub mod shared {
    #[derive(Debug)]
    pub struct Shared {}
}

#[cfg(not(feature = "first"))]
#[cfg(not(feature = "second"))]
pub mod gated {
    use crate::shared::Shared;

    pub trait Trait {
        fn f(&self) {
            eprintln!("neither first nor second")
        }
    }
    #[derive(Debug)]
    pub struct Struct {
        pub shared: Shared,
    }
    impl Trait for Struct {}
}

#[cfg(any(
    all(not(feature = "first"), feature = "second"),
    all(not(feature = "second"), feature = "first"),
))]
pub mod gated {
    use crate::shared::Shared;

    pub trait Trait<A> {
        fn f(&self) {
            eprintln!("first or second")
        }
    }
    #[derive(Debug)]
    pub struct Struct<A> {
        pub a: A,
        pub shared: Shared,
    }
    impl<A> Trait<A> for Struct<A> {}
}

#[cfg(feature = "first")]
#[cfg(feature = "second")]
pub mod gated {
    use crate::shared::Shared;

    pub trait Trait<A, B> {
        fn f(&self) {
            eprintln!("first and second")
        }
    }
    #[derive(Debug)]
    pub struct Struct<A, B> {
        pub a: A,
        pub b: B,
        pub shared: Shared,
    }
    impl<A, B> Trait<A, B> for Struct<A, B> {}
}
$ cargo r --no-default-features -q
Struct { shared: Shared }
neither first nor second

$ cargo r --features first -q
Struct { a: (), shared: Shared }
first or second

$ cargo r --features second -q
Struct { a: (), shared: Shared }
first or second

$ cargo r --features first,second -q
Struct { a: (), b: (), shared: Shared }
first and second

Update: for real module structure

mod gated {
    #[cfg(all(feature = "first", feature = "second"))]
    #[path = "first_and_second.rs"]
    pub mod inner;

    #[cfg(any(
        all(not(feature = "first"), feature = "second"),
        all(not(feature = "second"), feature = "first"),
    ))]
    #[path = "first_or_second.rs"]
    pub mod inner;

    #[cfg(all(not(feature = "first"), not(feature = "second")))]
    #[path = "neither_first_nor_second.rs"]
    pub mod inner;
}
pub use gated::inner::{Struct, Trait};

  โ”Œโ”€โ”€ main.rs
  โ”‚ โ”Œโ”€โ”€ first_and_second.rs
  โ”‚ โ”œโ”€โ”€ first_or_second.rs
  โ”‚ โ”œโ”€โ”€ neither_first_nor_second.rs
  โ”œโ”€โ”ด gated
โ”Œโ”€โ”ด src
3 Likes

Thanks for your answer. As I mentioned earlier, the amount of code one would be required to write scales up very significantly when more than 2 or 3 variable generic parameters have to be considered.
In my use-case, I have a struct with approx. 1000 lines of code in methods which all rely on its generic parameters. Writing out every implementation for only 3 variable generic parameters would require me to write 8000 lines of code.
This solution is not scalable or maintainable in my opinion. In addition, when doing changes to code that is being shared, one has to adjust all of them accordingly.
I also do not see how to separate common code since every impl<...> block still needs to be written out and will thus require individual #[cfg(feature = ...)] flags.

Another option which may or may not be viable in your case is to keep all of the type parameters regardless of features and just use a zero sized type like () to fill in the respective type parameter when that feature is disabled

2 Likes

Shared code is untouched.

The feature is gated on mod, instead of impl blocks, if you do read the example I give. And the flag is written only once.


Other ways:

  • Variadic parameters can be mock via tuples, and you can reduce the generic parameters to a single one.
  • Write the full generic parameters you need in all cases, and reduce them with default parameters in each separate case. This is what @semicoleon suggests.
2 Likes

You might be interested in this trick that I cobbled together recently. It's a hack, but it does allow a proc macro to determine the true/false state of a #[cfg(...)] predicate in a downstream crate, and it doesn't break any of Rust's rules for macros. In your particular case, if you know all the features ahead of time, it's actually easier. You could do something like the following:

  #[cfg(feature = "first")]
  macro_rules! check_first{
    (($($bools:expr),*), $($args:tt)*) => {
      check_second!(($($bools,)* true), $($args)*);
    }
  }
  #[cfg(not(feature = "first"))]
  macro_rules! check_first{
    (($($bools:expr),*), $($args:tt)*) => {
      check_second!(($($bools,)* false), $($args)*);
    }
  }

  #[cfg(feature = "second")]
  macro_rules! check_second{
    (($($bools:expr),*), $($args:tt)*) => {
      my_proc_macro!(($($bools,)* true), $($args)*);
    }
  }
  #[cfg(not(feature = "second"))]
  macro_rules! check_second{
    (($($bools:expr),*), $($args:tt)*) => {
      my_proc_macro!(($($bools,)* false), $($args)*);
    }
  }

And then using

check_first!((), args...);

will expand to

my_proc_macro!((true, false), args...);

where the bool literals are the state of each cfg predicate, in order. The proc macro will then see and be able to parse those literals to know each cfg predicate state.

One other alternative is that if you're only ever going to be generating these generics in your own crate, you could just forward the features directly to the proc macro in your Cargo.toml when you pull the proc macro crate in as a dependency. That way it will have the same features enabled/disabled locally.

1 Like

Don't forget that the compiler will still need to parse anything your proc-macro generates, and even assuming the conditional compilation lets us skip type checking for unused stuff, the time for that parsing process is roughly linear with respects to input sizes... You'll probably find that using this technique too much will cause your library to be prohibitively slow to compile.

Thanks to everyone for the many different answers.

Let me clarify. Suppose you have some functionality which should be present for feature = "first". Then I would still have to write the code multiple times for each impl<A, ...> block. I should have probably called it "partially shared code" (which should still not be duplicated).

I briefly thought about this too. It seems to be a good candidate as well. The compiler should be able to simply discard the additional T=() generics.

Yes I totally agree! It's not an actual solution but rather a coping mechanism.

I like this approach! It might actually be able to solve my problem. I will compare this approach to the other one mentioned earlier by @semicoleon and will let you know how I decided to solve it.

If you wanted to write a proc macro in a crate and release it publicly for anyone to use, it might be unwieldy from a users perspective. It seems to be that any procedure requires the user to additionally specify activated features in some other place than to only activate it in the respective library. At the moment, the task of writing a proc macro that works and can reliably generate correct code seems to be more daring than simply defaulting generics with <T=(), ...>. I will keep you posted.

Depending on how you're planning on structuring the "keep all type parameters" version, you may end up needing a type alias rather than using defaults on the type parameter. Rust doesn't currently use type parameter defaults when doing type inference, so if any impls include a type parameter in one of the "disabled" slots you may not be able to use that impl without specifying the defaulted types.

Playground

struct Sample<A, B = ()>(A, B);

impl<A, B> Sample<A, B> {
    fn new(a: A) -> Sample<A, B> {
        todo!()
    }
}

impl<A> Sample<A> {
    fn new_a(a: A) -> Self {
        todo!()
    }
}

fn main() {
    // Works
    Sample::new_a(1);
    // type annotations needed
    Sample::new(1);
}

In that example you could export a type alias type Whatever = Sample<A, ()>; which would avoid the problem since it would always specify the unused type parameters as ().

I think there are only two cases:

  1. You have a fixed set of features that MyLibraryCrate advertises for other crates that depend on MyLibraryCrate to enable or disable. If you know all the features ahead of time, you can use the hand-specified cfg checker macro chain I listed above.

  2. The user specifies #[cfg(...)] attributes as part of their input that you're parsing into tokens for a given macro, and the macro needs to be able to evaluate the state of the predicates in the #[cfg(...)] attribute as it parses. This is the use case that I had to solve and I go into more detail on that here: Supporting or evaluating #[cfg(...)] in proc macro parameters? - #2 by recatek -- the summarized version is that you parse the input twice: in the first pass you find all the #[cfg(...)] attributes and build a unique ordered set of their predicates. Then you generate the cfg checker macro chain I described above from that predicate set, and have the chain terminate in a call to your second "real" proc macro that takes in the cfg predicate states, and the data, and actually generates your output.

The two-parse thing isn't ideal but in cases like this where the alternative would be a combinatorial explosion of output code I think it's preferable. That said, in my case I explicitly couldn't do the T=() option since it would have had a runtime cost, so if you can then that's probably preferable.

I stumbled upon the exact problem you are describing. Since this part of the code is user-facing, I do not want to specify a type. I could simply write a macro that automatically does the correct stuff, but in my case the user-facing code is rather small and only serves as an entry point. Thus I chose to simply write two distinct functions depending on if the feature is active or not. This works for now but is of course what I wanted to circumvent.

Btw: I was searching the rust github repo and could not find any issues talking about this problem. In my opinion the language should support such functionality. I am thinking about filing an RFC on this.

1 Like

Do you mean "default type parameter fallback"? It's a well-known problem.

I think they meant allowing attributes in a generic parameter list so you could #[cfg()] generic parameters instead of doing the default type parameters dance

1 Like

Not quite what I was looking for. @semicoleon got it totally right. What I would like to do is the following:

// This syntax is totally fine and working right now
struct Container<T, #[cfg(feature = "some_feature")] S> {
    element: T,
    #[cfg(feature = "some_feature")]
    second_element: S,
}

// THIS is the part which is not working
impl<T, #[cfg(feature = "some_feature")] S> Container<T, #[cfg(feature = "some_feature")] S> {
    fn get_element(&self) -> T {
        self.element
    }

    #[cfg(feature = "some_feature")]
    fn get_second_element(&self) -> S {
        self.second_element
    }
}

What are your opinions? Should I file an RFC?

Oh somehow I missed that it was ONLY the type name part of the impl that caused an error. You actually have one more option, define a type name macro.

Playground

#![allow(unused)]

// This syntax is totally fine and working right now
struct Container<T, #[cfg(feature = "some_feature")] S> {
    element: T,
    #[cfg(feature = "some_feature")]
    second_element: S,
}

#[cfg(feature = "some_feature")]
macro_rules! ContainerType {
    ($t:ident, $s:ident) => {
        Container<$t, $s>
    };
}
#[cfg(not(feature = "some_feature"))]
macro_rules! ContainerType {
    ($t:ident, $s:ident) => {
        Container<$t>
    };
}

// THIS is the part which is not working
impl<T: Clone, #[cfg(feature = "some_feature")] S> ContainerType![T, S] {
    fn get_element(&self) -> T {
        self.element.clone()
    }

    #[cfg(feature = "some_feature")]
    fn get_second_element(&self) -> S {
        self.second_element
    }
}

The version of the macro when the feature isn't enabled just discards the S token so it has basically the same effect as the #[cfg()] you want to use.

It doesn't make it easy for other crates to abstract over whether or not the feature is enabled in your crate, but that might not matter.

2 Likes

Sweet. That's at least one step further again. Didn't think about using a macro in this place (and tbh did not know it was possible). But still the impl<..> block will be a pain :frowning:

Gotcha, sorry I misunderstood.

I don't have any strong opinions on the feature, [1] but if you want to pursue it I suggest bringing it up in the places teams hang out.


  1. and don't have the time to dig in and form one right now :slightly_smiling_face: โ†ฉ๏ธŽ