Seeking Guidance on Implementing Rust Macros for Trait and Struct Enhancement

help
1

I am relatively new to the Rust macro context and I am currently working on a project where I need to create two Rust macros basically:

  1. A macro to decorate a Rust trait, for example, [#trait_enhance] for a trait named trait_name.

  2. A macro to decorate a Rust struct, for example, [#trait_enhance(<trait_name>)].

In the first case, the decorated trait may contain illegal Rust syntax inside the trait block, which I want the trait to translate into legal Rust syntax finally. In the second case, the decorated struct has no illegal Rust syntax, but what it derived needs to refer to the final legal code of trait <trait_name>.

I am currently unsure about the overall structure of my project (i.e., how many crates are needed, and what specific crates are required). My basic question is: can I achieve my goals within a single proc-macro crate, similar to the enum_dispatch crate?

More specifically, I have two key concerns:

  1. The order of the two macros matters, as the first macro must be unwrapped before the second. However, I am aware that there are ways to work around this, as demonstrated by the enum_dispatch crate. The main issue is whether a proc-macro can accept input with illegal Rust syntax. If not, I presume I would need to implement the first macro in a separate declarative macro crate (since I understand that a declarative macro cannot be exported from a procedural macro crate).

  2. If another declarative macro crate is needed, can I then define another procedural macro within a normal crate (where "normal" means the crate is not configured as [lib] proc-macro=true and proc-macro2="1.0" can be used) and use it in the definition of the declarative macro defined in the same crate?

(For these issues, I've asked chatGpt multiple times, but the answers are sometimes contradictory, so I'm not sure what to do. I would be very grateful if you could give me some advice.)

2

for attribute macros, it's not possible to use it on syntactical invalid code, at least it needs to be successfully parsed into an item (a trait in your example), but the item can be semantically invalid.

but you can also define function-like procedural macros, which can be used on code with invalid syntax (just be sure the parenthesis pairs are balanced).

please give more contexts on this, but from your brief description, I don't see why it's the case. rust module-level items are order independent, you don't need to put a trait definition (lexically) before structs that uses the trait.

as I said above, attribute-like macros cannot accept syntactically invalid code, but function-like macros can.

generally I would prefer declarative macro if possible, but procedural macros can also be function-like if that's what you mean.

procedural macro crates are compiled differently than "normal" crates, so they must be marked specially with proc-macro = true.

on the other hand, proc-macro2 can be used as dependency by any crates, not necessarily procedural macros only.

you can re-export or create wrappers for procedural macros, but again, procedural macros must be defined in special crates. in fact, it's common for the "main" library crate to re-export procedural macros which are often defined in separate -macros crates. for example, tokio::main is defined in tokio-macros::main but re-exported from the tokio crate as well:

3

please give more contexts on this, but from your brief description, I don't see why it's the case. rust module-level items are order independent, you don't need to put a trait definition (lexically) before structs that uses the trait.

Thanks your reply. Let me put things more specifically.

I want to make a trait can accept variable fields, just looks like this:
(I don't want to argue whethere this is necessary, I just want to see if it is possbile be with the power of macro)

trait_enhance!{
trait MyTrait {
    // let the field definition feasible in trait
    // the below code is formatted arbitrarily for testing purpose.
    let  x: i32;
     let y :bool;
    let z : String ;

    fn trait_method_with_default_impl( &self ) {
        println!("trait_method_with_default_impl， the trait field x is `{}`", self.x);
    }
    // fn trait_method_mut_with_no_default_impl(&  self);

    // `&mut self` method with or without default impl
    fn trait_method_mut_with_default_impl(&  mut self) {
        self.x = 1;
        println!("trait_method_mut_with_default_impl",self.x); 
    }
    // fn trait_method_with_no_default_impl(&mut  self);
}
}

then any struct atrributed with [#trait_enhance(MyTrait)]
can directly use the variable fields defined inside the trait, like this:

#[trait_enhance(MyTrait)]
struct MyStruct {
  a:i32
}
impl MyStruct{
    fn new()->Self{
        Self{
        a:1,
        x:2,
        y:true,
        z:"hello".to_string()
        }
    }

    fn test(&mut self){
        // directly use methods with the trait variable fields.
        self.trait_method_with_default_impl();
    }
}
// optional impl
impl MyTrait for MyStruct{
   // any methods defined in MyTrait, be it has default impl or not, can be overrided here
}

And to make it possible, the plan is this:
the trait code be wrappred by trait_enhance!{ are translated into this:

trait _basicMyTrait{
    fn _x(&self)->&i32;
    fn _x_mut(&mut self)->&mut i32;
    fn _y(&self)->&bool;
    fn _y_mut(&mut self)->&mut bool;
    fn _z(&self)->&String;
    fn _z_mut(&mut self)->&mut String;
}

trait MyTrait: _basicMyTrait{
    fn trait_method_with_default_impl( &self ) {
        println!("trait_method_with_default_impl， the trait field x is `{}`", _x());
    }
    fn trait_method_mut_with_default_impl(&  mut self) {
        _x_mut() = 1;
        println!("trait_method_mut_with_default_impl", _x());
    }
}

Then, with #[trait_enhance(MyTrait)], the atrributed struct would be extended into this:

struct MyStruct {
    a: i32,
    x: i32,
    y: bool,
    z: String
  } 

  impl _basicMyTrait for MyStruct{
    fn _x(&self)->&i32{
        &self.x
    }
    fn _x_mut(&mut self)->&mut i32{
        &mut self.x
    }
    fn _y(&self)->&bool{
        &self.y
    }
    fn _y_mut(&mut self)->&mut bool{
        &mut self.y
    }
    fn _z(&self)->&String{
        &self.z
    }
    fn _z_mut(&mut self)->&mut String{
        &mut self.z
    }
  }

as long as it can get the final code of trait _basicMyTrait. I did make it work with declare-macro
with hard coded fields like this.

I wonder whether this idea is feasible techniqically with macro system.

4

The only inputs to an attribute macro are the token stream for the macro's arguments (MyTrait) and the token stream for the item that the macro is applied to. This macro invocation doesn't see anything from the prior definitions or macro invocation.

There might be an overcomplicated workaround. The first macro invocation could try defining some new macros related to MyTrait and the attribute macro could try inserting invocations of those macros. You'd have to deal with some nasty namespace issues.

5

I am not clear with the term token stream, do you mean the attribute macro can't get the derived _MyTraitBasic code derived by declare-macro?

6

now I see what you mean for the statement "the first macro must be unwrapped before the second".

but, even if the macro for the trait is expanded before the macro for the struct, unfortunately, it's not possible for one macro to pass information to other macros. (well, I think there exists hacky workaround, but I'm not into it).

I think the closest you can get is what @tbfleming has already mentioned, let the trait macro to generate another macro, and then the struct macro translate the struct definition into macro invocations (to the generated macros).

as for the namespace problem, you can take advantage of the fact that macro names and other names exist in different namespaces. one example is std::env, it's both a macro name and a module name, and if you use std::env, you import both at the same time.

7

@tbfleming @nerditation , thanks your suggestion. But I not get it. Do you mean that the attribute macro can't get the expanded code from any declare-macro, even inside the definition of the attribute macro?

And for this:

The first macro invocation could try defining some new macros related to MyTrait and the attribute macro could try inserting invocations of those macros. You'd have to deal with some nasty namespace issues.

I guess this is what I am doing, I am trying to let the attribute macro to catch the expanded code (the macro invocations), the _MyTraitBasic part code.

let the trait macro to generate another macro, and then the struct macro translate the struct definition into macro invocations (to the generated macros).

what does the another macro used for here---if it is generated, then why the attribute macro can catch it but can't catch the expanded trait code?

8

Do you have a fixed set of fields that might be included in the struct? Or, are you aiming to have each _basicMyTrait give an entirely different set of methods for each struct? If you want the generated _basicMyTrait to be a super trait of MyTrait it will need to be a single fixed interface.

// Provide some form of default/failing implementation for missing fields
trait _basicMyTrait {
    fn _x(&self) -> &f64 { &0.0 }
    fn _x_mut(&mut self) -> &mut f64 { panic!() }
    fn _z(&self) -> &i32 { &0 }
    fn _z_mut(&mut self) -> &mut i32 { panic!() }
}

Then each struct will be able to overrride just the methods for the fields it does have:

struct A { x: f64 }

impl _basicMyTrait for A {
    fn _x(&self) -> &f64 { &self.x }
    fn _x_mut(&mut self) -> &mut f64 { &mut self.x }
}

struct B { z: i32 }

impl _basicMyTrait for B {
    fn _z(&self) -> &i32 { &self.z }
    fn _z_mut(&mut self) -> &mut i32 { &mut self.z }
}

But you couldn't add an otherwise undefined field/method:

struct WouldFail { q: Foo }
// Unless you add `q` field access methods to _basicMyTrait

impl _basicMyTrait for WouldFail {
    fn _q(&self) -> &Foo { &self.q }
    fn _q_mut(&mut self) -> &mut Foo { &mut self.q }
}

My bigger question is why do you want field access methods in a trait? You can use a proc macro to implement field access methods as inherent methods on the struct. Thus the proc macro that reads the struct definition to generate the MyTrait implementation would know what fields there are, so it could know that the type it's being implemented for would provide a .x() or .bob(), as applicable. Going through unified field access methods via an intermediary trait seems.... unnecessary.

9

I'm also still not seeing why one macro is dependent on the other. This feels like an important bit that was pointed out:

if something is implemented on a type somewhere in the module, it is implemented for that type, full stop.

10

My bigger question is why do you want field access methods in a trait?

Simply speaking, sometimes we just need to avoid redundant code(variable fields) for structs having same traits(the traits doesn't only mean interface/function/method, but also means same data field), just like discussed in this link.

11

Hmm, my quick look at that initially made me think it was a question of struct splitting; i.e. so taking one small part of a big struct as mutable didn't unnecessarily flag all the other fields as mutably borrowed.

But a bit more reading also made it seem like you are interested in using the trait to guarantee memory layout for the field? Would associated types help here?

trait AccessField {
    type FieldType;
    fn field(&self) -> &Self::FieldType;
    fn field_mut(&mut self) -> &mut Self::FieldType;
}

Then if structs have the field in different types you can match to what the field is:

struct Field2D {
    point: Point2D,
}

impl AccessField for Field2D {
    type FieldType = Point2D;
    fn field(&self) -> &Self::FieldType { &self.point }
    fn field_mut(&mut self) -> &mut Self::FieldType { &mut self.point }
}

And if downstream accessors of Self::FieldType need, you can add trait bound requirements so that the access trait can only give out suitably capable types.

trait AccessField {
    type FieldType: Debug + Copy + PointMethods + OtherInfo;
    // .. access methods, etc.
}

That would let traits be blanket implemented as long as a given set of suitably implemented field values can be accessed. So if you build the access method traits as individual pieces, you can construct the combinations in a somewhat expressive manner. Probably a lazy example, but something like:

trait AccessPoint { type Point: PointMethods; }
trait AccessDiameter { type Diameter: Length; }

trait Circle: AccessPoint<Point = Point2D> + AccessDiameter { fn draw(&self) }
trait Sphere: AccessPoint<Point = Point3D> + AccessDiameter { fn draw(&self) }

impl<U: AccessPoint<Point = Point2D> + AccessDiameter> Circle for U { 2D stuff }
impl<U: AccessPoint<Point = Point3D> + AccessDiameter> Sphere for U { 3D stuff }
12

A token is an identifier, number, symbol (+, /, &), etc. A token stream is an ordered sequence of tokens. Macros take token streams as input and produce token streams as output.

Correct

Correct

Not what I meant. The first macro invocation could expand into a new, generated macro_rules definition (call it foo). The attribute macro could include the token sequence foo!(...) in its output, causing the compiler to expand that after your attribute macro is done.

I recommend against attempting this approach until you have experience writing proc and attribute macros though. It's full of pitfalls.

13

This! I think I'm starting to begin to feel somewhat proficient at making regular macros more adaptable. But I still often find it best to write one implementation I want without the macro to validate it, and then wrap that in a macro_rules! and start working in repeated type and ident bits as macro invocation inputs. And proc macros are a whole other can of worms!

As much as I end up using them, I do think esoteric solutions in Rust are a code smell. Whenever I find myself reaching for something tricky I try stepping back to looking at the problem from different perspectives, particularly trying to separate out and encapsulate specific capabilities. Composition over inheritance, and all that.

14

I think this approach is not feasible, since the trait pattern I offered contains invalid rust syntax, I guess the attribute macro would not even be initialed.

15

I tried out an approach using declarative macros in the playground and I got something that mostly works.

17

This is a brilliant idea! You just use declarative-macro with internal rules pattern to make almost everything work in a single macro. What excites me is that the let <var_name>: <type_name>; statement inside the trait is recognized by the rust compiler, so now it is easy to change these variable field names by using the rust extension(rust Analyzer) in vscode.

The only foible is that users still needs to take self.get_fields().x but not self.x like code in the fn body of the trait. I wonder if it is technically possible to replace all self.<trai_var_name> to _<trai_var_name>() or _<trai_var_name>_mut(), accordingly by just declarative macro? (I hope so, even crate syn is needed).

Anyway, thanks for your code(Before this post, I ran into an issue with ambiguity to distinguish functions with/without a body in my test declarative macro, I tried to use tt-muncher pattern, but not solved yet), it really inspired me, though I still can't fully understand it yet.

18

Thank you all again! Now I did make the crate out, trait_varaible with the idea from @Lej77 , but within a single procedural macro crate. And still there are some drawbacks as I stated in the section limitation.