Cannot access self fields in struct generated by macro_rules!

Anachoreta · October 22, 2022, 4:32pm

I want to access self fields from struct dynamically generated by macro_rules!, but got an error:

error[E0308]: mismatched types
    --> src/main.rs:102:45
     |
102  |                               packet.write_u8(self.$field_name).unwrap();
     |                                      -------- ^^^^^^^^^^^^^^^^ expected `u8`, found struct `String`
     |                                      |
     |                                      arguments to this function are incorrect
...
117  | /     packet! {
118  | |         properties {
119  | |             opcode 100;
120  | |             size: u16;
...    |
128  | |         }
129  | |     }
     | |_____- in this macro invocation
     |
note: associated function defined here
    --> /playground/.cargo/registry/src/github.com-1ecc6299db9ec823/byteorder-1.4.3/src/io.rs:1098:8
     |
1098 |     fn write_u8(&mut self, n: u8) -> Result<()> {
     |        ^^^^^^^^
     = note: this error originates in the macro `packet` (in Nightly builds, run with -Z macro-backtrace for more info)

For more information about this error, try `rustc --explain E0308`.

this is my code:

use std::io::{Write};
use std::net::Ipv4Addr;
use byteorder::{BigEndian, LittleEndian, WriteBytesExt};

macro_rules! packet {
    // first rule
    (
        properties {
            $(opcode $opcode_value:expr;)?
        }
        
        $(#[$outer:meta])*
        $vis:vis struct $PacketStruct:ident {
            $($field_name:ident: $field_type:ty),*$(,)?
        }
        
        $($StructImpl: item)*
    ) => {
        $(#[$outer])*
        #[derive(Clone)]
        $vis struct $PacketStruct {
            $($field_name: $field_type),*
        }
        
        $($StructImpl)*
        
        $(
            impl $PacketStruct {
                pub fn get_opcode() -> u32 {
                    $opcode_value
                }
            }
        )?
    };
    // eof first rule
    // second rule
    (
        properties {
            opcode $opcode_value:expr;
            $($prop_name:ident: $prop_type:ty;)*
        }
        
        $(#[$outer:meta])*
        $vis:vis struct $PacketStruct:ident {
            $($field_name:ident: $field_type:ty),*$(,)?
        }
        
        $($StructImpl: item)*
    ) => {
        packet! {
            properties {
                $($prop_name: $prop_type;)*
            }
            $(#[$outer])*
            $vis struct $PacketStruct {
                $($field_name: $field_type),*
            }
            
            $($StructImpl)*
            
            impl $PacketStruct {
                pub fn get_opcode() -> u32 {
                    $opcode_value
                }
            }
        }
    };
    // eof of second rule
    // third rule
    (
        properties {
            $($prop_name:ident: $prop_type:ty;)*
        }
        
        $(#[$outer:meta])*
        $vis:vis struct $PacketStruct:ident {
            $($field_name:ident: $field_type:ty),*$(,)?
        }
        
        $($StructImpl: item)*
    ) => {
        $(#[$outer])*
        #[derive(Clone)]
        $vis struct $PacketStruct {
            $($field_name: $field_type),*
        }
        
        $($StructImpl)*
        
        impl $PacketStruct {
            pub fn test() {
                $(
                    println!("{:?}", stringify!($prop_type));
                )*
            }
            
            pub fn to_binary(&mut self) {
                let packet: Vec<u8> = Vec::new();
                $(
                    match stringify!($field_type) {
                        "u8" => {
                            packet.write_u8(self.$field_name).unwrap();
                        },
                        "u16" => {},
                        "u32" => {},
                        "String" => {},
                        _ => {},
                    }
                )*
            }
        }
    };
}


fn main() {
    packet! {
        properties {
            opcode 100;
            size: u16;
        }
        
        #[derive(Hash)]
        pub struct Outcome {
            name: String,
            os: u8,
            game_name: String,
        }
    }
    
    println!("{:?}", Outcome::get_opcode());
    Outcome::test();
}

this is sandbox.

Could somebody explain what I did wrong ? How to access self fields in macro_rules! ?

H2CO3 · October 22, 2022, 4:55pm

Well, there's nothing wrong with accessing the field. Your type's name field is a String, though, so I'm not sure why you expect to be able to pass it to a method that takes u8.

Thinking about it, even the way you are trying to differentiate between field types doesn't make sense. You are matching on the stringification of the field type, and trying to dispatch to differently-typed writers on that. This can't possibly work: a field always has a given specific type, so if you have some code that uses it, all such code will see it as having that one type. And of course all of an expression has to compile without error in order for it to compile; match statements are no exception (match matches on runtime values of a specific type, not on different types), so you can't treat field as u8 in one branch and as a String in another.

What you probably wanted instead is implement a trait for all supported field types with a method that allows each such type to write a binary representation of itself into the passed buffer.

Anachoreta · October 22, 2022, 5:44pm

I am not sure if understand correctly where I should implement the trait. Should it be trait on global type ?
Like impl std::io::Write on String ? Or this should be the trait on struct generated by macro_rules! ?

If so how this trait should looks like ? Could you attach some small example ? (it doesn't necessary to provide sandbox, just to point me in correct direction).

H2CO3 · October 22, 2022, 5:52pm

On each type that you want to write into a buffer:

trait ToBinary {
    fn to_binary(&self, buf: &mut Vec<u8>) -> Result<(), Error>;
}

impl ToBinary for u8 {
    fn to_binary(&self, buf: &mut Vec<u8>) -> Result<(), Error> {
        buf.write_u8(*self)
    }
}

impl ToBinary for String {
    fn to_binary(&self, buf: &mut Vec<u8>) -> Result<(), Error> {
        buf.extend_from_slice(self.as_bytes());
        Ok(())
    }
}

Anachoreta · October 22, 2022, 5:55pm

I got it, thank you ! Could you also tell is such approach a common practice ? Or this one can lead to some bugs or undesired behavior ? I am not feel OK about extending standard types.

H2CO3 · October 22, 2022, 6:04pm

It's fine. It's basically the #1 solution when it comes to serialization (see e.g. serde's Serialize and Deserialize traits).

Why?

Cerber-Ursi · October 22, 2022, 6:06pm

Note that this "extension" must be explicitly acknowledged by the user (by importing the trait to scope or using generics), unlike e.g. JavaScript, where changes to prototype are immediately visible everywhere.

Anachoreta · October 22, 2022, 6:09pm

probably that's because of JavaScript approaches (my first language and still one of most usable for now), where when you add something to prototype it became global. So I used to avoid such approaches

Probably it's possible to make trait local ? So if I define it in same file (or module) where macros defined will it still visible outside ?

P.S. I answered to wrong post, not sure how to change that. It's the answer to "Why?"

Anachoreta · October 22, 2022, 6:10pm

Got it, so in case I disable import trait will be usable only within macros right ?

Cerber-Ursi · October 22, 2022, 6:12pm

Well, everything that user can do with macros they can do without them (by essentially copying over the output of cargo expand and probably adjusting some parentheses). Point is, they will have to use your trait to see the extension - they will not blunder into it unexpectedly.

H2CO3 · October 22, 2022, 6:13pm

You don't need to. Methods on a trait never conflict with any other method of any other trait. If two traits currently in scope define a method with the same name, and you try to call it, you get a compile error, so you can't call the wrong method by accident. And if your trait method has the same name as an inherent method, then the inherent method takes precedence, so you can't surprise-override what e.g. String::len() does. It also doesn't affect any code that doesn't explicitly bring the trait in scope, so you can rest assured that a new trait you created won't mess up any 3rd-party code – all 3rd-party code will be ignorant of your own trait unless it explicitly opts for using it.

Rust is an exceptionally well-designed language, unlike JavaScript. It's strongly typed, and the designers take a lot of care not to introduce footguns like this. You basically don't have to worry about this kind of problem unless you get an explicit compiler error.

Anachoreta · October 22, 2022, 6:17pm

Thank you very much for detailed explanation !

Michael-F-Bryan · October 22, 2022, 6:21pm

It's very common to define a trait which will be implemented on foreign types to facilitate certain behaviour.

Think of it like how other languages let you overload functions, letting you invoke different implementations depending on what type of object you pass to it. Rust has a much stronger type system than JavaScript, so there is no ambiguity or way to accidentally shoot yourself in the foot.

Anachoreta · October 22, 2022, 6:31pm

After I implemented the trait and imported it into file where my macros is I still got compile error like:

method not found in `u32`

But on play-rust all works fine. Could you advice what can be missed ?

My macros uses #[macro_export] attribute.

H2CO3 · October 22, 2022, 6:33pm

Please share actual code. It's very likely that you are not in fact use-ing the trait, or maybe you forgot to implement it for u32. If the type implements the trait, and the trait is in scope, you definitely can call it.

Anachoreta · October 22, 2022, 6:35pm

This is my traits.rs:

// ...
pub trait ConvertToBinary {
    fn convert_to_binary(&mut self, buffer: &mut Vec<u8>) -> Result<(), Error>;
}

impl ConvertToBinary for u8 {
    fn convert_to_binary(&mut self, buffer: &mut Vec<u8>) -> Result<(), Error> {
        buffer.write_u8(*self)
    }
}

impl ConvertToBinary for u16 {
    fn convert_to_binary(&mut self, buffer: &mut Vec<u8>) -> Result<(), Error> {
        buffer.write_u16::<LittleEndian>(*self)
    }
}

impl ConvertToBinary for u32 {
    fn convert_to_binary(&mut self, buffer: &mut Vec<u8>) -> Result<(), Error> {
        buffer.write_u32::<LittleEndian>(*self)
    }
}

impl ConvertToBinary for u64 {
    fn convert_to_binary(&mut self, buffer: &mut Vec<u8>) -> Result<(), Error> {
        buffer.write_u64::<LittleEndian>(*self)
    }
}

impl ConvertToBinary for String {
    fn convert_to_binary(&mut self, buffer: &mut Vec<u8>) -> Result<(), Error> {
        buffer.write_all(&self.to_string().into_bytes())
    }
}

This is my macro file:

use crate::types::traits::ConvertToBinary;

#[macro_export]
macro_rules! packet {
    // first rule
    (
        properties {
            $(opcode $opcode_value:expr;)?
        }

        $(#[$outer:meta])*
        $vis:vis struct $PacketStruct:ident {
            $($field_name:ident: $field_type:ty),*$(,)?
        }

        $($PacketStructImpl: item)*
    ) => {
        $(#[$outer])*
        #[derive(Clone, PartialEq, Debug)]
        $vis struct $PacketStruct {
            $($field_name: $field_type),*
        }

        $($PacketStructImpl)*

        $(
            impl $PacketStruct {
                pub fn get_opcode() -> u32 {
                    $opcode_value as u32
                }
            }
        )?

        impl $PacketStruct {
            pub fn to_binary(&mut self) -> Vec<u8> {
                let mut packet = Vec::new();
                $(
                    self.$field_name.convert_to_binary(&mut packet);
                )*;
                packet
            }
        }
    };
    // eof first rule
    // second rule
    (
        properties {
            opcode $opcode_value:expr;
            $($prop_name:ident: $prop_type:ty;)*
        }

        $(#[$outer:meta])*
        $vis:vis struct $PacketStruct:ident {
            $($field_name:ident: $field_type:ty),*$(,)?
        }

        $($PacketStructImpl: item)*
    ) => {
        packet! {
            properties {
                $($prop_name: $prop_type;)*
            }
            $(#[$outer])*
            $vis struct $PacketStruct {
                $($field_name: $field_type),*
            }

            $($PacketStructImpl)*

            impl $PacketStruct {
                pub fn get_opcode() -> u32 {
                    $opcode_value as u32
                }
            }
        }
    };
    // eof of second rule
    // third rule
    (
        properties {
            $($prop_name:ident: $prop_type:ty;)*
        }

        $(#[$outer:meta])*
        $vis:vis struct $PacketStruct:ident {
            $($field_name:ident: $field_type:ty),*$(,)?
        }

        $($PacketStructImpl: item)*
    ) => {
        $(#[$outer])*
        #[derive(Clone, PartialEq, Debug)]
        $vis struct $PacketStruct {
            $($field_name: $field_type),*
        }

        $($PacketStructImpl)*

        impl $PacketStruct {
            pub fn to_binary(&mut self) -> Vec<u8> {
                let mut packet = Vec::new();
                $(
                    self.$field_name.convert_to_binary(&mut packet);
                )*;
                packet
            }
        }
    };
}

Anachoreta · October 22, 2022, 6:38pm

I think I got what's happened. Seems like I need to bring the trait into each file where I want to use the macro. Could you tell does it possible to bring trait globally into scope ? To avoid import it everywhere. I expected that I can import it only in macro file.

H2CO3 · October 22, 2022, 6:40pm

I cannot reproduce the error with that setup.

H2CO3 · October 22, 2022, 6:42pm

You can use UFCS to explicitly call the trait by its name without importing it, which is incidentally what most macros do for this exact reason:

pub fn to_binary(&mut self) -> Vec<u8> {
    let mut packet = Vec::new();
    $(  
        path::to::MyTrait::convert_to_binary(&self.$field_name, &mut packet);
    )*;
    packet
}

system · January 20, 2023, 6:42pm

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.