Macro to call a function on each field of a struct

[RickRhops]

I'm learning Rust trying to make something like an ECS (Entity Component System)

I have a struct of multiple HashMaps for each component and I need to call HashMap::new(), remove_entry() and insert() on each field:

struct Components {
    component_a: HashMap<EntityId, DataA>,
    component_b: HashMap<EntityId, DataB>,
    component_c: HashMap<EntityId, DataC>,
    component_d: HashMap<EntityId, DataD>,
    component_e: HashMap<EntityId, DataE>,
}

impl Components {
    fn new() -> Self {
        Self {
            component_a: HashMap::new(),
            component_b: HashMap::new(),
            component_c: HashMap::new(),
            component_d: HashMap::new(),
            component_e: HashMap::new(),
        }
    }

    fn remove_entity(&mut self, entity: EntityId) {
        self.component_a.remove_entry(&entity);
        self.component_b.remove_entry(&entity);
        self.component_c.remove_entry(&entity);
        self.component_d.remove_entry(&entity);
        self.component_e.remove_entry(&entity);
    }

    fn add_entity(&mut self, entity: EntityBuilder) {
        match entity.component_a {
            Some(data) => self.component_a.insert(entity.entity_id, data),
            None => None,
        };
        match entity.component_b {
            Some(data) => self.component_b.insert(entity.entity_id, data),
            None => None,
        };
        match entity.component_c {
            Some(data) => self.component_c.insert(entity.entity_id, data),
            None => None,
        };
        match entity.component_d {
            Some(data) => self.component_d.insert(entity.entity_id, data),
            None => None,
        };
        match entity.component_e {
            Some(data) => self.component_e.insert(entity.entity_id, data),
            None => None,
        };
    }
}

This can get out of hand pretty quickly as the number of components scale, so my questiion is:

How can I call a function on every field of the struct?

I'd like it to be something like this: call_function!(remove_entry(&entity))) and call_function!(insert(&entity))) is this any good?

[2e71828]

Barring writing your own derive macro, there’s no way to automatically iterate over the fields of a struct. But you can hardcode those field names into a macro to help with code duplication. Unfortunately, it often results in a bad tradeoff between length and understandability. That said, here’s what I came up with:

macro_rules! map_components {
    ($($tok:tt)*) => {
        map_field!{component_a $($tok)*}
        map_field!{component_b $($tok)*}
        map_field!{component_c $($tok)*}
        map_field!{component_d $($tok)*}
        map_field!{component_e $($tok)*}
    }
}

macro_rules! map_field {
    ($field:ident $(($($spec:tt)*))* $body:block ) => {
            {
                $(_let_field!{$field => $($spec)*})*
                $body
            };
    } 
}

macro_rules! _let_field {
    ($field:ident => $var:ident : _ <= $expr:expr) => {
        let $var = $expr.$field;
    };
 
    ($field:ident => $var:ident : &mut _ <= $expr:expr) => {
        let $var = &mut $expr.$field;
    };

    ($field:ident => $var:ident : & _ <= $expr:expr) => {
        let $var = & $expr.$field;
    };
}

// …

impl Components {
    fn new() -> Self {
        Self::default()
    }

    fn remove_entity(&mut self, entity: EntityId) {
        map_components!( (map: &mut _ <= self) {
            map.remove_entry(&entity)
        } );
    }

    fn add_entity(&mut self, entity: EntityBuilder) {
        map_components!( (map: &mut _ <= self) (from_entity: _ <= entity) {
            match from_entity {
                Some(data) => map.insert(entity.entity_id, data),
                None => None,
            };
        });
    }
}

Rust Playground

[nerditation]

@RickRhops

I'd like it to be something like this: call_function!(remove_entry(&entity))) and call_function!(insert(&entity)))

both uses are possible, but it's not easy if you want one single macro to accept these exact syntax, and it'll be really hard to come up with a general syntax that can extend to future uses.

an intermediate solution is to use two level of macros, one macro to expands the field names, and feed the field name as an identifier to an action callback, then you can define an action macro each time you need to do something with each field name, example:

macro_rules! each_component {
  ($action:path) => {{
    $action!(component_a);
    $action!(component_b);
    $action!(component_c);
    $action!(component_d);
    $action!(component_e);
  }}
}

fn remove_entity(&mut self, entity: EntityId) {
  macro_rules! do_remove {
    ($field:ident) => {  self.$field.remove_entry(&entitiy)  }
  }
  each_component!(do_remove);
}

fn add_entity(&mut self, entity: EntityBuilder) {
  // I replaced `match` with `if-let` for brevity.
  macro_rules! do_add {
    ($field:ident) => {
      if let Some(data) = entitiy.$field {
        self.$field.insert(entity.entity_id, data);
      }
    }
  }
  each_component!(do_insert);
} 

for illustrate purpose, the each_component macro in this example simply expands to a sequence of statements, but it can do more complicated things too. and this macro can in turn be auto generated using a procedural macro if you need to.

[RickRhops]

This is better than what I have for sure,

But this gave me another idea, you used #[derive(Default)] on the Components struct, and then on

fn new() -> Self { Self::default() }

Could I make a Component trait that I can do #[derive(Default, Component)] and do someting like this?

#[derive(Default, Component)]
struct Components {
    component_a: HashMap<EntityId, DataA>,
    component_b: HashMap<EntityId, DataB>,
    component_c: HashMap<EntityId, DataC>,
    component_d: HashMap<EntityId, DataD>,
    component_e: HashMap<EntityId, DataE>,
}

impl Components {
    fn new() -> Self { Self::default() }
    fn remove_entity(&mut self, entity: EntityId) -> Self { Self::remove_entry(entity) }
    fn add_entity(&mut self, entity: EntityId) -> Self { Self::insert(entity) }
}

What if I changed the Components struct to another HashMap or something else, would that help with anything?

[RickRhops]

@nerditation

an intermediate solution is to use two level of macros, one macro to expands the field names, and feed the field name as an identifier to an action callback, then you can define an action macro each time you need to do something with each field name, example:

Thats a pretty great idea! Only having to write the components once helps a lot. Thanks for the reply!

@nerditation

for illustrate purpose, the each_component macro in this example simply expands to a sequence of statements, but it can do more complicated things too. and this macro can in turn be auto generated using a procedural macro if you need to.

Wait, so I can make a macro that generates the Components struct itself?

[2e71828]

@RickRhops

Could I make a Component trait that I can do #[derive(Default, Component)] and do someting like this?

Yes, but it’s not a particularly straightforward thing to accomplish.

@RickRhops

What if I changed the Components struct to another HashMap or something else, would that help with anything?

The tricky part is in figuring out how you want to deal with the fact that each component type wants a different payload type, which then need to be unified somehow if you want to store everything in a single collection. You could explore a design like this, but I’m not sure how the ergonomics will work out.

enum ComponentData {
    A(DataA),
    B(DataB),
    // …
}

impl From<DataA> for ComponentData { … }
impl TryFrom<ComponentData> for DataA { … }
// …

struct Components {
    components: HashMap<
        std::mem::Discriminant<ComponentData>,
        HashMap<EntityId, ComponentData>
    >
}

---

Edit: Another option would be something like this:

trait EntityMap {
    fn remove(&mut self, key:EntityId);
    fn try_downcast_mut<V:'static> (&mut self)->Option<&mut HashMap<EntityId, V>>;
}

impl<T:'static> EntityMap for HashMap<EntityId, T> {
    fn remove(…){…}
    fn try_downcast_mut<V:'static> (&mut self)->Option<&mut HashMap<EntityId, V>> {
        (self as &mut dyn Any).downcast_mut()
    }
}

struct Components ( HashMap<TypeId, Box<dyn EntityMap>>);

[RickRhops]

@2e71828

Yes, but it’s not a particularly straightforward thing to accomplish.

So it is possible then! I'll try seaching the docs on how the Default derive is implemented to get some pointers

edit: the implementation of the Default derive is pretty underwhelming...

/// Derive macro generating an impl of the trait `Default`.
#[rustc_builtin_macro(Default, attributes(default))]
#[stable(feature = "builtin_macro_prelude", since = "1.38.0")]
#[allow_internal_unstable(core_intrinsics)]
pub macro Default($item:item) {
    /* compiler built-in */
}

@2e71828

You could explore a design like this, but I’m not sure how the ergonomics will work out:

I'll try using a HashMap and see how it goes, one thing is that I'd be able to just do an Iteration on every component and call the function, no need for macros. But I'll see what I can do...

You sure helped me with some new ideas, thanks a lot for the reply!

[RickRhops]

@2e71828

Edit: Another option would be something like this:

trait EntityMap {
    fn remove(&mut self, key:EntityId);
    fn try_downcast_mut<V:'static> (&mut self)->Option<&mut HashMap<EntityId, V>>;
}

impl<T:'static> EntityMap for HashMap<EntityId, T> {
    fn remove(…){…}
    fn try_downcast_mut<V:'static> (&mut self)->Option<&mut HashMap<EntityId, V>> {
        (self as &mut dyn Any).downcast_mut()
    }
}

struct Components ( HashMap<TypeId, Box<dyn EntityMap>>);

That seems interesting! I'm not good enough in Rust to understand it quite yet, but I get the idea. The HashMap with a TypeId as key is something I'll try to use for sure!

[RickRhops]

I realized the replies are straying further from the main topic of "macros over struct fields",

So, for people looking for an answer to this specific problem, I'll mark the first answer by @2e71828 as the Solution since it uses macros.

I'll make a new topic to continue the discussion there.

Thanks @2e71828 and @nerditation for the replies!