Generate an identical struct without one field

help
1

Hi,
I'm using Diesel and currently am stuck in a pattern where I will have a main struct for querying and updating, and then have a separate struct for inserting, which will be identical but without an ID column:

#[derive(Identifiable, Queryable, Associations, AsChangeset)]
#[table_name="customers"]
pub struct Customer {
   id: i32,
   name: String,
   ...
}

#[derive(Insertable)]
#[table_name="customers"]
pub struct NewCustomer {
   name: String,
   ...
}

And then when I want to insert a new customer, I need to take the customer struct, and copy everything over to the new customer struct. I understand that Diesel was designed on purpose this way, but it seems possible to have a macro that can auto-generate the new struct without the id field using declarative or proc macros somehow.

It would be great to just do:

pub fn create_customer(customer: &Customer) -> Result<usize, diesel::result::Error> {
   // Here is where a new struct would be defined, instantiated,
   // and all values would be copied over
   create_new_struct!(customer);

   // Do actual insertion of generated new_customer
   diesel::insert_into(customers::table)
        .values(&new_customer)
        .execute(connection)
}

Unfortunately I'm pretty trash at macros right now, so I don't know if this is possible or if their's some reason it can't work. Is this doable?

1 Like
2

To reduce boilerplate, you could do something like e.g.

macro_rules! diesel_struct_insertable {
    (
        #[derive($($path:path),+)]
        $(#[$attr:meta])*
        $vis:vis struct $Name:ident {
        $($fields:tt)*
    }) => {
        ::paste::paste! {
            #[derive($($path),+)]
            $(#[$attr])*
            $vis struct $Name {
                id: i32,
                $($fields)*
            }
            #[derive(::diesel::prelude::Insertable)]
            $(#[$attr])*
            $vis struct [<New $Name>] {
                $($fields)*
            }
        }
    }
}

diesel_struct_insertable! {
    #[derive(Identifiable, Queryable, Associations, AsChangeset)]
    #[table_name = "customers"]
    pub struct Customer {
       name: String,
       ...
    }
}

(note that I’ve never used Diesel; mostly untested code above; uses/requires the paste crate)

3

Ormx does this. It uses sqlx, not diesel, but it has good ideas about how one can write such macros.

Ormx has an insertable attribute. If you apply it to a struct Foo that has an ID field, ormx will generate a struct InsertFoo that does not have an ID field.

Ormx is a layer on top of sqlx, and it currently supports postgres and mysql. It's not a full blown ORM framework like diesel, but rather is lightweight, focusing on the most common patterns such as: reading a record by id, inserting a record, reading all the records.

1 Like
4

This is a fantastic first step, thank you!

1 Like
5

For anyone wondering how I fully solved it:

#[macro_export]
macro_rules! diesel_struct_insertable {
    (
        #[derive($($path:path),+)]
        $(#[$attr:meta])*
        $vis:vis struct $Name:ident {
            $($field_vis:vis $field_name:ident: $field_type:ty,)*
        }
    ) => {
        ::paste::paste! {
            #[derive(Identifiable, $($path),+)]
            $(#[$attr])*
            $vis struct $Name {
                pub id: i32,
                $($field_vis $field_name: $field_type,)*
            }
            #[derive(::diesel::prelude::Insertable)]
            $(#[$attr])*
            pub(crate) struct [<New $Name>]<'a> {
                $($field_vis $field_name: &'a $field_type,)*
            }

            impl <'a>[<New $Name>]<'a> {
                pub fn from_original(original: &'a $Name) -> Self {
                    [<New $Name>] {
                        $(
                            $field_name: &original.$field_name,
                        )*
                    }
                }
            }
        }
    }
}

When used like this:

diesel_struct_insertable! {
   #[table_name="customers"]
   pub struct Customer {
      name: String,
      ...
   }
}

It generates the actual struct Customer with all fields and an ID, as well as an insertable struct called NewCustomer which derives Insertable as well as implements a function from_original that takes in a reference to the original struct and instantiates a new struct holding references to all its fields. This removes the need to manually copy each field from the original, and it doesn't even copy anything at all, only uses references!

1 Like
6

It seems like the "need" for a additional struct for insert is coming from a general misunderstanding of how inserts in diesel work. Let me try to clarify that (and as I see this question reocurring again and again, please tell me what else should be added to the documentation to make this clearer):

First of all there it is definitively not required to have a separate struct for inserts at all. Let me cite the "All about inserts" guide here (emphasis added by me):

Working with tuples is the typical way to do an insert if you just have some values that you want to stick in the database. But what if your data is coming from another source, like a web form deserialized by Serde? It’d be annoying to have to write (name.eq(user.name), hair_color.eq(user.hair_color)).

Diesel provides the Insertable trait for this case. Insertable maps your struct to columns in the database. We can derive this automatically by adding #[derive(Insertable)] to our type.

That means instead of going through all the trouble to create a new struct just for inserts you could just reference the corresponding fields from the original struct directly:

diesel::insert_into(customers::table)
    .values((customers::name.eq(&customer.name), …))
    .execute(connection)

Now one could argue that it should be possible to just insert the customer struct directly. The answer here is: That's definitively possible, as long as that type implements Insertable. Any type can implement Insertable by manually implementing it, which allows to do whatever you want. For structs it's also possible to use the derive, which will then just insert all fields.

Now using #[derive(Insertable)] on Customer would insert all fields including the id field into the database. It seems like you don't want to have this behaviour. In that case there is always to option to manually implement the trait, just like that's the case for #[derive(Debug)] as well.

Which brings be to the reasoning behind this behaviour: In rust every field must have a value. That means if you insert a new customer you need to set the id field to something to populate the struct. Now you could just set it to some invalid value and ignore that value on insert. This makes it very easy to mix up structs coming from loads and those that are desalinates for inserts. So having this split here just enforces type safety just like you cannot mix String and Path easily. Now one could argue that we could just use Option<i32> there to communicate the absence of a valid ID. That's true for the insert case, but would in turn mean that as soon as you need to access the id of a loaded entity you need to call .unwrap() or a similar function as you basically know that the id must be there. We as diesel developer feel that this is a bad solution so we've opted for not implementing it for Option<i32>. That does not mean that you cannot write your own enum Id { Set(i32), Unset} enum to be used in that location. It just requires implementing a few traits (Insertable, FromSql, Queryable, and FromSqlRow (the last two can be derived via #[derive(FromSqlRow))

For the sake of completeness here is an manually impl of Insertable that would skip the corresponding field. Knowing which field to skip is user knowledge so nothing the derive could do automatically:

use diesel::prelude::*;
use diesel::dsl;

impl<'a> Insertable<customers::table> for &'a Customer {
    type Values = (dsl::Eq<customers::name, &'a str>, …);

    fn values(self) -> Self::Values {
        (customers::name.eq(&self.name), …)
    }
}
7

I understand why Diesel is desgined this way, but I want a system where I can simply change the fields in Customer (which I will do often) and not have to worry about changing any other insertable code anywhere. I'm not saying its a perfect or general solution to this problem (of course it only works when your ID field is named "id").

See how each of the solutions you listed out requires you to write out each and every field in the struct? (The "..." in your examples). This is what I'm trying to avoid, because I've had experiences when I add new fields and forget to copy them to the insert code, and I get no compile time errors or warnings because the code is still "correct", but it ends up not actually saving that field to the database, and silently failing. That's why I wanted automatically generated insert code.

8

See how each of the solutions you listed out requires you to write out each and every field in the struct?

That's not true at least for this variant

In this case you don't need to think about updating things at all, as there are only one struct.

Also I believe the custom Insertable implementation may work as well here, as long as you just generate that impl automatically (either by using a custom derive macro, or even by a macro by example implementation similar to yours).

That written there was existing discussion at the diesel issue tracker about adding a attribute that allows to skip fields for the #[derive(Insertable)] and the #[derive(AsChangeset)] attribute. This feature would allow just use the existing derive for that use case. At least for me that's not something that has currently priority, but I would likely accept a PR there adding support for this as long as it includes some tests + documentation.

This is what I'm trying to avoid, because I've had experiences when I add new fields and forget to copy them to the insert code, and I get no compile time errors or warnings because the code is still "correct", but it ends up not actually saving that field to the database, and silently failing. That's why I wanted automatically generated insert code.

That's because it's technically correct to have insert request that do not set all fields. It's even often the thing you want to have. (That written I totally understand that this could be a problem for fast moving code, but again that's something the user (out of diesels perspective) needs actively decide so all defaults will feel wrong here for someone. We've just chosen that default that allows all other variants to coexist.)

9

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.