[New crate] dbstruct: derive a typed embedded db

Hi all,

A few days ago I published the first version of dbstruct and I would love some feedback. It is a proc macro that turns an annotated struct into a typed embedded database. You then interact with the database through getters and setters. You can choose how values missing in the database are represented using attributes. Standard library types Vec, HashMap and Option have special getters and setters that mimic (some of) their standard library functionality.

If you like what you see I would love a star on GitHub.

Motivation

When we do not need persistence we usually collect variables in a struct. Key-Value databases provide persistence however they store values as raw bytes, and we access them through a key. One way to make databases easier to work with is to wrap them in a struct. That struct the struct then implement getters and setters that return and take typed variables.

This takes quite some code to set up, not ideal for small projects. We can work around this by extending the database's API to enable type getting and setting:

pub trait TypedDb { // an extension trait
    fn get_val<T: DeserializeOwned>(&self, key: impl AsRef<[u8]>) -> Option<T>;
    fn set_val<T: Serialize>(&self, key: impl AsRef<[u8]>, val: T);

The type we (de-)serialize is determined by the caller of the get_val or set_val function. It is an easy mistake to mix up the types, in the best case this leads to a crash when getting the value. Another possible mistake is using the wrong key. A typo in the key can lead to get_val always returning None.

dbstruct constructs an API that makes it impossible to mix up types, and it determines the keys for you. It can even provide an API similar to standard library types for some fields.

Usage

You define a struct and add the dbstuct attribute. Then you decide for each field that is not an Option, Vec or HashMap how you want to treat missing values.

#[dbstruct::dbstruct(db=sled)]
pub struct TheDatabase {
	#[dbstruct(Default)]
	the_answer: u8,
	primes: Vec<u32>,
	#[dbstruct(Default="format!(\"{}\", 20+2+20)")]
	the_result: String,
}

Now you can use the struct as a database!

fn main() -> Result<(), Box<dyn std::error::Error>>  {
	let db = TheDatabase::new("the_db")?;

	db.the_answer().set(&42)?;
	assert_eq!(42u8, db.the_awnser().get()?);

	db.primes().push(2)?;
	db.primes().push(3)?;
	db.primes().push(5)?;
	db.primes().push(7)?;
	assert_eq!(Some(7), db.primes().pop()?);

	assert_eq!(String::from("42"), db.the_result().get()?);
}

Implementation

dbstruct replaces the fields in your struct with methods. Each method returns a wrapper type. These have methods for setting and getting values. The wrapper can also expose an API similar to a standard library collection type.

The wrappers are generic over the type they provide access to. They are created by the derived methods. These pass the wrappers the key they should use and borrow them the database. The derived methods are not generic and specify the wrappers generics to equal original the field's types.

This is what the derived method for the field the_awnser looks like

    pub fn the_awnser(&self) -> wrapper::DefaultTrait<u8, DS> { // DS is the database's type
        wrapper::DefaultTrait::new(&self.ds, 1) // 1 is the database key
    }

The get function for the DefaultTrait wrapper then looks like this:

    pub fn get(&self) -> Result<T, Error<E>> {
        Ok(self.ds.get(&self.key)?.unwrap_or_default())
    }

Any database that implements the dbstruct::DataStore trait can be used. To make things easier you can select a database in the dbstruct attribute. If you do so we add a new function to the struct. Given a path it will open the database and return the struct.

Future Work

These are some features I am planning to work on, in no particular order. If you have any ideas please let me know via a comment, or open an issue!

  • Example workflow for migrations.
  • (Dis)Allow access from multiple threads cloning the struct
  • Flushing the database, explicitly via a function on the struct and implicitly whenever a field changes. Will be configurable through an attribute on the struct and a field specifically.
  • Expand the wrapper API to more closely match that of their standard library counterparts.
  • Async support for flushing the database.
  • Figure out how to represent transactions (hard if even possible)
2 Likes