Create a temporary in-memory db to store gRPC request?

I am going all brute-force learning by trial-and-error with two things: rust and gRPC.
I am trying to understand what gRPC is by reading the gRPC: Up and Running Book and trying to implement the examples in the book (written in go and java ) in Rust using tonic.

TBH, it is taking time but I am really getting hold of what it means to write apps in Rust.

I am in a bit stuck at this point though:

I have written a gRPC server where a user provides: id , name , description of a Product and to a addProduct service, where I replace the id of the incoming request of the product with a UUID v4 and respond back with the updated UUID of the product.

Code Gist

https://gist.github.com/shantanoo-desai/59fb65362b819dd99611d249ccd95666

This is working great and I can actually get a UUID back as a response

I wish to however write a get_product implementation for the ProductInfo service where I wish to stored the added product via add_product temporarily and use the saved UUID to get the product information back.

I wish to know what is the simplest Beginner's way to do this for this example, without any external crates.

I would define a Db struct like the one below:

use uuid::Uuid;
use std::sync::{Arc, Mutex};
use std::collections::HashMap;

#[derive(Clone)]
pub struct Product {
    name: String,
    desc: String,
}

#[derive(Clone)]
pub struct Db {
    shared: Arc<Shared>,
}

struct Shared {
    products: Mutex<HashMap<Uuid, Product>>,
}

impl Db {
    pub fn new() -> Self {
        Self {
            shared: Arc::new(Shared {
                products: Mutex::new(HashMap::new()),
            })
        }
    }
    pub fn insert(&self, key: Uuid, prod: Product) {
        let mut products = self.shared.products.lock().unwrap();
        products.insert(key, prod);
    }
    pub fn get(&self, key: Uuid) -> Option<Product> {
        let products = self.shared.products.lock().unwrap();
        match products.get(&key) {
            Some(product) => Some(product.clone()),
            None => None,
        }
    }
}

playground

Note the following details:

  • The Db struct can be cloned. Due to the use of Arc, this results in a separate handle to the same HashMap, meaning that cloning the Db is a way to share the hash map. Inserts to any clone are visible from all clones.
  • All the methods take &self. This means that you can store the Db in your MyProductInfo struct and use both methods even though the add_product method in your code takes &self instead of &mut self.

For more details, please see:

  • Shared state - a chapter on sharing data in the Tokio tutorial
  • Async: What is blocking? - a blog post that talks about blocking, and at the end it notes why it is ok to use a blocking mutex in the example I provided you

One important detail is that the methods on the Db struct are not async. This is because I am following the pattern described under the heading "Restructure your code to not hold the lock across an .await" in the shared state chapter I linked above.

1 Like

How come this works without requiring ‘&mut self’? Doesn’t this method mutate the Db?

Generally the real difference between & and &mut references is not mutability, but that an & allows shared access, whereas &mut can only be called if you have exclusive access to the struct. It's just that shared/exclusive corresponds very closely to immutable/mutable in most cases since mutation of a shared variable can cause all sorts of problems.

So the answer is that, yes, it does mutate the Db, but it is allowed because I use a Mutex to guarantee that, even if there are many simultaneous calls to Db::insert, only one at the time can get past the lock(), so whenever one of them calls insert, they do have exclusive access to the hash map, as all other callers will be waiting for us to release the lock before they can access the map.

1 Like

Thanks for this. I am a bit confused as to why is there another Product struct needed here. I think the same name is causing namespace conflicts for me since the generated Protocol Buff file also provides a public struct Product.

Also another question is if I instantiate a Db in one of the method e.g. async fn add_product() like the following:


      let mut in_mem_db = Db::new();

will in_mem_db also be available for search in the async fn get_product() method also?

Feel free to combine them into one struct. I'm just trying to show you the general structure you are looking for.

You are going to need to store the Db somewhere where both functions have access to it. The easiest is probably as a field in your MyProductInfo struct, but you could also try to put it in a global variable. (You will need the lazy_static crate for the global solution, since Db::new is not a const fn.)

Thanks for the patience in replying. After literally trying different things out I got the Server Code running thanks to your tips. As you mentioned refactored the Product struct in to pub struct InternalProduct and added the Db as a member of the MyProductInfo.

It took me so much time to figure out how I can keep on using the incoming request to move and alter the data to be stored in the temporary in-memory-db and learnt a valuable lesson on Clone attribute.

The Refactored code is as follows:

https://gist.github.com/shantanoo-desai/59fb65362b819dd99611d249ccd95666#file-server-rs

I have terribly exploited .clone() whenever I could not access the data but this got the work done. Thanks for the helpful replies.

I think that the only clones you should be able to get rid of are the two places where you write found_product.1.clone(). You can do that like this:

async fn get_product(&self, request: Request<ProductId>) -> Result<Response<Product>, Status> {
    println!("Received Request: {:?}", request.remote_addr());

    let search_result = self.product_map.get(request.into_inner().value);
    match search_result {
        Some((id, product)) => {
            Ok(
                Response::new(
                    ecommerce::Product {
                        id,
                        name: product.name,
                        description: product.description,
                    }
                )
            )
        },
        None => { Err(Status::not_found("Product Not Found")) }
    }
}

The rest of the clones are necessary.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.