Having trouble caching database data

help
1

I have a database I'm deserializing a MyStruct from a "my_structs" table, which are basically user-generated configurations for things. Each MyStruct has an ID and some other settings.

The problem is, I am using Tokio and async / await and have some Futures I want to use this in. If I ever receive a MyStructTask at some point in time, which includes an ID of a MyStruct to use for the task, I want to get the config from the database by that ID, deserialize it into a MyStruct, and then cache it so I don't have to repeatedly make database calls and deserialize them.

I've been trying to get this working with lazy_static!, but I haven't been able to get it to compile. If I have a:

static ref MY_STRUCTS: Mutex<HashMap<'static str, MyStruct>> = {

	let mut my_structs = HashMap::new();
	let default = MyStruct::new();
	
	my_structs.insert("default", default);

	Mutex::new(my_structs)
    };

it yells at me saying it can't be sent across threads safely. I thought since it is static ref that I don't need something like an Arc. Is that not true?

What is weird is, if I change it to return Arc<Mutex<HashMap<'static str, MyStruct>>> it will still yell at me saying it isn't Send.

Any pointers on what I'm doing wrong would be appreciated.

2

The compiler should explain why exactly this is not Sync. My guess is that MyStruct contains something like Cell, which makes it not thread safe.

3

Cell would be fine because of the Mutex, it looks like MyStruct has something which is not Send in it, like Rc. If you can't use lazy_static, then try thread_local,

thread_local! {
    static MY_STRUCTS: RefCell<HashMap<>&'static str, MyStruct> = ...;
}
1 Like
4

It turned out the issue was something further into my code. I'm calling task::spawn and have some code in an async move block. Within that, I'm doing this:

                let my_structs = Arc::clone(&*MY_STRUCTS);
                let my_structs= my_structs.read().unwrap();

		match my_structs.get(my_struct_id) {
		    Some(s) => {
			let my_struct = s.my_struct(content.as_bytes());
			// if let Err(e) = my_struct.update_doc(&self.index, &self.doc_id).await {
			//     error!(
			//         "Couldn't update document: {:#?}",
			//         e
			//     );
			// };
			Ok(())
		    }
		    None => Err(format!("MyStruct with ID \"{}\" does not exist.", my_struct_id))
		}

If I uncomment that, it yells at me with:

error[E0277]: `std::sync::RwLockReadGuard<'_, std::collections::HashMap<&str, MyStruct >>` cannot be sent between threads safely
   --> src\queue.rs:110:13
    |
110 |             task::spawn(async move {
    |             ^^^^^^^^^^^ `std::sync::RwLockReadGuard<'_, std::collections::HashMap<&str, MyStruct >>` cannot be sent between threads saf
ely
    |
   ::: C:\Users\me\.cargo\registry\src\github.com-1ecc6299db9ec823\tokio-0.2.6\src\task\spawn.rs:123:21
    |
123 |         T: Future + Send + 'static,
    |                     ---- required by this bound in `tokio::task::spawn::spawn`
    |
    = help: within `impl core::future::future::Future`, the trait `std::marker::Send` is not implemented for `std::sync::RwLockReadGuard<'_,
 std::collections::HashMap<&str, MyStruct >>`

I'm confused about that though since I thought putting something in an Arc makes it thread safe.

BTW, I changed the Mutex to a RwLock

5

You can't keep lock guards across await points.

6

How should I handle this then? I thought Arc would handle making it thread safe and that RwLock would handle the sync stuff so that only one thread could mutate it at a time?

7

I didn't want to share the lock guard across the await point. I just need to get an item from its inner HashMap, use that to create a MyStructResult, and then pass that across an await point.

8

Drop the guards right before you await then reacquire them after the await. That way you don't hold the same lock across an await point. You could clone the data inside, that way you don't have to borrow the guard across await points.

Note: Arc doesn't make anything but the reference count thread safe. RwLock us responsible for making you data thread safe. But, the guards that the RwLock gives you are not thread safe, they must be dropped on the same thread they were created on due to platform specific requirements.

9

I couldn't figure out how to drop it normally, and even after calling std::mem::drop(my_structs) on the guard, it had the same error message. Thanks for your help though!

I ended up seeing that Tokio 2.9 includes a tokio::sync::RwLock that works perfectly for this, so that solved it.

10

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.