Async refreshing cache using Tokio: help

help
#1

Hey there!
I think I am doing something basically wrong here. I am designing a TTL cache that has an internal async updater that listens for stale keys it needs to update. This requires me to have something like:

pub struct Cache<K: 'static + Send, V: 'static + Send> {
    data: RwLock<HashMap<K, V>>,
    updates_tx: flume::Sender<WriteOp<K, V>>,
}

full code here.

implementing get (still logically wrong but I wan the compiler to handle it) is not compiling:

pub fn get<Q>(&self, k: &Q) -> Option<&V>
    where
        K: Borrow<Q>,
        Q: Hash + Eq,
    {
        let map = self.data.read();
        let res = map.get(k);
        res
    }

error[E0515]: cannot return value referencing local variable map

Now I understand that the internal map is borrowed by get but actually, what I want to express here, is that map is just a temporary value and the real owner is self. The fact that the map owns its data and that get should return a reference to the reader is fine.
Maybe something is basically wrong in my design here?

#2

Well, the problem is with the RwLock. You can’t return a reference to something protected by RwLock without holding on to the lock guard. There’s two options that come to mind: You could try returning something that contains the lock next to the &V reference, a kind-of “self-referential” struct (though that’s not trivial), or you could not make get return a value but instead just pass it to a call-back that will live higher up on the stack.

Another option would be to use something like RwLock<HashMap<K, Arc<V>>> and hand out a clone of the Arc.

What’s best probably also depends on your use-case.

#3

Just found this discussion which is similar to my question above.
So, you are saying the same as @cuviper:

Oh, but you also want to return the borrowed value without holding the lock? Rust doesn’t allow this, at least not in safe code.

another valid option I see here is to return a clone of the value:
So:
defining the impl with:

V: 'static + Send + Clone,

and then change the API of get to:

pub fn get<Q>(&self, k: &Q) -> Option<V>
where
    K: Borrow<Q>,
    Q: Hash + Eq,
{
    let map = self.data.read();
    let res = map.get(k);
    res.map(|v| v.clone())
}

now compiles. WDYT?

#4

Sure, that might be an approach. Kind-of a generalization of the Arc<V>. I don’t know anything about TTL caches, so I don’t know what the indended use-case/pattern is for this, what the constraints are, etc. I’m really not able to say what the best approach here is ;-)

#5

For some minor improvement, check out the Option::cloned().

pub fn get<Q>(&self, k: &Q) -> Option<V>
where
    K: Borrow<Q>,
    Q: Hash + Eq,
{
    self.data.read().get(k).cloned()
}
1 Like
#6

Well, it is not related to the cache eviction policy at all (TTL), but rather accessing a map that is behind a Rwlock and the implications of obtaining a reference/clone after acquiring the value. So as you observed (the borrow checker also did :slight_smile: ): the reference to the value cannot outlive the read lock. So either you force the user to handle the reference inside the lock (pass a closure/ return the lock) or you give her a clone of it.

#7

Cloning it is one good idea. Another is to ask yourself, what operations do you need to perform? Could you define those operations as real methods on the struct, locking the RwLock inside those methods, performing the operation, then unlocking it once you no longer need the value?

1 Like
#8

Totally. This might grow to a library one day, so my intention currently is to have this cloning API. The goal is to remain as general as possible.
Later, when needed I can add additional APIs that perhaps pass clojure.