Most efficient way to cache result of method in struct using borrowed self

What is the most efficient way of cacheing a result of a getter for future use while using a borrowed self?

As far as I can tell it can be done only with RwLock and it's this:

struct Foo<T> {
    elem: Vec<T>,
    _sum: RwLock<Option<i32>>,
}

impl<T> Foo<T> {
    fn get_sum(&self) -> i32 {
        {
            let op = *(self._sum.read().unwrap());
            if let Some(s) = op {
                return s;
            }
        }


        let mut op = self._sum.write().unwrap();

        let s = 3;
        *op = Some(s);
        s
    }
}

You're going to need some sort of shared mutability, but not necessarily RwLock:

  • If you don't intend to access Foo from multiple threads, consider RefCell as a way to avoid the semi-expensive atomic RMW operations of the RWLock implementation.
    • They typically cost of the order of ~100 of CPU cycles, which is okay per se but can get expensive if you call them in a tight loop. They also hamper some compiler optimizations.
    • If your result is actually an i32 or a similarly small Copy type, use Cell instead of RefCell as it's both nicer to use and more efficient.
  • If you want to use Foo in multi-threaded code, consider Mutex
    • RWLock has a more complicated locking protocol than Mutex, and its extra overhead is only amortized if you really spend significant time in the reader critical section.
    • If your result is smaller than a machine pointer (as i32 is), consider the corresponding AtomicXyz type as it can be much more efficient and somewhat simpler.
    • If your cache ends up becoming a thread synchronization bottlenecks (as these things have a nasty tendency to), consider using thread-local caches instead as a space/time tradeoff. The simplest way to do it is to use the built-in thread_local primitive from std.

These are your basic options from std, depending on use cases more advanced caching primitives (lock-free hashmaps, RCU...) of which you can find some implementations on crates.io may be even faster, at the cost of extra implementation and/or API complexity.

1 Like

Thank you for your through post. It looks like I should read the whole https://doc.rust-lang.org/book/ch15-00-smart-pointers.html again.