I need a cache with "time-to-refresh" semantics

I have some data which I access over a (slow) API call. I need to access this data repeatedly, so naturally I'd like to use a cache to speed things up. However, the data at the source can change, and if that happens I'd like my program to react fairly quickly and start using the new data (say within a minute).

The most feature rich async cache implementation in Rust seems to be from the moka crate. This has some nice semantics - time-to-live, time-to-idle and eviction listeners (but no time-to-refresh).

The simple solution would be to give the cache entries a TTL of one minute. The main problem here is that, well, each cache entry only lasts a minute. Ideally, I'd like to keep the entry in the cache until it's evicted due to capacity issues, and keep it more-or-less in sync with the original data source until then.

A combination of TTL and eviction listeners also doesn't work because Moka's eviction listeners are called after the entry is already removed. The eviction listener would have to keep inserting the entry back into the cache, which could confuse the algorithm which decides on the entries to evict when space is low.

(Worth noting that quick_cache has a before_evict hook which get called before the entry is removed, but at that point the only way to prevent entry removal is to set the item weight to zero, which prevents it from ever being removed)

Another solution would be to spawn an asynchronous task for each entry in the cache, which fires every minute and keeps inserting the updated value back into the cache. This is again undesirable because it messes with the cache's internal popularity metrics, affecting how and when entries are evicted.

Is there a way to get the kind of functionality I want out of existing libraries? I'd rather avoid writing my own cache implementation, which would (1) be a lot of work and (2) much less efficient than existing libraries.

What do you mean exactly for "more or less". Before that you say:

A TTL of 1 minute is exactly what you want: each entry is kept in the cache for 1 minute and refreshed on next access after that. It seems that you want to keep entries in the cache and refresh them every minute even if no other code accesses them. How is that different from refreshing on first access after the TTL has expired?

For one thing that would mean it's already in cache up to date, so there's no latency. But I'm not sure exactly what OP is meaning here either; that would have to be done pretty carefully to avoid pointlessly hammering the origin.

So there are two metrics I need to juggle here: cache latency and staleness of entries. By modifying the TTL I can manage the trade-off between the two. A lower TTL is better for staleness, but reduces the cache performance by increasing the number of cache misses, while a higher TTL is better for performance, but gives increasingly outdated results.

If I need to have both, the best option seems to be a periodic refresh of the cache entries. This gives me the best of both worlds. As @simonbuchan pointed out, the cost of this is that the volume of requests to the origin increases significantly, but in my case this is an acceptable trade-off as long as neither latency nor maximum staleness are compromised.

Again, this boils down to avoiding the extra latency. For example, if I have a TTL of 1 min, trying to access the entry after 2 min would incur the latency cost of an origin lookup. But, if I have a TTR of 1 min, while having a TTL of 5 min, I can wait up to 5 mins between lookups without incurring a latency overhead, while also making sure the result I get is at most a minute old.

So, you need to keep the cache "primed" until an entry is evicted by its TTL expiring. I never needed this but I guess that unless you find a library that cover this case you will need to set the TTL to the higher value (5m for example) and refresh unexpired items using a timer. What complicates things here is that you want to find the items already refreshed on access. If it is acceptable to refresh on access, then uou can just write a translation layer that checks the underlying cache and refresh when needed.