I'm curious about the operation logic of refreshAfterWrite options

[jojaeng2]

Hello,

I'm curious about how the refreshAfterWrite option in the Caffeine builder works.
Based on the documentation, here's what I understand:

When using refreshAfterWrite, the local cache's TTL expiration does not immediately remove the entry.
If a request is made for a value after its TTL has expired, the current information in the local cache is returned.
An Executor asynchronously runs the mapping function to refresh the cache.
I've been checking the code in the IDE to see if my understanding is correct, but I haven't been able to locate where this logic is implemented. I've reviewed the code from UnboundedLocalAsyncLoadingCache and LocalAsyncCache. Eventually, I found that the get method in LocalAsyncCache executes computeIfAbsent in UnboundedLocalCache, but I don't see any logic related to refreshAfterWrite.

So, I am guessing that the refreshAfterWrite logic is handled by the Executor. If that's correct, I'm curious about how the Executor handles this logic.

I would appreciate any help you can provide.

[ben-manes]

Hi @jojaeng2,

That feature is implemented in BoundedLocalCache#refreshIfNeeded. Unfortunately refresh is one of the most complicated, gnarly bit of logic because of the various interactions, which those explicit refresh methods also must cope with.

The feature is sometimes referred to as soft vs hard TTL or refresh ahead caching. When an entry has reached the expiration time then it is unusable, the cache should discard it, and if present but requested then it should be silently removed and treated as a cache miss. The refresh period is the duration when the entry is considered stale so it can be used if requested, but it would be desirable to reload it opportunistically to have a fresh copy. Thus, when the entry is requested within this period then the cache will return the existing value and reload asynchronously, but if it is not accessed then the entry expires and will be discarded.

The benefit of this feature is to hide latency and avoid periodic spikes when popular items expire. For example a user's profile might be needed by every request but have a 10 minute expiration time to eventually reflect changes. When it expires then all requests by that user block waiting for it to be retrieved, which causes a noticeable impact on tail latencies. Instead if it has a 5 minute refresh time then a reload is triggered and the user-facing latencies stay flat. Later on when the user signs off then their profile is allowed to expire, reclaiming memory for use by more active users. Thus we maintain the benefits of a bounded cache policy.

A downside of this feature is that it can cause many individual loads as it is reactive to what keys are requested, which could happen in a spike. In those cases the user may want to batch the requests or spread them out to avoid overloading their data source. This can be accomplished using a coalescing cache loader to buffer over a time period, which adds to the load time but that delay isn't user-visible since the cache is returning the existing value.

Some users mistakenly think this feature is an automatic reload of the cache contents on a periodic basis. As in, every 5 minutes load all of the cache contents in a background thread. That's not really a cache as there is no bounding policy and instead its a read replica, so a simple map with a ScheduledExecutorService fulfills those requirements.

I hope that helps clarify the feature and let me know if you have any other questions.

[jojaeng2]

Thank you for your answer! It was helpful