Blocking behavior on expired entry with a synchronous load?

[brent-holland]

I was watching a video on the behavior of guava cache. In this video the presenter mentions that when one goes to fetch an expired entry with a synchronous loader, that it blocks all threads trying to access that key until the load method is able to fetch an updated value. Confusingly, later in the video he stated that only the first thread trying to access an expired entry is blocked, and the rest get the stale value.

I was looking to understand how Caffeine handles this? I ask because we have some apps that need the correct value, and are perfectly willing to wait on it, and other apps that can handle getting a stale value as they deal with much higher tps.

Thanks for any help you can provide.

[ben-manes]

I'll try to watch this evening (morning my time, so meeting hours). In both caches, a get(key) that loads a missing entry will block all threads requesting that key, with only one thread performing the work. Whereas Guava wrote a custom hashmap to support this, Caffeine is able to leverage Java 8's compute methods (e.g. computeIfAbsent). An entry that expired is not usable, so it forces the cache to treat the lookup as a miss. All of this is to avoid a cache stampede which can cause can cause performance degradations and outages.

Expiration is a hard edge, so popular items that expire can cause periodic latency spikes as threads block waiting for the cache load. This latency can be hidden by reloading the entry earlier and asynchronously, so that the entry is kept fresh is always a cache hit. This is where refreshAfterWrite is useful, as it triggers a refresh if accessed after this duration but before the entry expires. In that case a stale value is returned while the reload is in-flight. If the entry is not accessed then it is allowed to expire, so that unpopular items fade away.

The presenter might be talking about those two features or mixed up their wording.

[ben-manes]

Oh, this was Charles! Yes, the second link is discussing automatic refresh. The two caches behave similarly for these features.