What routine maintenance are carried out when cache is expired?

[Oluwatobiadeoye]

I'm setting my cache to expire after some time using expireAfterWrite so I saw in the documentation some routine maintenance are done after such expiration. I wanted to know what the routine maintenance are and the frequency with which it happens?

Reason because I'm caching some database connection which I'd love be to cleaned up after the cache is cleared up to prevent connection leaks.

[ben-manes]

In a single threaded cache, one could perform housekeeping work on every request. For example reorder on an LRU for size eviction, check if an expiration occurred, or peek at the ReferenceQueue if a weak/soft entry was collected. These operations are fast but not friendly for concurrency, e.g. an LRU is a doubly-linked list where the entry is moved to the tail (many non-atomic pointer changes). A traditional cache like LinkedHashMap required synchronization, but at high concurrency the locking is more expensive than the policy maintenance.

Caffeine instead performs this policy work periodically. It captures events like reads and writes into a small queue (ring buffers), which are cheap to append into. When enough work piles up then the events are replayed against the eviction policy under an exclusive lock. So instead locking to perform each operation like LRU reordering on every access, we batch up that work and do not block threads. This allows us to achieve high concurrency while maintaining the eviction policies, which are cheap O(1) algorithms.

However, expiration is only checked as a side effect of activity on the cache. If the cache is used frequently then maintenance will be triggered regularly and the entries removed. If the cache is not used very often, though, then the expired entries will remain and won't be cleaned up until some time later. Typically a cache is used often enough for this to not be a concern, but if you rely on prompt removal notifications then you might want a stronger guarantee.

Caffeine does not create its own threads, but one can be provided to schedule against. This way a maintenance cycle will be performed when the next entry is set to expire. For example if you are on Java 9+ then you can use the JVM's system-wide scheduler, as shown below. Otherwise a ScheduledExecutorService will be required.

LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .scheduler(Scheduler.systemScheduler())
    .expireAfterWrite(10, TimeUnit.MINUTES)
    .build(key -> createExpensiveGraph(key));

[Oluwatobiadeoye]

Hi ben-manes,

Apologies for the late response.

Thanks for this detailed reply and I really understood your explanation. My cache is used often so I should not need to add a scheduler to the cache as demonstrated in the code above.

However, just to confirm, when the cache entries are expired they are marked for garbage collection, right? if not, what happens to those entries after the maintenance is carried out.

[ben-manes]

When the entries have expired then they are pending automatic removal. They include a timestamp and are held on an O(1) priority queue. If an access for that key occurs then the entry will appear externally to be absent and handled appropriately. When routine maintenance kicks in then the priority queue will be queried and all expired entries removed from the map. At that point they may become eligible for garbage collection from the JVM's perspective.

The routine maintenance is primarily triggered when enough reads occur or any write. If the expired entry is queried for directly, e.g. getIfPresent or an iterator, then it will trigger the maintenance optimistically as well. Since we use only O(1) algorithms the work involved is very small and we can do it frequently, and is handled in a way that allows for high concurrency.

[Oluwatobiadeoye]

Thank you for this detailed explanation. Really useful!