Cache and refence counting

[AndreiSolodinTR]

Hello, we have a complex use case where we cache a value that has internal resources. The resources must not be freed until all users are done using the object. To accomplish this we use reference counting. When the object is added to the cache, we set the ref count to 1. The object imeplements AutoCloseable and decrements on each close. Also decrements when it is evicted from the cache (via eviction listener). That way when ref count is 0 the resources are released. The issue is that we need to increment the internal reference count when the object is obtained from the cache. Is there a hook/listener that would be invoked when the cache hands out the obeject that we could use? We can of course have a facade on top of the cacheable that increments the ref count, but it's not a very elegant solution. Thanks

[ben-manes]

The cache is best viewed as a bounded ConcurrentMap, so like HashMap your own logic would act as a facade above it rather than being the direct client api. If you simply need to increment a per-entry counter on access then that is trivial to do in your code and the library offering something doesn't add much value.

The faq discusses a similar problem called pinning, where ref counted entries should not be evictable. To do that the counter can be modified within a Map computation, so that when the cache evaluates its expiration and weight those can be set to be disabled. You might find that useful or else a similar approach if you want the increment to be atomic with the entry (but as a cache write, this is more expensive than a lock-free read).

[AndreiSolodinTR]

Thanks. I forgot to mention we use Spring @Cacheable paradigm. Since normally the rest of the code interacts directly with the component exposing the @Cacheable method, it becomes a lot harder: not only we must implement a facade, but we also need to hide the @Cacheable impl, so some user doesn't accidentally use the wrong way to get a cached object. In that case I believe a listener that is invoked for a cache hit read would be a valuable addition.

[ben-manes]

That sounds like a feature for Spring then, since it is providing an abstraction which makes this harder and could add this feature generically across all providers. You might be able to do something via AOP to intercept after theirs is called. I haven't used Spring since 1.x days (xml), so I am very rusty with its best practices.

[AndreiSolodinTR]

Hello, just wanted to briefly revisit this. I do not really see how adding this function to Spring would help, since incrementing the refCount needs to be atomic with cache hit. Correct me if I am wrong, but Spring is not aware of the caffeine internal expiration/refresh lifecycle. So it is no better than us using a kludge where we risk a race condition where the value is expired before we have a chance to increment the refCount. We of course can synchronize and add our own handing of this condition, but it's just messy any way you look at it.

[ben-manes]

The native apis allow for this complex usage, but the Spring ones do not. We do that using a Map.compute so that the entry is locked, whereas Map.get is lock-free so you would not have an atomic increment if done after the read. The asMap() view gives access to this, so no internals are needed. The problem is that Spring Cache is only meant for simplistic cases and they recommend using the vendor's apis beyond that.

The hard part is that you want to maintain Spring Cache and not hide the details under a facade. The various tricks don't work in that case, like using a secondary map, our native apis, a proxy type (e.g. see jdbc connection pools). That leads to one approach that may not be ideal but might work for you: phantom references. That would close the resource by piggybacking on the GC (see `java.lang.ref.Cleaner). That is not always ideal for resources as the GC is unpredictable, but can be useful in a pinch.

[jay663]

I'm looking to do something similar here. I want to save the cache entries to a file when the cache shuts down. How can I add an event to determine if the cache is shutting down? I have a singleton in front of the cache. I thought about using AutoCloseable and running code when it's shutting down.

[ben-manes]

The cache does not have its own lifecycle. It is a ConcurrentHashMap with added features, so you can think of it as any other data structure. You can provide the lifecycle logic, e.g. we do not manage any threads ourselves or other resources. You may find Cache.policy() methods helpful if you want to persist the hottest subset of entries.