Limiting share of objects in the cache based on their type

[pugupta-linkedin]

Hi,
We use caffeine cache on top of a persistent database. In the caffeine cache we store (1) Positive lookups ie entries that do exist in the database (2) Negative lookups ie entries that don't exist. The value is simply a static object for the latter. The cache is created to store "maximumSize" entries - which i believe is the number of entries.

We notice that due to our usage pattern - entries of type (2) occupy 90%+ of the cache space (via a heap analysis).
I don't know why caffeine has a preference for them - our read traffic is much more balanced than that.

Is there a way we can limit the number of (2) entries in the cache relative to (1)?

I'm thinking of using "maximumWeight" to create the cache and associate a weight of 1 with (1) and let's say 10 with (2). Would this sort of an approach work? Please suggest other ideas

cc @ben-manes

[ben-manes]

The cache's policy uses frequency and recency, so I don't know why else one might be preferred. If a weight is specified then a zero weight would not be size evicted and skipped over. Otherwise, currently the weight isn't used to make an eviction decision and the cache simply evicts until under the capacity threshold. Your idea might work but could also be error prone as hard to predict the resulting behavior.

Potentially you could use variable expiration to expire negative lookups faster than positive ones. A refresh could hide the resulting latencies if actively used.

You might consider logging an access trace to see actual production behavior which can differ from expectations. That could be a log message after a cache.get of the key's id (or hash) and if a positive/negative lookup result. Then a simple script could tell you the relative frequencies and if it is balanced in practice.

[pugupta-linkedin]

Thanks for the quick response.

To limit the cache composition of negative entries -

1. I will try the variable expiration duration as you suggested
2. Or if that doesn't work well, i am thinking of having a separate bounded cache for negatives.

To debug why negative lookups are retained, is there a way to debug caffeine eviction internals?

1. I will try tracking counts on cache.get and removalListener
2. Calling getAsMap periodically seems to be errorprone since there are constant writes to the cache when the app is running.

[ben-manes]

I'm not sure what you would want to debug. The listener (or evictionListener) is a good choice.

For deeper internal state, you can do some basic queries through cache.policy().eviction(), e.g. obtain a snapshot in hot to cold order. While I think it's fun to share the low-level policy details for technical discussions, the API does tries to not expose that implementation detail to allow for a cleaner evolution (e.g. frequency counts). To peek at the internal state would be best done via an access trace to replay offline and to figure out why decisions are being made and compare that to other eviction policies (e.g. via our simulator).

[pugupta-linkedin]

Sounds good, thanks