Does Caffeine support cache regions or a cache manager?

[lmajano]

Does Caffeine support cache regions or a cache manager?

[ben-manes]

The core library is just a data structure with no extra lifecycle, like a HashMap. It doesn't include any concepts of a manager or region owner, it's just a map that includes eviction policies.

The JCache module implements jsr107 which defines a CacheManager and lifecycle. This is a standard that was rejected by JEE / Jakarta, most modern frameworks, and has low adoption. The API is error prone so it is mostly used in cases like Hibernate which was an early adopter.

Otherwise frameworks like Quarkus, Spring, and others incorporate their own cache managers. Their integrations are simple because they don't have to bridge different lifecycles and can manage everything directly. These tend to differ substantially, like Apache HBase's cache management is very different to Apache Solr's, yet both use Caffeine to manage the state.

[lmajano]

Thank you so much for your detailed answer. I really appreciate you taking the time. This guides me further in our decision of choice. We want a lightweight and performance option for our framework.

If it's not much to ask, how would you compare caffeine vs. guava cache? Do you have any guidance, pros, cons, or personal experience?

[ben-manes]

I coauthored Guava's cache and, while no longer involved, the maintainers kindly recommend using Caffeine instead in their JavaDoc.

The only time to prefer Guava's is that by historic accident, it is unintentionally compatible with Android & GWT whereas Caffeine is probably not. Neither were intended or optimized for those platforms, where there are better choices but might be used for convenience. There are now adapters like cache-api to specialize if compiling cross-platform.

Caffeine is based on the original design I proposed for Guava, modernizations and learnings, and the latest academic research. It has an extensive test suite, is used more broadly, and the algorithms independently validated by engineers when porting to their language (C#, Go, Rust, etc). If you run into any problems please feel welcome to ask.

[lmajano]

Thanks again @ben-manes , really appreciate it. I am guessing you are in CET since you are answering quite fast :). I am in Spain currently. This makes our choice easier to make.

[ben-manes]

Nope, PT. It’s just insomnia

[lmajano]

Ok, another question. I don't see a way to provide an expiration on a per-item case. It's this by design?

[Stephan202]

Hey @lmajano! Have a look at the Caffeine#expireAfter(Expiry) builder method. Custom expiry is defined by implementing an instance of Expiry.

[ben-manes]

Thanks @Stephan202! fyi, while we strongly encourage the callback style, we'll stash various escape hatches under cache.policy(). That offers configuration-specific methods for advanced cases or to assist migrations from a homegrown cache. These are a bit hidden as disfavored in that most developers won't consider how concurrency or features will interact, so the core apis are designed to steer you towards robust solutions. We just don't want to limit you in getting work done, which sometimes isn't pretty but necessary, so use those escape hatches sparingly.

[lmajano]

I have seen the expireAfter and actually all policies seem to apply to all entries. However, we have several apps that require some elements to have different expiration times. There might be items that track session or other items I need in cache for 90 minutes, or some for 2 weeks. Having the flexibility to attach a timespan to entries is a requirement in some cases. However, I don't seem to see this in the API.

[ben-manes]

Let's use the user guide's example. The interface is a bit verbose but also very flexible,

LoadingCache<Key, Graph> graphs = Caffeine.newBuilder()
    .expireAfter(new Expiry<Key, Graph>() {
      public long expireAfterCreate(Key key, Graph graph, long currentTime) {
        // Use wall clock time, rather than nanotime, if from an external resource
        long seconds = graph.creationDate().plusHours(5)
            .minus(System.currentTimeMillis(), MILLIS)
            .toEpochSecond();
        return TimeUnit.SECONDS.toNanos(seconds);
      }
      public long expireAfterUpdate(Key key, Graph graph, 
          long currentTime, long currentDuration) {
        return currentDuration;
      }
      public long expireAfterRead(Key key, Graph graph,
          long currentTime, long currentDuration) {
        return currentDuration;
      }
    })
    .build(key -> createExpensiveGraph(key));

The Expiry inspects the key and value to determine the remaining duration, or else returns the current duration if unchanged. This example uses the external creation time as a reference point for how long an entry is valid, which could be a license restriction of a third party service. For example Google Maps allows for 30 days of local caching for geocode results, so it is common to have those stored that in a database to during this period to minimize costs.

While in that example it inspects a timestamp, you might vary based on the domain model. For example if you wish to leverage negative caching where the absence is stored then you may wish to apply a short TTL for the miss and a longer duration on a hit. This can be modeled caching an Optional<V>, which avoids ambiguity if null values were allowed. One might implement this like,

public long expireAfterCreate(Key key, Optional<Graph> graph, long currentTime) {
  return graph.isEmpty() ? TimeUnit.MINUTES(5) : TimeUnit.HOURS(1);
}
public long expireAfterUpdate(Key key, Optional<Graph> graph, 
    long currentTime, long currentDuration) {
  return graph.isEmpty() ? TimeUnit.MINUTES(5) : TimeUnit.HOURS(1);
}
public long expireAfterRead(Key key, Optional<Graph> graph,
    long currentTime, long currentDuration) {
  return currentDuration;
}

You can calculate the timestamp based on the entry's creation, its last update, or its last read. This maintains a single timestamp for when the entry expires.

[ben-manes]

The callback approach is ideal by avoiding many gotchas, but if you want the caller to pass in the duration explicitly then you can default to an infinite policy like,

public long expireAfterCreate(Key key, Graph graph, long currentTime) {
  return Long.MAX_VALUE;
}
public long expireAfterUpdate(Key key, Graph graph, 
    long currentTime, long currentDuration) {
  return currentDuration;
}
public long expireAfterRead(Key key, Graph graph,
    long currentTime, long currentDuration) {
  return currentDuration;
}

This way the cache is constructed with variable expiration so you can use the policy api to perform the explicit writes,

// eternal (via expiry)
cache.put(key1, value1);

// custom (via duration)
cache.policy().expireVariably().orElseThrow()
    .put(key2, value2, Duration.ofSeconds(30));