Avoiding peak loads

[emitskevich]

We have a highload LoadingCache with expireAfterWrite(5, MINUTES). It caches the results of DB selects. The cache contains 10k entries, each is accessed hundreds times a second. On server start we warm the cache, using many threads for speed, which means all 10k entries are loaded almost simultaneously. Having very frequent reads of each entry, every 5 minutes in the future we see the peak load of loading all entries at once.

Using refreshAfterWrite(3, MINUTES) doesn't help, because all entries still start to refresh at once, but in background. I can use fixed threadpool with the queue for refreshes, it theoretically will spread refreshes on some time distance. But is it proper way? Is there the option not to immediately refresh the entry on the first read after 3 minutes, but to assign random "refresh time" to each entry between 3 and 5 minutes? It would nicely solve the problem of such peak loads.

[ben-manes]

For variable refresh, #504 is the consolidated issue for that feature request. That would let you add a fuzz factor.

Another approach is to coalesce and process refresh in a managed way, e.g. @sheepdreamofandroids's CoalescingBulkloader example. This converts per-key futures into a bulk load by aggregating over a delay period.

If the cache holds your entire keyspace, which the warmup fetches, then you might be able to replace the cache with a simple ImmutableMap, a scheduled thread that periodically reloads it anew, and updating a volatile map field.

Would either of those approaches help?

[ben-manes]

Closing due to lack of feedback