-
Notifications
You must be signed in to change notification settings - Fork 31
Latency
In these benchmarks, a cache miss is essentially free. These tests exist purely to compare the raw execution speed of the cache bookkeeping code. In a real setting, where a cache miss is presumably quite expensive, the relative overhead of the cache will be very small.
Benchmarks are based on BenchmarkDotNet, so are single threaded. The ConcurrentLru family of classes are composed internally of ConcurrentDictionary.GetOrAdd and ConcurrentQueue.Enqueue/Dequeue method calls, and scale well to concurrent workloads.
In this test the same items are fetched repeatedly, no items are evicted. Representative of high hit rate scenario, when there are a low number of hot items.
ConcurrentLru family does not move items in the queues, it is just marking as accessed for pure cache hits. Classic Lru must maintain item order, and is internally splicing the fetched item to the head of the linked list. MemoryCache and ConcurrentDictionary represent a pure lookup. This is the best case scenario for MemoryCache, since the lookup key is a string (if the key were a Guid, using MemoryCache adds string conversion overhead). FastConcurrentLru does not allocate and is approximately 5-10x faster than System.Runtime.Caching.MemoryCache or the newer Microsoft.Extensions.Caching.Memory.MemoryCache.
| Method | Runtime | Mean | StdDev | Ratio | Allocated |
|---|---|---|---|---|---|
| ConcurrentDictionary | .NET 6.0 | 7.783 ns | 0.0720 ns | 1.00 | - |
| FastConcurrentLru | .NET 6.0 | 9.773 ns | 0.0361 ns | 1.26 | - |
| ConcurrentLru | .NET 6.0 | 13.615 ns | 0.0606 ns | 1.75 | - |
| FastConcurrentTLru | .NET 6.0 | 25.480 ns | 0.0935 ns | 3.28 | - |
| ConcurrentTLru | .NET 6.0 | 29.890 ns | 0.2107 ns | 3.84 | - |
| ClassicLru | .NET 6.0 | 54.422 ns | 0.2935 ns | 7.00 | - |
| RuntimeMemoryCacheGet | .NET 6.0 | 115.016 ns | 0.6619 ns | 14.79 | 32 B |
| ExtensionsMemoryCacheGet | .NET 6.0 | 53.328 ns | 0.2130 ns | 6.85 | 24 B |
| ConcurrentDictionary | .NET Framework 4.8 | 13.644 ns | 0.0601 ns | 1.00 | - |
| FastConcurrentLru | .NET Framework 4.8 | 14.639 ns | 0.0892 ns | 1.07 | - |
| ConcurrentLru | .NET Framework 4.8 | 17.008 ns | 0.2538 ns | 1.25 | - |
| FastConcurrentTLru | .NET Framework 4.8 | 43.854 ns | 0.0827 ns | 3.22 | - |
| ConcurrentTLru | .NET Framework 4.8 | 47.954 ns | 1.2772 ns | 3.52 | - |
| ClassicLru | .NET Framework 4.8 | 62.683 ns | 0.8105 ns | 4.60 | - |
| RuntimeMemoryCacheGet | .NET Framework 4.8 | 287.627 ns | 1.3691 ns | 21.08 | 32 B |
| ExtensionsMemoryCacheGet | .NET Framework 4.8 | 114.511 ns | 0.5902 ns | 8.39 | 24 B |