Skip to content

Commit

Permalink
accounts-db: Improve sharding of the read-only cache
Browse files Browse the repository at this point in the history
The default amount of shards in `DashMap` is `num_cpus * 4`. That means
256 on validators with 64 threads. A number of accounts held in caches
on mainnet beta validators with default configuration is around 44-56K,
which results in around 200 accounts per shard. That means, locking a
shard locks an access to 200 accounts.

Fix that by increasing the amount of shards to 65536, which in the best
case  will keep just one account per shard. Acquiring a lock shouldn't
lock access to any other accounts.

A single `RwLock<RawTable<(K, V)>>`, without the key and value included,
takes 48 bytes. Therefore, 65536 shards are taking 3145728 B, which
rounds up to 3MB. It's an acceptable increase in memopry usage.
  • Loading branch information
vadorovsky committed Jan 6, 2025
1 parent 69ae934 commit d0a8348
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion accounts-db/src/read_only_accounts_cache.rs
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,11 @@ use {
const CACHE_ENTRY_SIZE: usize =
std::mem::size_of::<ReadOnlyAccountCacheEntry>() + 2 * std::mem::size_of::<ReadOnlyCacheKey>();

/// Number of cache shards. This number is close to the number of accounts
/// in cache observed on mainnet beta validators, therefore it should result
/// in each accout having its own shard and reduced amount of locks.
const SHARDS: usize = 65536;

type ReadOnlyCacheKey = Pubkey;

#[derive(Debug)]
Expand Down Expand Up @@ -98,7 +103,7 @@ impl ReadOnlyAccountsCache {
) -> Self {
assert!(max_data_size_lo <= max_data_size_hi);
assert!(evict_sample_size > 0);
let cache = Arc::new(DashMap::default());
let cache = Arc::new(DashMap::with_shard_amount(SHARDS));
let data_size = Arc::new(AtomicUsize::default());
let stats = Arc::new(AtomicReadOnlyCacheStats::default());
let evictor_exit_flag = Arc::new(AtomicBool::new(false));
Expand Down

0 comments on commit d0a8348

Please sign in to comment.