[Chunk Data Pack] Refactor Chunk data packs modules to use generic storage interface #6892
Conversation
| @@ -60,18 +69,28 @@ func run(*cobra.Command, []string) { | |||
| metrics := &metrics.NoopCollector{} | |||
| transactionResults := badger.NewTransactionResults(metrics, db, badger.DefaultCacheSize) | |||
| commits := badger.NewCommits(metrics, db) | |||
| chunkDataPacks := badger.NewChunkDataPacks(metrics, db, badger.NewCollections(db, badger.NewTransactions(metrics, db)), badger.DefaultCacheSize) | |||
There was a problem hiding this comment.
The previous implementation was wrong. Since we have moved chunk data packs to pebble, we should use pebble based chunk data packs module for rolling executed height.
| headers := bstorage.NewHeaders(metrics, db) | ||
| txResults := bstorage.NewTransactionResults(metrics, db, bstorage.DefaultCacheSize) | ||
| commits := bstorage.NewCommits(metrics, db) | ||
| chunkDataPacks := store.NewChunkDataPacks(metrics, pebbleimpl.ToDB(pdb), bstorage.NewCollections(db, bstorage.NewTransactions(metrics, db)), bstorage.DefaultCacheSize) |
There was a problem hiding this comment.
This is the actual change.
| require.NoError(t, err) | ||
| require.NoError(t, chunkBatch.Commit()) | ||
| err2 := batch.Flush() |
There was a problem hiding this comment.
This is the actual change.
| require.NoError(t, err) | ||
| require.NoError(t, chunkBatch.Commit()) | ||
| err2 := batch.Flush() | ||
| require.NoError(t, err2) |
There was a problem hiding this comment.
This is the actual change.
| @@ -0,0 +1,24 @@ | |||
| package operation | |||
There was a problem hiding this comment.
Moved from storage/pebble/operation/chunk_data_packs.go
| @@ -0,0 +1,133 @@ | |||
| package store_test | |||
There was a problem hiding this comment.
Moved from storage/pebble/chunk_data_packs_test.go
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #6892 +/- ##
=======================================
Coverage 41.11% 41.12%
=======================================
Files 2116 2116
Lines 185737 185709 -28
=======================================
+ Hits 76368 76369 +1
+ Misses 102952 102942 -10
+ Partials 6417 6398 -19
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
This PR refactors the chunk data packs modules to use the generic storage interfaces. This allows us to reuse functions that made for the generic storage interface to be used on chunk data packs when implementing chunk data pack pruner.
Since chunk data packs are stored in pebble now, this PR also fixed some test cases and tools that was still dealing with chunk data packs stored in badger.