feat: Add the ability to checkpoint an existing server, and spawn a read-only server on that view.

[nathanlo99]

It would be nice to be able to spawn a read-only server on a snapshot of a given server.

This PR implements this feature, and introduces the flag --snapshot-dir to manage the snapshot.

[nathanlo99]

--- FAIL: TestBitmap (846.21s)
    --- FAIL: TestBitmap/SETBIT/GETBIT/BITCOUNT/BITPOS_boundary_check_(type_bitmap) (150.90s)
        bitmap_test.go:204: 
            	Error Trace:	/Users/runner/work/kvrocks/kvrocks/tests/gocase/unit/type/bitmap/bitmap_test.go:204
            	Error:      	Received unexpected error:
            	            	read tcp 127.0.0.1:50435->127.0.0.1:50429: i/o timeout
            	Test:       	TestBitmap/SETBIT/GETBIT/BITCOUNT/BITPOS_boundary_check_(type_bitmap)

The above check failure seems to be an unrelated i/o timeout; is there a way to rerun the checks?

[git-hulk]

--- FAIL: TestBitmap (846.21s)
    --- FAIL: TestBitmap/SETBIT/GETBIT/BITCOUNT/BITPOS_boundary_check_(type_bitmap) (150.90s)
        bitmap_test.go:204: 
            	Error Trace:	/Users/runner/work/kvrocks/kvrocks/tests/gocase/unit/type/bitmap/bitmap_test.go:204
            	Error:      	Received unexpected error:
            	            	read tcp 127.0.0.1:50435->127.0.0.1:50429: i/o timeout
            	Test:       	TestBitmap/SETBIT/GETBIT/BITCOUNT/BITPOS_boundary_check_(type_bitmap)

The above check failure seems to be an unrelated i/o timeout; is there a way to rerun the checks?

Yes, it's a flaky test and not related to this PR.

[nathanlo-hrt]

@PragmaTwice @git-hulk could you take another look at this? It seems the previous opportunity to merge was missed because new changes made it to the unstable branch.

[mapleFU]

Actually I don't really understand how this works, seems this create a checkpoint when opening the file? Do you need create it on server and consume it on cli?

[nathanlo99]

Do you need create it on server and consume it on cli?

I'm not sure i understand the second part of your question

Actually I don't really understand how this works, seems this create a checkpoint when opening the file?

As for how this works, it will read the data from the dir passed by the user, then use RocksDB snapshot functionality to make a read-only copy of that data into the snapshot-dir directory, then spawn a read-only server on that copy.

This is helpful for my team to debug a running kvrocks app without accidentally changing the values of the data or closing the existing read-write server.

[mapleFU]

Would you mind also add example for how you'd like use this? ( better on doc of cli or doc of project?)

[nathanlo99]

apologies for the linter back and forth: i'm unable to build this locally with the same set-up as the linter uses, so a lot of these issues fall through the cracks

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

[nathanlo99]

@mapleFU I'm not sure how to edit the docs, but one example workflow is the following.

Start a read-write Kvrocks server writing data to ~/kvrocks-dir, for the purposes of writing important data.

$ kvrocks [...] --dir ~/kvrocks-dir

Say the user realizes some of the important data is incorrect: they'd like to debug their script and run it on a copy of the data, possibly changing it. They'd like to do this without fear of corrupting or changing the original server's copy of the data:

$ mkdir ~/debug-dir
$ kvrocks [...] --dir ~/kvrocks-dir --snapshot-dir ~/debug-dir