Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

faster checksum phase #194

Open
morgo opened this issue Sep 6, 2023 · 0 comments
Open

faster checksum phase #194

morgo opened this issue Sep 6, 2023 · 0 comments
Labels
enhancement New feature or request for-future-development Not currently planned

Comments

@morgo
Copy link
Collaborator

morgo commented Sep 6, 2023

Usually the checksum phase takes about 10-20% of the time that the copier phase does. However, on some very IO bound cases I have noticed it take significantly longer.

Much earlier in development we offered a separate checksum-threads concurrency setting, but unified it with threads to simplify configuration.

Because in resume-from-checkpoint we resume from the start of the checksum again, it's important that we keep the checksum fast. We could either use a static rule like double the threads used on copier + double the chunk time, or we could offer a configuration variable. TBD.

@morgo morgo added the enhancement New feature or request label Sep 6, 2023
@morgo morgo added the for-future-development Not currently planned label Feb 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request for-future-development Not currently planned
Projects
None yet
Development

No branches or pull requests

1 participant