Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[3.0] Protect against chunk reordering and truncation in ChunkedEncryption authentication #158

Open
bchess opened this issue Jun 27, 2024 · 1 comment
Assignees
Milestone

Comments

@bchess
Copy link
Contributor

bchess commented Jun 27, 2024

Leaving this as placeholder for @Eta0 to make accurate description.
Copying from https://github.com/coreweave/tensorizer/pull/127#pullrequestreview-2133569874https://github.com/coreweave/tensorizer/pull/127#pullrequestreview-2133569874

Tweak ChunkedEncryption slightly in a way that is incompatible with older deserializers, so I'd like to add it in the new data version 5 alongside the changes in this PR

@bchess bchess added this to the 3.0 milestone Jun 27, 2024
@Eta0
Copy link
Contributor

Eta0 commented Jun 27, 2024

This is referring to resolving a limitation currently listed in docs/encryption.md:

This level of encryption [ . . . ] does not protect against reordering or truncation of chunks.

We can protect against reordering and truncation of chunks by creating a hash list complete with a top hash. We currently use a hash list that is missing a top hash, so individual chunks of a tensor are verifiable, but their count and positions are not.

A good title for this issue would be "Protect against chunk reordering and truncation in ChunkedEncryption authentication."

@bchess bchess changed the title [3.0] tweak ChunkedEncryption [3.0] Protect against chunk reordering and truncation in ChunkedEncryption authentication Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants