Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support buffer/Uint8Array prefixes #111

Open
CMCDragonkai opened this issue Mar 16, 2022 · 7 comments
Open

Support buffer/Uint8Array prefixes #111

CMCDragonkai opened this issue Mar 16, 2022 · 7 comments

Comments

@CMCDragonkai
Copy link

I want to use raw buffers as my prefixes, because I have some complex sublevel work going on.

Can we make subleveldown support buffers?

That is raw buffers so that the resulting prefix will just be a buffer concatenation, no encoding of the buffer as a string, then as utf8 encoded buffer. I just want the raw buffers to be the prefix itself.

@vweevers
Copy link
Member

You wouldn't be able to use the full range of bytes, as two must be reserved for separator and lt logic. Limiting the prefix to ASCII characters makes that easier to enforce.

@CMCDragonkai
Copy link
Author

The prefix separator is by default ! right? It's still possible to conflict with ! just by giving a prefix like abc!dsfgd. So why not allow raw buffers as prefixes, since you can always check if the prefix separator bytes is within the raw buffer bytes? One can use the buf.includes method to do this.

A more robust solution may use escaping applied to the prefix to always escape any occurrences of the prefix separator...

Not sure what you mean by lt logic.

@vweevers
Copy link
Member

Sure, it's possible, at the cost of complexity. Can you explain your use case? I'm curious about the benefit of buffer prefixes, if you can't use the full range of bytes.

It's still possible to conflict with ! just by giving a prefix like abc!dsfgd

Not important for the discussion but FYI: subleveldown will throw a RangeError if you do that.

@CMCDragonkai
Copy link
Author

I'm using subleveldown to store dynamic sub-levels. Depending on the usecase, I would prefer to use the name of the sublevel to be a raw buffer instead because sometimes I'm using the names as indexes. And the value I'm indexing on is itself a buffer. Right now it just involves converting to a string which seems unnecessary. However I guess a robust solution would require prefix-separator escaping.

@vweevers
Copy link
Member

Wouldn't escaping hurt the ability to use it as an index?

@CMCDragonkai
Copy link
Author

Only for the sublevel prefix/names. We're not affecting the keys. Not sure what you mean by hurting its ability to use it as an index.

@vweevers
Copy link
Member

Can you share some code to clarify the use case?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants