Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Research][QOL] JSON/CSV vs Single file soln #3

Open
brianleect opened this issue Jul 28, 2022 · 2 comments
Open

[Research][QOL] JSON/CSV vs Single file soln #3

brianleect opened this issue Jul 28, 2022 · 2 comments
Labels
enhancement New feature or request question Further information is requested

Comments

@brianleect
Copy link
Owner

Currently we have each run of getLabels to be saved as {labelName}.json {labelName}.csv

Might be a major QOL improvement to consolidate into a single file?

address:{'name':NAMETAG, 'label':LABEL}

Might need to take a look into the size of all label information and whether it makes sense to leave it in a single large JSON file or into a DB (SQlite perhaps?)

@brianleect brianleect added enhancement New feature or request question Further information is requested labels Jul 28, 2022
@brianleect
Copy link
Owner Author

image

Current size is only 4-5mb. Not sure if it makes sense to use a DB due to the additional overhead/complexity added.

Considering 4-5mb is JSON + CSV, the consolidated file themselves is likely only ~2.5mb.

@brianleect
Copy link
Owner Author

Current single file consolidated JSON is located
https://github.com/brianleect/etherscan-labels/blob/main/data/combinedLabels.json

2.7mb space taken. Don't think it justifies usage of DBMS since we can simply load it into memory once and get object of interest. Since we are using address:nameTags format retrievals should be O(1) as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant