Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Batch Inserts into Db when Uplaoding Papers #84

Open
rajivharlalka opened this issue Aug 29, 2024 · 4 comments
Open

Batch Inserts into Db when Uplaoding Papers #84

rajivharlalka opened this issue Aug 29, 2024 · 4 comments

Comments

@rajivharlalka
Copy link
Member

Is your feature request related to a problem? Please describe.
Currently when multiple papers are uploaded, metadata for each paper is inserted in individual insert statement. These inserts can be batched into a single one for performance bonus.

@proffapt
Copy link
Member

@rajivharlalka or @harshkhandeparkar please update the state of this issue to be reflected on the kanban.

@harshkhandeparkar
Copy link
Member

@rajivharlalka db god :nam:

@rajivharlalka
Copy link
Member Author

rajivharlalka commented Sep 29, 2024 via email

@harshkhandeparkar harshkhandeparkar added enhancement New feature or request good first issue Good for newcomers labels Sep 29, 2024
@harshkhandeparkar harshkhandeparkar removed the good first issue Good for newcomers label Oct 17, 2024
@harshkhandeparkar
Copy link
Member

Putting this on hold since the change would make error handling a lot more complicated. Since we send a per-paper error message, it would be harder to handle db rollbacks if only one of the files fails to write. Also, since all would be inserted into the db at once, an error would fail to upload all papers. The performance benefit for our use case is most probably not worth the extra code complication.

Ref: https://metakgp.slack.com/archives/C05KTAL5ARM/p1729064846865189

@harshkhandeparkar harshkhandeparkar added discussion and removed enhancement New feature or request labels Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Development

No branches or pull requests

3 participants