Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve throughput and resiliency of batching algo #1371

Closed
wants to merge 8 commits into from
Closed

Improve throughput and resiliency of batching algo #1371

wants to merge 8 commits into from

Conversation

tsmith023
Copy link
Contributor

@tsmith023 tsmith023 commented Oct 29, 2024 •
edited
Loading

  • Removes locks from event loop
  • Handles mapping async resp in batch thread instead of event loop thread to remove data locks in async fn
  • Uses return of loop.schedule in batching thread to determine number of active requests without locks in async fn

Closes: #1332

@tsmith023
Improve throughput and resiliency of batching algo:
89c65c8 
- Removes locks from event loop
- Handles mapping async resp in batch thread instead of event loop thread to remove data locks in async fn
- Uses return of loop.schedule in batching thread to determine number of active requests without locks outside of async fn
@tsmith023
Ensure awaited coro's raised exception can be caught properly
f554d45
@weaviate-git-bot
Copy link

Great to see you again! Thanks for the contribution.

beep boop - the Weaviate bot 👋🤖

PS:
Are you already a member of the Weaviate Slack channel?

@codecov-commenter
Copy link

Codecov Report

Attention: Patch coverage is 75.81699% with 37 lines in your changes missing coverage. Please review.

Project coverage is 93.91%. Comparing base (3f68082) to head (a12c055).
Report is 138 commits behind head on main.

Files with missing lines Patch % Lines
weaviate/collections/batch/base.py 75.49% 37 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main    #1371      +/-   ##
==========================================
- Coverage   94.38%   93.91%   -0.48%     
==========================================
  Files         220      221       +1     
  Lines       20956    21301     +345     
==========================================
+ Hits        19779    20004     +225     
- Misses       1177     1297     +120

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tsmith023 tsmith023 marked this pull request as ready for review October 30, 2024 12:12
@tsmith023 tsmith023 mentioned this pull request Oct 30, 2024
Open
@tsmith023 tsmith023 changed the title Improve throughput and resiliency of batching algo: Improve throughput and resiliency of batching algo Oct 30, 2024
@tsmith023
Copy link
Contributor Author

Additional overhead of locking in the main batch looping thread and doing all CPU there slows down the logic considerably. Closing in favour of #1270

@tsmith023 tsmith023 closed this Nov 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dirkkul dirkkul Awaiting requested review from dirkkul

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Fix event loop blocking and deadlocks in batching loop
3 participants
@tsmith023 @weaviate-git-bot @codecov-commenter