Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

wand optimize cursor class #968

Merged
merged 1 commit into from
Dec 2, 2024
Merged

Conversation

zhengbuqian
Copy link
Collaborator

@zhengbuqian zhengbuqian commented Nov 29, 2024

the main change is to update the cursor class to cache certain info, as per the perf result those size() calls used quite a lot of extra CPU cycles

also simplifed code to drop less important elements

simple benchmark result(time used to search all queries once on the full dataset) with drop ratio build 0.32 and drop ratio search 0.6:

Before After Improvement %
bigann SPLADE vectors of MSMarco 141,473ms 89,604ms 36%
Milvus 2.5 Full Text Search Vectors of MSMarco(using IP/TF-IDF metric though, not BM25) 4,758ms 3,154ms 34%

@sre-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: zhengbuqian

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

mergify bot commented Nov 29, 2024

@zhengbuqian 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

  1. If you're fixing a bug, label it as kind/bug.
  2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
  3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
  4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

@zhengbuqian
Copy link
Collaborator Author

/kind improvement

@zhengbuqian zhengbuqian changed the title wand: optimize cursor class [WIP] wand optimize cursor class Nov 29, 2024
Copy link

codecov bot commented Nov 29, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.37%. Comparing base (3c46f4c) to head (128cdb5).
Report is 260 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           main     #968       +/-   ##
=========================================
+ Coverage      0   74.37%   +74.37%     
=========================================
  Files         0       82       +82     
  Lines         0     6685     +6685     
=========================================
+ Hits          0     4972     +4972     
- Misses        0     1713     +1713     

see 82 files with indirect coverage changes

inline void
update_cur_vec_id() {
if (loc_ >= lut_size_) {
cur_vec_id_ = total_num_vec_;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just double-checking that this is not total_num_vec_ - 1. According to standard naming conventions, id_ is assumed to be something within [0, num) range.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it is not total_num_vec_ - 1. When id_ is in [0, num) range it is still in range and the cursor is pointing at some doc, loc_ >= lut_size_ means it is out of range and the cursor points at no doc.

@zhengbuqian zhengbuqian changed the title [WIP] wand optimize cursor class wand optimize cursor class Nov 30, 2024
@alexanderguzhva
Copy link
Collaborator

/lgtm

@sre-ci-robot sre-ci-robot merged commit 75538c5 into zilliztech:main Dec 2, 2024
14 checks passed
@zhengbuqian zhengbuqian deleted the wand-optimiz branch December 2, 2024 15:07
@zhengbuqian
Copy link
Collaborator Author

issue: #967

foxspy pushed a commit to foxspy/knowhere that referenced this pull request Dec 6, 2024
…liztech#968)

simplify code to drop

Signed-off-by: Buqian Zheng <[email protected]>
Signed-off-by: xianliang.li <[email protected]>
sre-ci-robot pushed a commit that referenced this pull request Dec 6, 2024
* wand: optimize cursor class to get ~10% performance improvement; (#968)

simplify code to drop

Signed-off-by: Buqian Zheng <[email protected]>
Signed-off-by: xianliang.li <[email protected]>

* update cardinal version

Signed-off-by: xianliang.li <[email protected]>

---------

Signed-off-by: Buqian Zheng <[email protected]>
Signed-off-by: xianliang.li <[email protected]>
Co-authored-by: Buqian Zheng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants