Use search_pool to control iterator->Next()

[alwayslove2013]

issue: #997

The current AnnIterator function utilizes the search pool for concurrency control only when initializing the Iterator. Once the returned iterator is handed over to the upper layer, the ->next() calls are not subjected to any thread restrictions, which may lead to issues such as OMP conflicts.

To address this, we propose the following considerations:

• All iterators will accept a use_knowhere_search_pool parameter during construction.
  • When set to True (the default), the iterator->next() will be scheduled by the knowhere_search_thread_pool.
  • When set to False, iterator->next() will not involve thread scheduling internally, so please take caution.
• The initialization of iterator in the AnnIterator function will no longer be concurrent, which helps to avoid potential deadlocks.
  • Furthermore, to enhance performance for large_nq, we will try to streamline the initialization process for all iterators by deferring heavier pre-computation tasks to the first call of ->next().

[sre-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: alwayslove2013
To complete the pull request process, please assign presburger after the PR has been reviewed.
You can assign the PR to them by writing /assign @presburger in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mergify]

@alwayslove2013 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

1. If you're fixing a bug, label it as kind/bug.
2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

[alexanderguzhva]

Please add a comment about why this parameter is false

[alwayslove2013]

added.

The range_search function has utilized the search_pool to concurrently handle various queries.
To prevent potential deadlocks, the iterator for a single query no longer requires additional thread control over the next() call.

[alexanderguzhva]

What are the justifications for adding use_knowhere_search_pool as a template parameter instead of making a regular IndexIterator::use_knowhere_search_pool field? I'm not sure I follow

[alwayslove2013]

Just to reduce the judgment of use_knowhere_search_pool in each iterator->next() call.

[alexanderguzhva]

just to confirm: this is evaluated as distances_id.id = a ? b : c, not `distances_id.id = (a ? b : c) + xb_id_offset'?

[alwayslove2013]

@cqy123456 help check~