Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Fix CAGRA graph optimization bug #565

Open
wants to merge 11 commits into
base: branch-25.02
Choose a base branch
from

Conversation

enp1s0
Copy link
Member

@enp1s0 enp1s0 commented Jan 11, 2025

The merging process of the pruned and revedge graphs is inappropriate, resulting in neighbor index duplication. This PR fixes the issue and adds the validation of graph indices at the end of the graph::optimze function.

The simplest solution to fix this issue is 1151511, but it ignores the memory space optimization by rapidsai/raft@12480cf. The code changes in this PR apply the memory space optimization to the simplest solution to remove the memory space for the pruned graph (pruned_graph).

This PR also fixes a problem that throwing an exception from an OMP loop does not work as expected.

@enp1s0 enp1s0 requested a review from a team as a code owner January 11, 2025 06:51
@enp1s0 enp1s0 self-assigned this Jan 11, 2025
@github-actions github-actions bot added the cpp label Jan 11, 2025
@enp1s0 enp1s0 added bug Something isn't working non-breaking Introduces a non-breaking change and removed cpp labels Jan 11, 2025
@enp1s0 enp1s0 changed the title Fix CAGRA graph optimization bug [BUG] Fix CAGRA graph optimization bug Jan 11, 2025
@github-actions github-actions bot added the cpp label Jan 11, 2025
@enp1s0
Copy link
Member Author

enp1s0 commented Jan 12, 2025

The new optimize function gets stuck in cagra_extreme_inputs_oob_test because there are some invalid index (~0u) nodes in the initial knn graph. It would be better to check the invalid nodes first in the function and abort the process if there are any.

It seems too severe to abort the optimization process if there is even one invalid node, so I updated the code not to abort unless the pruned graph can be generated even if there are invalid nodes in the initial kNN graph.

@enp1s0
Copy link
Member Author

enp1s0 commented Jan 13, 2025

@anaruse Can you review the update? It is related to merging the pruned graph and MST optimization edges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working cpp non-breaking Introduces a non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant