Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PLAT-108920] not skip samples during dedup #38

Closed
wants to merge 2 commits into from

Conversation

jnyi
Copy link
Collaborator

@jnyi jnyi commented May 21, 2024

A lot of unit tests broken to be fixed, but the idea is currently both iters proceed and potentially skip data points in between. Instead, we only scan iterators and move forward if previous value is being used.

For example:
A: 1, 4, 5
B: 1, 2, 3, 4, 5

Before the change: dedup'ed result will be 1, 4, 5

After the change, the result will be 1, 2, 3, 4, 5

  • I added CHANGELOG entry for this change.
  • Change is not relevant to the end user.

Changes

Verification

@jnyi
Copy link
Collaborator Author

jnyi commented May 29, 2024

closing this one, we'd better not to change existing behavior and introduce another merge dedup iterator, fyi @hczhu-db

@jnyi jnyi closed this May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant