Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modify in-place compacted check to include rebase source partition locator #318

Merged
merged 10 commits into from
Jun 11, 2024

Conversation

yankevn
Copy link
Collaborator

@yankevn yankevn commented Jun 7, 2024

This change adds a check for the rebase source partition locator. This is to prevent erroneously marking tables as in-place compacted during rebase jobs.

Tested by running a rebase job in personal stack.

Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2.

Benchmark suite Current: b72e210 Previous: d77a208 Ratio
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[4-incremental-duplicate-pk_V2] 1.1538268689613613 iter/sec (stddev: 0) 2.3100160932581546 iter/sec (stddev: 0) 2.00

This comment was automatically generated by workflow using github-action-benchmark.

@yankevn yankevn requested review from raghumdani and pfaraone June 11, 2024 22:26
Copy link
Collaborator

@raghumdani raghumdani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Please create a separate private func for merge_and_promote. We spend more than necessary to debug issues due to cyclomatic complexity of code.

@yankevn yankevn merged commit ce1fcde into main Jun 11, 2024
4 checks passed
@yankevn yankevn deleted the in_place_compacted_determination branch June 11, 2024 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants