-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Upgrade daft and adding two more tests for schema evolution usecase #337
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Collaborator
raghumdani
commented
Jul 19, 2024
- Added couple of tests.
- Upgraded daft version
- This fixes an issue when column name not in parquet schema is passed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 2
.
Benchmark suite | Current: 932e39d | Previous: d77a208 | Ratio |
---|---|---|---|
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[1-incremental-pkstr-sknone-norcf_V1] |
1.0810036052605032 iter/sec (stddev: 0 ) |
2.2521535582165826 iter/sec (stddev: 0 ) |
2.08 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[1-incremental-pkstr-sknone-norcf_V2] |
1.073704052871053 iter/sec (stddev: 0 ) |
2.264678604965184 iter/sec (stddev: 0 ) |
2.11 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[2-incremental-pkstr-skstr-norcf_V1] |
1.060370284060169 iter/sec (stddev: 0 ) |
2.2689989361845813 iter/sec (stddev: 0 ) |
2.14 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[2-incremental-pkstr-skstr-norcf_V2] |
1.067061887040253 iter/sec (stddev: 0 ) |
2.266440743018205 iter/sec (stddev: 0 ) |
2.12 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[3-incremental-pkstr-multiskstr-norcf_V1] |
1.0861229707623132 iter/sec (stddev: 0 ) |
2.18698162316299 iter/sec (stddev: 0 ) |
2.01 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[3-incremental-pkstr-multiskstr-norcf_V2] |
1.0873349355706126 iter/sec (stddev: 0 ) |
2.2595342936230303 iter/sec (stddev: 0 ) |
2.08 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[5-incremental-decimal-pk-simple_V1] |
1.0849034327942895 iter/sec (stddev: 0 ) |
2.253116921393015 iter/sec (stddev: 0 ) |
2.08 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[5-incremental-decimal-pk-simple_V2] |
0.9345476087354807 iter/sec (stddev: 0 ) |
2.2460476215775427 iter/sec (stddev: 0 ) |
2.40 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[6-incremental-integer-pk-simple_V1] |
1.0529803093998353 iter/sec (stddev: 0 ) |
2.2710662045823544 iter/sec (stddev: 0 ) |
2.16 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[7-incremental-timestamp-pk-simple_V1] |
1.091837877291071 iter/sec (stddev: 0 ) |
2.2212402859331584 iter/sec (stddev: 0 ) |
2.03 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[7-incremental-timestamp-pk-simple_V2] |
1.0553010952581514 iter/sec (stddev: 0 ) |
2.1956702695295776 iter/sec (stddev: 0 ) |
2.08 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[8-incremental-decimal-timestamp-pk-multi_V1] |
1.0384340841016202 iter/sec (stddev: 0 ) |
2.264246475800043 iter/sec (stddev: 0 ) |
2.18 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[8-incremental-decimal-timestamp-pk-multi_V2] |
1.0439911837241 iter/sec (stddev: 0 ) |
2.215446181611629 iter/sec (stddev: 0 ) |
2.12 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[9-incremental-decimal-pk-multi-dup_V1] |
1.0728523039100528 iter/sec (stddev: 0 ) |
2.294583661883228 iter/sec (stddev: 0 ) |
2.14 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[10-incremental-decimal-pk-partitionless_V1] |
1.02521479495587 iter/sec (stddev: 0 ) |
2.2115377574614943 iter/sec (stddev: 0 ) |
2.16 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[10-incremental-decimal-pk-partitionless_V2] |
1.076874352334765 iter/sec (stddev: 0 ) |
2.253822151056081 iter/sec (stddev: 0 ) |
2.09 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[11-incremental-decimal-hash-bucket-single_V1] |
0.8820309461810104 iter/sec (stddev: 0 ) |
2.2273418119238033 iter/sec (stddev: 0 ) |
2.53 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[11-incremental-decimal-hash-bucket-single_V2] |
1.0530298370486406 iter/sec (stddev: 0 ) |
2.2643017544376827 iter/sec (stddev: 0 ) |
2.15 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[12-incremental-decimal-single-hash-bucket_V1] |
1.029124782968862 iter/sec (stddev: 0 ) |
2.297777655168341 iter/sec (stddev: 0 ) |
2.23 |
deltacat/tests/compute/test_compact_partition_incremental.py::test_compact_partition_incremental[12-incremental-decimal-single-hash-bucket_V2] |
1.1820900091113142 iter/sec (stddev: 0 ) |
2.6295975732771018 iter/sec (stddev: 0 ) |
2.22 |
This comment was automatically generated by workflow using github-action-benchmark.
DmitriGekhtman
approved these changes
Jul 19, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.