Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow parsing ISO8601 timestamp in Delta lake transaction log + Update docker-images to 108 #24635

Merged
merged 3 commits into from
Jan 8, 2025

Conversation

ebyhr
Copy link
Member

@ebyhr ebyhr commented Jan 7, 2025

Description

Relates to trinodb/docker-images#221

protocol entry has been changed when creates a table on Spark:

{"protocol":{"minReaderVersion":2,"minWriterVersion":5}}

{"protocol":{"minReaderVersion":2,"minWriterVersion":7,"writerFeatures":["columnMapping","changeDataFeed","appendOnly","invariants"]}}

Fixes #24647

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

@cla-bot cla-bot bot added the cla-signed label Jan 7, 2025
@github-actions github-actions bot added the delta-lake Delta Lake connector label Jan 7, 2025
"Both reader and writer features must have the same value for 'columnMapping'. reader: %s, writer: %s", supportsColumnMappingReader, supportsColumnMappingWriter);
if (!supportsColumnMappingReader) {
return ColumnMappingMode.NONE;
if (protocolEntry.supportsReaderFeatures() && protocolEntry.supportsWriterFeatures()) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is required because Delta protocol allows a combination of minReaderVersion:2 and minWriterVersion:7.

@ebyhr ebyhr changed the title Update docker-images to 108 Allow parsing ISO8601 timestamp in Delta lake transaction log + Update docker-images to 108 Jan 8, 2025
@ebyhr ebyhr marked this pull request as ready for review January 8, 2025 08:01
@ebyhr ebyhr merged commit 3fffb1b into trinodb:master Jan 8, 2025
95 checks passed
@ebyhr ebyhr deleted the ebi/docker-108 branch January 8, 2025 23:08
@github-actions github-actions bot added this to the 469 milestone Jan 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cla-signed delta-lake Delta Lake connector
Development

Successfully merging this pull request may close these issues.

Delta Lake connector doesn't support ISO8601 formatted timestamp for partition values
2 participants