-
Notifications
You must be signed in to change notification settings - Fork 233
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Improve]Separate the record and schema-change in JsonDebeziumSchemaSerializer #279
Conversation
It seems that many JsonDebezium related classes have been added. Would it be better to create a separate package below? |
...or/src/main/java/org/apache/doris/flink/sink/writer/serializer/JsonDebeziumSchemaChange.java
Outdated
Show resolved
Hide resolved
6b915e6
to
223f870
Compare
.../java/org/apache/doris/flink/sink/writer/serializer/jsondebezium/JsonDebeziumDataChange.java
Show resolved
Hide resolved
Please also rebase the master to run the latest workflow. Thanks |
223f870
to
c339b9e
Compare
Thanks for your suggestion, just done. |
Thanks Dongliang, can you add ITCase to the function of automatically adding tables? like this https://github.com/apache/doris-flink-connector/blob/master/flink-doris-connector/src/test/java/org/apache/doris/flink/tools/cdc/MySQLDorisE2ECase.java |
Thanks for your suggestion, in the last commit, I have added. |
fa9163c
to
63f5bbe
Compare
63f5bbe
to
21501ae
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Proposed changes
Issue Number: close #xxx
Problem Summary:
The original
JsonDebeziumSchemaSerializer
contains two serialization logics: record and schema change, which makes the entireJsonDebeziumSchemaSerializer
code logic particularly confusing and complex, and the maintenance cost is high.Now record and schema change are separated into
JsonDebeziumRecord
andJsonDebeziumSchemaChange
.There are two versions of schema implementation in
JsonDebeziumSchemaChange
. One is the old version(JsonDebeziumSchemaChangeImpl
), which has limited support for schema change in actual business; the other is a new version(JsonDebeziumSchemaChangeImplV2
), which supports newer versions and can support multi-column changes and field renaming.Checklist(Required)
Further comments
If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...