Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix](cdc)fix the error mapping default values from RDBMS to Doris #308

Closed
wants to merge 4 commits into from

Conversation

vinlee19
Copy link
Contributor

@vinlee19 vinlee19 commented Jan 26, 2024

Proposed changes

Issue Number: close #xxx

Problem Summary:

Currently, Doris only supports current_timestamp as the default function.
In relational databases, the composition of different default values varies. Here are some examples:
1.MySQL
In MySQL, default values for numeric types are represented as 1.0 or 0, while for string and date types, they are enclosed in single quotes.
In MySQL5.7.x,you can create SQL statements using the following syntax:

create table test_str
(
    id         int auto_increment primary key,
    t2_varchar varchar(10) default '''doris'''''   null,
    t1_varchar varchar(10) default 'doris'''       null,
    t3_varchar varchar(20) default '''''doris''''' null,
    t_int int default 1,
   t_float default 1.1
);

In MySQL8.0.x,you can create table with default function.

create table t1
(
    i int auto_increment
        primary key,
    c varchar(10) default ''                              null,
    f float       default ((rand() * rand()))             null,
    b binary(16)  default (uuid_to_bin(uuid()))           null,
    d date        default ((curdate() + interval 1 year)) null,
    j json        default (json_array())                  null
);

2.Oracle
In Oracle, character types are enclosed in single quotes, while the default values for timestamps and dates are obtained through the TO_TIMESTAMP and TO_DATE functions, respectively. Here are some specific examples:

create table TEST_ALL_TYPES
(
    ID              NUMBER not null constraint TEST_ALL_TYPES_PK primary key,
    N1              NUMBER,
    NUM2        NUMBER(5, -2)                default (100 / 3),
    NUM4        NUMBER(5, 7),
    T1              DATE                         default to_date('2010-2-12 10:20:30', 'YYYY-MM-DD HH24:MI:SS'),
    T2            TIMESTAMP(7)                 default TO_TIMESTAMP('2023-01-01 00:00:00.0000001'

3.PostgreSQL
In PostgreSQL, default values for character and numeric types are composed of the value itself, followed by :: and the type.

create table test_all_types
(
    id              integer        default 1 not null,
    char_value      char(100)      default 'doris'::bpchar,
    varchar_value   varchar(128)   default 'make doris greate'::character varying,
    date_value      date           default '2024-01-01'::date,
    smallint_value  smallint       default 12,
    int_value       integer        default 100,
    bigint_value    bigint         default 10000,
    timestamp_value timestamp      default '2024-01-01 01:01:01.000001'::timestamp without time zone,
    decimal_value   numeric(10, 3) default 1.23,
    bit_value       bit,
    ts_decimal      numeric(38)    default 12,
    text_value      text           default 'make doris greate'::text
);

4.SQLSever
In SQL Server, numeric types are enclosed in double parentheses (), while string and time types are enclosed in single parentheses ().

create table all_type
(
    id                   int not null primary key,
    name                 varchar(10)     default 'kevin',
    age                      int             default 1,
    smalldatetime_value  smalldatetime   default '2024-01-01 00:00:00.00000001',
    datetimeoffset_value datetimeoffset  default '2024-01-01 00:00:00.0000001',
    bit_value            bit             default 1,
    c1                   varchar(255)    default 12.11
)
go

In this PR, you can automatically map default values from MySQL, Oracle, PostgreSQL, and SQL Server to Doris default values. Currently, only the translation of the time default function current_timestamp is supported.

Checklist(Required)

  1. Does it affect the original behavior: (Yes/No/I Don't know)
  2. Has unit tests been added: (Yes/No/No Need)
  3. Has document been added or modified: (Yes/No/No Need)
  4. Does it need to update dependencies: (Yes/No)
  5. Are there any changes that cannot be rolled back: (Yes/No)

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@vinlee19 vinlee19 changed the title [fix](cdc) fix precision loss when sync table default value is current_timestamp [fix](cdc)fix the error mapping default values from RDBMS to Doris Feb 7, 2024
@vinlee19 vinlee19 closed this Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant