[FLINK-34677][cdc] Synchronize Chinese documentations for Flink CDC (a…

…pache#3169)
czy006 · Mar 19, 2024 · 88c9cf2 · 88c9cf2
1 parent 8098a9c
commit 88c9cf2
Show file tree

Hide file tree

Showing 32 changed files with 833 additions and 68 deletions.
diff --git a/docs/content.zh/docs/connectors/doris.md b/docs/content.zh/docs/connectors/doris.md
@@ -24,13 +24,12 @@ specific language governing permissions and limitations
 under the License.
 -->
 
-# Doris Pipeline Connector
-
-This article introduces of Doris Pipeline Connector
+# Doris Connector
 
+This article introduces of Doris Connector
 
 ## Example
-----------------
+
 
 ```yaml
 source:
@@ -50,8 +49,7 @@ pipeline:
 
 ```
 
-## Pipeline options
-----------------
+## Connector Options
 
 <div class="highlight">
 <table class="colwidths-auto docutils">
@@ -180,17 +178,16 @@ pipeline:
      </tbody>
 </table>
 </div>
-## Data Type Mapping
 
-----------------
+## Data Type Mapping
 
 <div class="wy-table-responsive">
 <table class="colwidths-auto docutils">
     <thead>
       <tr>
-        <th class="text-left" style="width:10%;">CDC type</th>
-        <th class="text-left" style="width:30%;">Doris type<a href="https://doris.apache.org/docs/dev/sql-manual/sql-reference/Data-Types/BOOLEAN/"></a></th>
-        <th class="text-left" style="width:60%;">NOTE</th>
+        <th class="text-left" style="width:10%;">Flink CDC Type</th>
+        <th class="text-left" style="width:30%;"><a href="https://doris.apache.org/docs/dev/sql-manual/sql-reference/Data-Types/BOOLEAN/">Doris Type</a></th>
+        <th class="text-left" style="width:60%;">Note</th>
       </tr>
     </thead>
     <tbody>

diff --git a/....zh/docs/connectors/legacy-flink-cdc-sources/datastream-api-package-guidance.md b/....zh/docs/connectors/legacy-flink-cdc-sources/datastream-api-package-guidance.md
@@ -1,6 +1,6 @@
 ---
 title: "DataStream API Package Guidance"
-weight: 999
+weight: 998
 type: docs
 aliases:
 - /connectors/legacy-flink-cdc-sources/datastream-api-package-guidance

diff --git a/docs/content.zh/docs/connectors/legacy-flink-cdc-sources/db2-cdc.md b/docs/content.zh/docs/connectors/legacy-flink-cdc-sources/db2-cdc.md
@@ -53,7 +53,7 @@ using a build automation tool (such as Maven or SBT) and SQL Client with SQL JAR
 Download flink-sql-connector-db2-cdc-3.0-SNAPSHOT.jar and 
 put it under `<FLINK_HOME>/lib/`.
 
-**Note:** flink-sql-connector-db2-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as 
+**Note:** flink-sql-connector-db2-cdc-XXX-SNAPSHOT version is the code corresponding to the development branch. Users should use the released version, such as
 [flink-sql-connector-db2-cdc-3.0.0.jar](https://mvnrepository.com/artifact/com.ververica/flink-connector-db2-cdc),
 the released version will be available in the Maven central warehouse.
 

diff --git a/docs/content.zh/docs/connectors/legacy-flink-cdc-sources/mongodb-cdc.md b/docs/content.zh/docs/connectors/legacy-flink-cdc-sources/mongodb-cdc.md
@@ -52,7 +52,7 @@ Setup MongoDB
 - MongoDB version
 
   MongoDB version >= 3.6 <br>
-  We use [change streams](https://docs.mongodb.com/manual/changeStreams/) feature (new in version 3.6) to capture change data.
+We use [change streams](https://docs.mongodb.com/manual/changeStreams/) feature (new in version 3.6) to capture change data.
 
 - Cluster Deployment
 

diff --git a/docs/content.zh/docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc.md b/docs/content.zh/docs/connectors/legacy-flink-cdc-sources/oceanbase-cdc.md
@@ -154,7 +154,7 @@ Flink SQL> CREATE TABLE orders (
 );
 ```
 
-You can also try the quickstart tutorial that sync data from OceanBase to Elasticsearch, please refer [Flink CDC Tutorial](<{{ ref "docs/connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial" }}>) for more information.
+You can also try the quickstart tutorial that sync data from OceanBase to Elasticsearch, please refer [Flink CDC Tutorial]({{< ref "docs/connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial" >}}) for more information.
 
 Connector Options
 ----------------

diff --git a/docs/content.zh/docs/connectors/legacy-flink-cdc-sources/overview.md b/docs/content.zh/docs/connectors/legacy-flink-cdc-sources/overview.md
@@ -29,6 +29,7 @@ under the License.
 Flink CDC sources is a set of source connectors for <a href="https://flink.apache.org/">Apache Flink<sup>®</sup></a>, ingesting changes from different databases using change data capture (CDC).
 Some CDC sources integrate Debezium as the engine to capture data changes. So it can fully leverage the ability of Debezium. See more about what is [Debezium](https://github.com/debezium/debezium).
 
+You can also read [tutorials]({{< ref "docs/connectors/legacy-flink-cdc-sources/tutorials/mysql-postgres-tutorial" >}}) about how to use these sources.
 
 {{< img src="/fig/cdc-flow.png" width="600px" alt="Flink CDC" >}}
 

diff --git a/...quickstart/using-legacy-sources/_index.md → ...acy-flink-cdc-sources/tutorials/_index.md b/...quickstart/using-legacy-sources/_index.md → ...acy-flink-cdc-sources/tutorials/_index.md
@@ -1,7 +1,7 @@
 ---
-title: "Using Legacy Sources"
+title: Tutorials
 bookCollapseSection: true
-weight: 3
+weight: 999
 ---
 <!--
 Licensed to the Apache Software Foundation (ASF) under one

diff --git a/...ces/build-real-time-data-lake-tutorial.md → ...als/build-real-time-data-lake-tutorial.md b/...ces/build-real-time-data-lake-tutorial.md → ...als/build-real-time-data-lake-tutorial.md
@@ -3,7 +3,7 @@ title: "Building a Real-time Data Lake with Flink CDC"
 weight: 999
 type: docs
 aliases:
-- /development/build-real-time-data-lake-tutorial.html
+- /connectors/legacy-flink-cdc-sources/tutorials/build-real-time-data-lake-tutorial.html
 
 ---
 

diff --git a/...tart/using-legacy-sources/db2-tutorial.md → ...ink-cdc-sources/tutorials/db2-tutorial.md b/...tart/using-legacy-sources/db2-tutorial.md → ...ink-cdc-sources/tutorials/db2-tutorial.md
@@ -3,7 +3,7 @@ title: "Db2 Tutorial"
 weight: 8
 type: docs
 aliases:
-- /get-started/quickstart/using-legacy-sources/db2-tutorial.html
+- /connectors/legacy-flink-cdc-sources/tutorials/db2-tutorial.html
 ---
 <!--
 Licensed to the Apache Software Foundation (ASF) under one

diff --git a/.../using-legacy-sources/mongodb-tutorial.md → ...cdc-sources/tutorials/mongodb-tutorial.md b/.../using-legacy-sources/mongodb-tutorial.md → ...cdc-sources/tutorials/mongodb-tutorial.md
@@ -3,7 +3,7 @@ title: "MongoDB Tutorial"
 weight: 1
 type: docs
 aliases:
-- /get-started/quickstart/using-legacy-sources/mongodb-tutorial.html
+- /connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.html
 ---
 <!--
 Licensed to the Apache Software Foundation (ASF) under one

diff --git a/...legacy-sources/mysql-postgres-tutorial.md → ...rces/tutorials/mysql-postgres-tutorial.md b/...legacy-sources/mysql-postgres-tutorial.md → ...rces/tutorials/mysql-postgres-tutorial.md
@@ -3,7 +3,7 @@ title: "MySQL & PostgreSQL Tutorial"
 weight: 2
 type: docs
 aliases:
-- /get-started/quickstart/using-legacy-sources/mysql-postgres-tutorial.html
+- /connectors/legacy-flink-cdc-sources/tutorials/mysql-postgres-tutorial.html
 ---
 <!--
 Licensed to the Apache Software Foundation (ASF) under one

diff --git a/...sing-legacy-sources/oceanbase-tutorial.md → ...c-sources/tutorials/oceanbase-tutorial.md b/...sing-legacy-sources/oceanbase-tutorial.md → ...c-sources/tutorials/oceanbase-tutorial.md
@@ -3,7 +3,7 @@ title: "OceanBase Tutorial"
 weight: 3
 type: docs
 aliases:
-- /get-started/quickstart/using-legacy-sources/oceanbase-tutorial.html
+- /connectors/legacy-flink-cdc-sources/tutorials/oceanbase-tutorial.html
 ---
 <!--
 Licensed to the Apache Software Foundation (ASF) under one

diff --git a/...t/using-legacy-sources/oracle-tutorial.md → ...-cdc-sources/tutorials/oracle-tutorial.md b/...t/using-legacy-sources/oracle-tutorial.md → ...-cdc-sources/tutorials/oracle-tutorial.md
@@ -3,7 +3,7 @@ title: "Oracle Tutorial"
 weight: 4
 type: docs
 aliases:
-- /get-started/quickstart/using-legacy-sources/oracle-tutorial.html
+- /connectors/legacy-flink-cdc-sources/tutorials/oracle-tutorial.html
 ---
 <!--
 Licensed to the Apache Software Foundation (ASF) under one

diff --git a/...using-legacy-sources/polardbx-tutorial.md → ...dc-sources/tutorials/polardbx-tutorial.md b/...using-legacy-sources/polardbx-tutorial.md → ...dc-sources/tutorials/polardbx-tutorial.md
@@ -3,7 +3,7 @@ title: "PolarDB-X Tutorial"
 weight: 5
 type: docs
 aliases:
-- /get-started/quickstart/using-legacy-sources/mongodb-tutorial.html
+- /connectors/legacy-flink-cdc-sources/tutorials/mongodb-tutorial.html
 ---
 <!--
 Licensed to the Apache Software Foundation (ASF) under one

diff --git a/...sing-legacy-sources/sqlserver-tutorial.md → ...c-sources/tutorials/sqlserver-tutorial.md b/...sing-legacy-sources/sqlserver-tutorial.md → ...c-sources/tutorials/sqlserver-tutorial.md
@@ -3,7 +3,7 @@ title: "SqlServer Tutorial"
 weight: 6
 type: docs
 aliases:
-- /get-started/quickstart/using-legacy-sources/sqlserver-tutorial.html
+- /connectors/legacy-flink-cdc-sources/tutorials/sqlserver-tutorial.html
 ---
 <!--
 Licensed to the Apache Software Foundation (ASF) under one

diff --git a/...art/using-legacy-sources/tidb-tutorial.md → ...nk-cdc-sources/tutorials/tidb-tutorial.md b/...art/using-legacy-sources/tidb-tutorial.md → ...nk-cdc-sources/tutorials/tidb-tutorial.md
@@ -3,7 +3,7 @@ title: "TiDB Tutorial"
 weight: 7
 type: docs
 aliases:
-- /get-started/quickstart/using-legacy-sources/tidb-tutorial.html
+- /connectors/legacy-flink-cdc-sources/tutorials/tidb-tutorial.html
 ---
 <!--
 Licensed to the Apache Software Foundation (ASF) under one

diff --git a/docs/content.zh/docs/connectors/mysql.md b/docs/content.zh/docs/connectors/mysql.md
@@ -24,16 +24,15 @@ specific language governing permissions and limitations
 under the License.
 -->
 
-# MySQL CDC Pipeline Connector
+# MySQL Connector
 
-The MySQL CDC Pipeline Connector allows for reading snapshot data and incremental data from MySQL database and provides end-to-end full-database data synchronization capabilities.
-This document describes how to setup the MySQL CDC Pipeline connector.
+MySQL connector allows reading snapshot data and incremental data from MySQL database and provides end-to-end full-database data synchronization capabilities.
+This document describes how to setup the MySQL connector.
 
 
-How to create Pipeline
-----------------
+## Example
 
-The pipeline for reading data from MySQL and sink to Doris can be defined as follows:
+An example of the pipeline for reading data from MySQL and sink to Doris can be defined as follows:
 
 ```yaml
 source:
@@ -58,8 +57,7 @@ pipeline:
    parallelism: 4
 ```
 
-Pipeline Connector Options
-----------------
+## Connector Options
 
 <div class="highlight">
 <table class="colwidths-auto docutils">
@@ -245,8 +243,7 @@ Pipeline Connector Options
 </table>
 </div>
 
-Startup Reading Position
---------
+## Startup Reading Position
 
 The config option `scan.startup.mode` specifies the startup mode for MySQL CDC consumer. The valid enumerations are:
 
@@ -259,16 +256,15 @@ The config option `scan.startup.mode` specifies the startup mode for MySQL CDC c
 - `timestamp`: Skip snapshot phase and start reading binlog events from a specific timestamp.
 
 
-Data Type Mapping
-----------------
+## Data Type Mapping
 
 <div class="wy-table-responsive">
 <table class="colwidths-auto docutils">
     <thead>
       <tr>
-        <th class="text-left" style="width:30%;">MySQL type<a href="https://dev.mysql.com/doc/man/8.0/en/data-types.html"></a></th>
-        <th class="text-left" style="width:10%;">CDC type</th>
-        <th class="text-left" style="width:60%;">NOTE</th>
+        <th class="text-left" style="width:30%;"><a href="https://dev.mysql.com/doc/man/8.0/en/data-types.html">MySQL type</a></th>
+        <th class="text-left" style="width:10%;">Flink CDC type</th>
+        <th class="text-left" style="width:60%;">Note</th>
       </tr>
     </thead>
     <tbody>

diff --git a/docs/content.zh/docs/connectors/overview.md b/docs/content.zh/docs/connectors/overview.md
@@ -24,21 +24,33 @@ specific language governing permissions and limitations
 under the License.
 -->
 
-# Pipeline Connectors Of CDC Streaming ELT Framework
+# Connectors
+
+Flink CDC provides several source and sink connectors to interact with external
+systems. You can use these connectors out-of-box, by adding released JARs to
+your Flink CDC environment, and specifying the connector in your YAML pipeline
+definition.
 
 ## Supported Connectors
 
-| Connector                                   | Database                                                                                                                                                                                                                                                                                                                                                                                               | 
-|---------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| [doris-pipeline](doris-pipeline.md)         | <li> [Doris](https://doris.apache.org/): 1.2.x, 2.x.x                                                                                                                                                                                                                                                                                                                                                  | 
-| [mysql-pipeline](mysql-pipeline.md)         | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | 
-| [starrocks-pipeline](starrocks-pipeline.md) | <li> [StarRocks](https://www.starrocks.io/): 2.x, 3.x                                                                                                                                                                                                                                                                                                                                                  | 
+| Connector                                            | Supported Type | External System                                                                                                                                                                                                                                                                                                                                                                                       | 
+|------------------------------------------------------|----------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| [Apache Doris]({{< ref "docs/connectors/doris" >}})  | Sink           | <li> [Apache Doris](https://doris.apache.org/): 1.2.x, 2.x.x                                                                                                                                                                                                                                                                                                                                          | 
+| [MySQL]({{< ref "docs/connectors/mysql" >}})         | Source         | <li> [MySQL](https://dev.mysql.com/doc): 5.6, 5.7, 8.0.x <li> [RDS MySQL](https://www.aliyun.com/product/rds/mysql): 5.6, 5.7, 8.0.x <li> [PolarDB MySQL](https://www.aliyun.com/product/polardb): 5.6, 5.7, 8.0.x <li> [Aurora MySQL](https://aws.amazon.com/cn/rds/aurora): 5.6, 5.7, 8.0.x <li> [MariaDB](https://mariadb.org): 10.x <li> [PolarDB X](https://github.com/ApsaraDB/galaxysql): 2.0.1 | 
+| [StarRocks]({{< ref "docs/connectors/starrocks" >}}) | Sink           | <li> [StarRocks](https://www.starrocks.io/): 2.x, 3.x                                                                                                                                                                                                                                                                                                                                                 |
+
+## Develop Your Own Connector
+
+If provided connectors cannot fulfill your requirement, you can always develop
+your own connector to get your external system involved in Flink CDC pipelines.
+Check out [Flink CDC APIs]({{< ref "docs/developer-guide/understand-flink-cdc-api" >}})
+to learn how to develop your own connectors.
 
-## Supported Flink Versions
-The following table shows the version mapping between Flink<sup>®</sup> CDC Pipeline and Flink<sup>®</sup>:
+## Legacy Flink CDC Sources
 
-|    Flink<sup>®</sup> CDC Version    |                                                                                                      Flink<sup>®</sup> Version                                                                                                       |
-|:-----------------------------------:|:------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
-| <font color="DarkCyan">3.0.*</font> | <font color="MediumVioletRed">1.14.\*</font>, <font color="MediumVioletRed">1.15.\*</font>, <font color="MediumVioletRed">1.16.\*</font>, <font color="MediumVioletRed">1.17.\*</font>, <font color="MediumVioletRed">1.18.\*</font> |
+Flink CDC sources introduces before 3.0 are still available as normal Flink 
+connector sources. You can find more details in the 
+[overview page]({{< ref "docs/connectors/legacy-flink-cdc-sources/overview" >}})
+of legacy Flink CDC sources.
 
 {{< top >}}
diff --git a/docs/content.zh/docs/connectors/starrocks.md b/docs/content.zh/docs/connectors/starrocks.md
@@ -24,17 +24,16 @@ specific language governing permissions and limitations
 under the License.
 -->
 
-# StarRocks Pipeline Connector
+# StarRocks Connector
 
-The StarRocks Pipeline connector can be used as the *Data Sink* of the pipeline, and write data to [StarRocks](https://github.com/StarRocks/starrocks). This document describes how to set up the StarRocks Pipeline connector.
+StarRocks connector can be used as the *Data Sink* of the pipeline, and write data to [StarRocks](https://github.com/StarRocks/starrocks). This document describes how to set up the StarRocks connector.
 
 ## What can the connector do?
 * Create table automatically if not exist
 * Schema change synchronization
 * Data synchronization
 
-How to create Pipeline
-----------------
+## Example
 
 The pipeline for reading data from MySQL and sink to StarRocks can be defined as follows:
 
@@ -62,8 +61,8 @@ pipeline:
    parallelism: 2
 ```
 
-Pipeline Connector Options
-----------------
+## Connector Options
+
 <div class="highlight">
 <table class="colwidths-auto docutils">
    <thead>
@@ -220,8 +219,8 @@ Pipeline Connector Options
 </table>    
 </div>
 
-Usage Notes
---------
+
+## Usage Notes
 
 * Only support StarRocks primary key table, so the source table must have primary keys.
 
@@ -244,15 +243,14 @@ Usage Notes
   to write data to StarRocks. You can see [sink documentation](https://github.com/StarRocks/starrocks-connector-for-apache-flink/blob/main/docs/content/connector-sink.md)
   for how it works. 
 
-Data Type Mapping
-----------------
+## Data Type Mapping
 <div class="wy-table-responsive">
 <table class="colwidths-auto docutils">
     <thead>
       <tr>
-        <th class="text-left">CDC type</th>
+        <th class="text-left">Flink CDC type</th>
         <th class="text-left">StarRocks type</th>
-        <th class="text-left" style="width:60%;">NOTE</th>
+        <th class="text-left" style="width:60%;">Note</th>
       </tr>
     </thead>
     <tbody>