diff --git a/README.md b/README.md index e082677c..0d12beb3 100644 --- a/README.md +++ b/README.md @@ -3,7 +3,7 @@ English | [中文](docs/zh_CN/ch1) ## Overview -Qualitis is a one-stop data quality management platform that supports quality verification, notification, and management for various datasource. It is used to solve various data quality problems caused by data processing. +Qualitis is a data quality management platform that supports quality verification, notification, and management for various datasource. It is used to solve various data quality problems caused by data processing. Based on Spring Boot, Qualitis submits quality model task to [Linkis](https://github.com/WeBankFinTech/Linkis) platform. It provides functions such as data quality model construction, data quality model execution, data quality verification, reports of data quality generation and so on. diff --git a/docs/en_US/ch1/Linkis Access Guide.md b/docs/en_US/ch1/Linkis Access Guide.md new file mode 100644 index 00000000..00357331 --- /dev/null +++ b/docs/en_US/ch1/Linkis Access Guide.md @@ -0,0 +1,31 @@ +# How to get Linkis Token + +## Linkis Configuration +### Add Linkis Token +``` +cd $LINKIS_HOME/linkis-gateway/conf +vim token.properties +``` + +Add line as follow: +``` +QUALITIS-AUTH=* +``` + +### Modify Linkis Default Queue of User +1. Login in DataSphere Studio using the user same as qualitis. + +2. Enter 'Linkis Console' -> Settings -> wds.linkis.yarnqueue, and then set a queue in yarn. + +### Add Lib of MySQL +``` +cd $LINKIS_HOME/linkis-ujes-spark-enginemanager/lib +``` +Add MySQL Lib in it. + +At last, restart Linkis. + +## Qualitis Configuration +Type in 'QUALITIS-AUTH' in the blank of Linkis Token: +Here is an example： +![](../../../images/en_US/ch1/ClusterConfigurationDemo.png) \ No newline at end of file diff --git a/docs/en_US/ch1/QuickDeploy.md b/docs/en_US/ch1/QuickDeploy.md index 69100513..54c013b7 100644 --- a/docs/en_US/ch1/QuickDeploy.md +++ b/docs/en_US/ch1/QuickDeploy.md @@ -7,8 +7,8 @@ JDK (1.8.0_141) Hadoop (2.7.2) Hive (1.2.1) Zookeeper (3.4.9) -Linkis (0.9.0), Spark engine required.[How to install Linkis?](https://github.com/WeBankFinTech/Linkis) -DataSphereStudio (0.5.0) Optional. If you want to use workflow, it it required. [How to install DataSphereStudio?](https://github.com/WeBankFinTech/DataSphereStudio) +Linkis (0.9.1), Spark engine required.[How to install Linkis?](https://github.com/WeBankFinTech/Linkis) +DataSphereStudio (0.6.0) Optional. If you want to use workflow, it it required. [How to install DataSphereStudio?](https://github.com/WeBankFinTech/DataSphereStudio) ## 2. Download [Download](https://github.com/WeBankFinTech/Qualitis/releases) @@ -83,10 +83,7 @@ Cluster type HiveMetaStore address HiveServer2 address Linkis address -Linkis Token -Hive database address -Hive database username -Hive database password +Linkis Token(How to get Linkis Token：[Linkis Access Guide](Linkis Access Guide.md)) Here is an example: ![](../../../images/en_US/ch1/ClusterConfigurationDemo.png) diff --git a/docs/en_US/ch1/UpdateGuide.md b/docs/en_US/ch1/UpdateGuide.md new file mode 100644 index 00000000..123775a1 --- /dev/null +++ b/docs/en_US/ch1/UpdateGuide.md @@ -0,0 +1,10 @@ +# Update Guide + +## update from 0.5.0 to 0.6.0 +Download code of 0.6.0 version. + +Execute the following code below: +``` +mysql -u {USERNAME} -p {PASSWORD} -h {IP} --default-character-set=utf8 +source conf/database/update/update_0.5.0-0.6.0.sql +``` \ No newline at end of file diff --git "a/docs/zh_CN/ch1/\345\215\207\347\272\247\346\214\207\345\215\227.md" "b/docs/zh_CN/ch1/\345\215\207\347\272\247\346\214\207\345\215\227.md" new file mode 100644 index 00000000..7c1b8b4d --- /dev/null +++ "b/docs/zh_CN/ch1/\345\215\207\347\272\247\346\214\207\345\215\227.md" @@ -0,0 +1,10 @@ +# 升级指南 + +## 0.5.0 升级 0.6.0 +下载0.6.0版本代码，并解压。 + +执行以下数据库脚本: + ``` + mysql -u {USERNAME} -p {PASSWORD} -h {IP} --default-character-set=utf8 + source conf/database/update/update_0.5.0-0.6.0.sql + ``` \ No newline at end of file diff --git "a/docs/zh_CN/ch1/\345\277\253\351\200\237\346\220\255\345\273\272\346\211\213\345\206\214.md" "b/docs/zh_CN/ch1/\345\277\253\351\200\237\346\220\255\345\273\272\346\211\213\345\206\214.md" index 4f28dd20..c4c11243 100644 --- "a/docs/zh_CN/ch1/\345\277\253\351\200\237\346\220\255\345\273\272\346\211\213\345\206\214.md" +++ "b/docs/zh_CN/ch1/\345\277\253\351\200\237\346\220\255\345\273\272\346\211\213\345\206\214.md" @@ -7,8 +7,8 @@ JDK (1.8.0_141) Hadoop (2.7.2) Hive (1.2.1) Zookeeper (3.4.9) -Linkis（0.9.0), 必装Spark引擎。[如何安装Linkis](https://github.com/WeBankFinTech/Linkis) -DataSphereStudio (0.5.0) 可选. 如果你想使用工作流，必装DataSphereStudio [如何安装DataSphereStudio?](https://github.com/WeBankFinTech/DataSphereStudio) +Linkis（0.9.1), 必装Spark引擎。[如何安装Linkis](https://github.com/WeBankFinTech/Linkis) +DataSphereStudio (0.6.0) 可选. 如果你想使用工作流，必装DataSphereStudio [如何安装DataSphereStudio?](https://github.com/WeBankFinTech/DataSphereStudio) ## 二、安装包下载 [下载](https://github.com/WeBankFinTech/Qualitis/releases) @@ -83,10 +83,7 @@ sh bin/start.sh HiveMetaStore地址 HiveServer2地址 Linkis地址 -Linkis Token -Hive数据库地址(用于连接HiveMetaStore的MySQL数据库) -Hive数据库用户名(用于连接HiveMetaStore的MySQL数据库) -Hive数据库密码(用于连接HiveMetaStore的MySQL数据库) +Linkis Token(Linkis Token配置方式请查看文档：[接入Linkis文档](接入Linkis文档.md)) 可参考以下例子： ![](../../../images/zh_CN/ch1/规则配置样例.png) diff --git "a/docs/zh_CN/ch1/\346\216\245\345\205\245Linkis\346\226\207\346\241\243.md" "b/docs/zh_CN/ch1/\346\216\245\345\205\245Linkis\346\226\207\346\241\243.md" new file mode 100644 index 00000000..69582479 --- /dev/null +++ "b/docs/zh_CN/ch1/\346\216\245\345\205\245Linkis\346\226\207\346\241\243.md" @@ -0,0 +1,31 @@ +# 接入Linkis文档 + +## Linkis配置 +### 增加Linkis Token +``` +cd $LINKIS_HOME/linkis-gateway/conf +vim token.properties +``` + +增加以下行： +``` +QUALITIS-AUTH=* +``` + +### 修改用户默认提交队列 +1. 在DataSphere Studio中登录Qualitis同名的用户名 + +2. 进入Linkis管理台 -> 设置 -> yarn队列名中，并设置一个可用的yarn队列 + +### 加入MySQL依赖包 +``` +cd $LINKIS_HOME/linkis-ujes-spark-enginemanager/lib +``` +并加入mysql依赖包 + +最后，重启Linkis + +## Qualitis配置 +在Linkis Token中填入'QUALITIS-AUTH' +如下图所示： +![](../../../images/zh_CN/ch1/规则配置样例.png) \ No newline at end of file diff --git a/images/en_US/ch1/ClusterConfigurationDemo.png b/images/en_US/ch1/ClusterConfigurationDemo.png index e4f9f8b5..5e6bc0e9 100644 Binary files a/images/en_US/ch1/ClusterConfigurationDemo.png and b/images/en_US/ch1/ClusterConfigurationDemo.png differ diff --git "a/images/zh_CN/ch1/\350\247\204\345\210\231\351\205\215\347\275\256\346\240\267\344\276\213.png" "b/images/zh_CN/ch1/\350\247\204\345\210\231\351\205\215\347\275\256\346\240\267\344\276\213.png" index 0cd500b5..ede080ca 100644 Binary files "a/images/zh_CN/ch1/\350\247\204\345\210\231\351\205\215\347\275\256\346\240\267\344\276\213.png" and "b/images/zh_CN/ch1/\350\247\204\345\210\231\351\205\215\347\275\256\346\240\267\344\276\213.png" differ