Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev2 #668

Closed
wants to merge 31 commits into from
Closed

Dev2 #668

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
477e6af
new proposal for data quality tool
guoyuepeng Jun 23, 2024
b73688e
repolish
guoyuepeng Jun 23, 2024
711fda5
enterprise job scheduler might retry or stop the downstream scheduler…
guoyuepeng Jun 24, 2024
51f4639
illustrate data quality etl phase, append after business phase
guoyuepeng Jun 24, 2024
f31a63b
elaborate integrate with business workflow
guoyuepeng Jun 24, 2024
76b4834
init DQDiagrams
guoyuepeng Jun 25, 2024
bcc32c7
add metric storage service in DQDiagrams
guoyuepeng Jun 25, 2024
572b761
triggered on demand
guoyuepeng Jun 26, 2024
5bb4a54
add more metrics
guoyuepeng Jun 26, 2024
87abf10
update arch diagram
guoyuepeng Jun 28, 2024
00f2aca
Merge branch 'master' of https://github.com/apache/griffin
guoyuepeng Jul 1, 2024
f622eb3
init two table diff result set
guoyuepeng Jul 1, 2024
7c8742d
data platform upgrades data quality checking pipeline
guoyuepeng Jul 2, 2024
b68dd52
typo
guoyuepeng Jul 2, 2024
a3c8999
update it as data quality constrains
guoyuepeng Jul 3, 2024
764b777
Merge branch 'master' of https://github.com/apache/griffin
guoyuepeng Jul 3, 2024
9caa6f1
Let's roll up a new page of Griffin. (#657)
toyboxman Jul 10, 2024
3605c26
Update the whole pom structure of Griffin-2.0.0
toyboxman Jul 23, 2024
5fab2b1
Update griffin submodules pom structure (#659)
toyboxman Aug 5, 2024
164a031
add griffin bom for third party dependencies management
guoyuepeng Aug 5, 2024
47d1e6e
standarize pom structure
guoyuepeng Aug 6, 2024
0703f8b
add griffin bom for third party dependencies management (#660)
guoyuepeng Aug 6, 2024
0bc04a5
Merge branch 'griffin-2.0.0-dev' of https://github.com/apache/griffin…
guoyuepeng Aug 6, 2024
294f3c7
Initialize base metric model (#662)
toyboxman Aug 28, 2024
4c07dc0
add metric test (#663)
guoyuepeng Sep 4, 2024
3973c4f
Griffin 2.0.0 dev (#664)
guoyuepeng Sep 6, 2024
4065cbf
Griffin 2.0.0 dev (#666)
guoyuepeng Sep 20, 2024
69fe18a
add service interface for metricD table. (#665)
toyboxman Oct 9, 2024
ecdc7a3
add service interface for metricD table.
toyboxman Sep 13, 2024
bd9fe40
Merge remote-tracking branch 'origin/griffin-2.0.0-dev' into dev2
toyboxman Oct 10, 2024
99bd9a1
add the process of exception.
toyboxman Oct 10, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions griffin-bom/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
this is for third party dependencies' management.
198 changes: 198 additions & 0 deletions griffin-bom/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>org.apache.griffin</groupId>
<artifactId>griffin</artifactId>
<version>2.0.0-SNAPSHOT</version>
</parent>

<groupId>org.apache.griffin</groupId>
<artifactId>griffin-bom</artifactId>
<version>2.0.0-SNAPSHOT</version>
<packaging>pom</packaging>
<name>Apache Griffin BOM ${project.version}</name>
<url>https://griffin.apache.org</url>

<properties>
<spring-boot.version>2.7.3</spring-boot.version>
<mybatis-plus.version>3.5.2</mybatis-plus.version>
<commons-codec.version>1.11</commons-codec.version>
<commons-logging.version>1.1.1</commons-logging.version>
<commons-lang3.version>3.12.0</commons-lang3.version>
<commons-httpclient>3.0.1</commons-httpclient>
<commons-beanutils.version>1.9.4</commons-beanutils.version>
<commons-configuration.version>1.10</commons-configuration.version>
<commons-email.version>1.5</commons-email.version>
<commons-collections4.version>4.3</commons-collections4.version>
<httpclient.version>4.5.13</httpclient.version>
<httpcore.version>4.4.15</httpcore.version>
<jackson.version>2.13.4</jackson.version>
<slf4j.version>1.7.36</slf4j.version>
<logback.version>1.2.11</logback.version>
<h2.version>2.2.220</h2.version>
<mysql-connector.version>8.0.28</mysql-connector.version>
<commons-io.version>2.11.0</commons-io.version>
<presto-jdbc.version>0.238.1</presto-jdbc.version>
<joda-time.version>2.10.13</joda-time.version>
<spring-cloud-dependencies.version>2021.0.3</spring-cloud-dependencies.version>
<snakeyaml.version>1.33</snakeyaml.version>
</properties>


<dependencyManagement>
<dependencies>
<!-- spring -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>${spring-boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<scope>test</scope>
</dependency>

<!-- mybatis -->
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>mybatis-plus-boot-starter</artifactId>
<version>${mybatis-plus.version}</version>
</dependency>
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>mybatis-plus</artifactId>
<version>${mybatis-plus.version}</version>
</dependency>
<dependency>
<groupId>com.baomidou</groupId>
<artifactId>mybatis-plus-annotation</artifactId>
<version>${mybatis-plus.version}</version>
</dependency>

<!-- jackson -->
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-annotations</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-core</artifactId>
<version>${jackson.version}</version>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.datatype</groupId>
<artifactId>jackson-datatype-jsr310</artifactId>
<version>${jackson.version}</version>
</dependency>

<!-- db connectors -->
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>${mysql-connector.version}</version>
</dependency>

<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
<version>${h2.version}</version>
</dependency>

<!-- sl4j -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
<version>${slf4j.version}</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>${slf4j.version}</version>
</dependency>
<!-- logback -->
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>${logback.version}</version>
</dependency>
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-core</artifactId>
<version>${logback.version}</version>
</dependency>

<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>${guava.version}</version>
</dependency>

<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-collections4</artifactId>
<version>${commons-collections4.version}</version>
</dependency>

<dependency>
<groupId>commons-io</groupId>
<artifactId>commons-io</artifactId>
<version>${commons-io.version}</version>
</dependency>

<dependency>
<groupId>joda-time</groupId>
<artifactId>joda-time</artifactId>
<version>${joda-time.version}</version>
</dependency>

<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-dependencies</artifactId>
<version>${spring-cloud-dependencies.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>

<dependency>
<groupId>org.yaml</groupId>
<artifactId>snakeyaml</artifactId>
<version>${snakeyaml.version}</version>
</dependency>

</dependencies>
</dependencyManagement>



</project>
37 changes: 37 additions & 0 deletions griffin-connectors/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>org.apache.griffin</groupId>
<artifactId>griffin</artifactId>
<version>2.0.0-SNAPSHOT</version>
</parent>


<artifactId>griffin-connectors</artifactId>
<name>${project.artifactId}</name>
<url>https://griffin.apache.org</url>


</project>
37 changes: 37 additions & 0 deletions griffin-dqc/pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>

<parent>
<groupId>org.apache.griffin</groupId>
<artifactId>griffin</artifactId>
<version>2.0.0-SNAPSHOT</version>
</parent>


<artifactId>griffin-dqc</artifactId>
<name>${project.artifactId}</name>
<url>https://griffin.apache.org</url>


</project>
Loading