Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add guide to use Clickhouse Cloud audit logs Splunk module #3073

Merged
merged 5 commits into from
Jan 15, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
96 changes: 96 additions & 0 deletions docs/en/integrations/data-integration/splunk/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
---
sidebar_label: Splunk
slug: /en/integrations/audit-splunk
keywords: [clickhouse, splunk, audit, cloud]
description: Store ClickHouse Cloud audit logs into Splunk.
---
# Storing ClickHouse Cloud Audit logs into Splunk

[Splunk](https://www.splunk.com/) is a data analytics and monitoring platform.

This add-on allows users to store the [ClickHouse Cloud audit logs](https://clickhouse.com/docs/en/cloud/security/audit-logging) into Splunk. It uses [ClickHouse Cloud API](https://clickhouse.com/docs/en/cloud/manage/api/api-overview) to download the audit logs.

This add-on contains only a modular input, no additional UI are provided with this add-on.

# Installation

## For Splunk Enterprise

Download the ClickHouse Cloud Audit Add-on for Splunk from [Splunkbase](https://splunkbase.splunk.com/app/7709).

<img src={require('./images/splunk_001.png').default} className="image" alt="Download from Splunkbase" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

In Splunk Enterprise, navigate to Apps -> Manage. Then click on Install app from file.

<img src={require('./images/splunk_002.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

Select the archived file downloaded from Splunkbase and click on Upload.

<img src={require('./images/splunk_003.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

If everything goes fine, you should now see the ClickHouse Audit logs application installed. If not, consult the splunkd logs for any errors.

# Modular input configuration

To configure the modular input, you'll first need information from your ClickHouse Cloud deployment:

- The organization ID
- An admin [API Key](https://clickhouse.com/docs/en/cloud/manage/openapi)

## Getting information from ClickHouse Cloud

Log in to the [ClickHouse Cloud console](https://console.clickhouse.cloud/).

Navigate to your Organization -> Organization details. There you can copy the Organization ID.

<img src={require('./images/splunk_004.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

Then, navigate to API Keys from the left-end menu.

<img src={require('./images/splunk_005.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

Create an API Key, give a meaningful name and select `Admin` privileges. Click on Generate API Key.

<img src={require('./images/splunk_006.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

Save the API Key and secret in a safe place.

<img src={require('./images/splunk_007.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

## Configure data input in Splunk

Back in Splunk, navigate to Settings -> Data inputs.

<img src={require('./images/splunk_008.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

Select the ClickHouse Cloud Audit Logs data input.

<img src={require('./images/splunk_009.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

Click "New" to configure a new instance of the data input.

<img src={require('./images/splunk_010.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

Once you have entered all the information, click Next.

<img src={require('./images/splunk_011.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>

The input is configured, you can start browsing the audit logs.

# Usage

The modular input stores data in Splunk. To view the data, you can use the general search view in Splunk.

<img src={require('./images/splunk_012.png').default} className="image" alt="Manage apps" style={{width: '75%', 'backgroundColor': 'transparent'}}/>
<br/>
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 2 additions & 1 deletion docs/en/integrations/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -218,6 +218,7 @@ We are actively compiling this list of ClickHouse integrations below, so it's no
|SigNoz|<img src={require('./images/logos/signoz-logo.png').default} className="image" alt="SigNoz logo" style={{width: '3rem', 'backgroundColor': 'transparent'}}/>|Data visualization|Open Source Observability Platform|[Documentation](https://www.signoz.io/docs/architecture/)|
|Snappy Flow|<img src={require('./images/logos/snappy_flow_logo.png').default} className="image" alt="Snappy Flow logo" style={{width: '3rem', 'backgroundColor': 'transparent'}}/>|Data management|Collects ClickHouse database metrics via plugin.|[Documentation](https://docs.snappyflow.io/docs/Integrations/clickhouse/instance)|
|Soda|<img src={require('./images/logos/soda_logo.png').default} className="image" alt="Soda logo" style={{width: '3rem', 'backgroundColor':'transparent'}}/>|Data quality|Soda integration makes it easy for organziations to detect, resolve, and prevent data quality issues by running data quality checks on data before it is loaded into the database.|[Website](https://www.soda.io/integrations/clickhouse)|
|Splunk|<img src={require('./images/logos/splunk_logo.png').default} className="image" alt="Splunk logo" style={{width: '3rem', 'backgroundColor':'transparent'}}/>|Data integration|Splunk modular input to import to Splunk the ClickHouse Cloud Audit logs.|[Website](https://splunkbase.splunk.com/app/7709),<br/>[Documentation](/docs/en/integrations/data-integration/splunk/index.md)|
|StreamingFast|<img src={require('./images/logos/streamingfast.png').default} className="image" alt="StreamingFast logo" style={{width: '3rem', 'backgroundColor': 'transparent'}}/>|Data ingestion| Blockchain-agnostic, parallelized and streaming-first data engine. |[Website](https://www.streamingfast.io/)|
|Streamkap|<img src={require('./images/logos/streamkap-logo.png').default} className="image" alt="Streamkap logo" style={{width: '3rem', 'backgroundColor': 'transparent'}}/>|Data ingestion|Setup real-time CDC (Change Data Capture) streaming to ClickHouse with high throughput in minutes.|[Documentation](https://docs.streamkap.com/docs/clickhouse)|
|Supabase|<img src={require('./images/logos/logo_supabase.png').default} className="image" alt="Supabase logo" style={{width: '3rem', 'backgroundColor': 'transparent'}}/>|Data ingestion|Open source Firebase alternative|[GitHub](https://github.com/supabase/wrappers/tree/main/wrappers/src/fdw/clickhouse_fdw),[Blog](https://clickhouse.com/blog/migrating-data-between-clickhouse-postgres)|
Expand Down Expand Up @@ -258,7 +259,7 @@ We are actively compiling this list of ClickHouse integrations below, so it's no
|Datahub|<img src={require('./images/logos/logo_datahub.png').default} className="image" alt="Datahub logo" style={{width: '3rem', 'backgroundColor': 'transparent'}}/>|Data management|Open Source Data Catalog that enables data discovery, data observability and federated governance|[Documentation](https://datahubproject.io/docs/generated/ingestion/sources/clickhouse/)|
|Dbmate| |Data management|Database migration tool that will keep your database schema in sync across multiple developers and servers|[GitHub](https://github.com/amacneil/dbmate#clickhouse)|
|DeepFlow|<img src={require('./images/logos/logo_deepflow.png').default} className="image" alt="Deepflow logo" style={{width: '3rem', 'backgroundColor': 'transparent'}}/>|Data ingestion|Application Observability using eBPF|[Website](https://deepflow.io)|
|Easypanel|<img src={require('./images/logos/logo-easypanel.png').default} className="image" alt="Easypanel logo" style={{width: '3rem', 'backgroundColor': 'transparent'}} />|Deployment method|It's a modern server control panel. You can use it to deploy ClickHouse on your own server.|[Website](https://easypanel.io), <br/>[Documentation](docs/en/integrations/deployment/easypanel/index.md)|
|Easypanel|<img src={require('./images/logos/logo-easypanel.png').default} className="image" alt="Easypanel logo" style={{width: '3rem', 'backgroundColor': 'transparent'}} />|Deployment method|It's a modern server control panel. You can use it to deploy ClickHouse on your own server.|[Website](https://easypanel.io), <br/>[Documentation](docs/en/integrations/data-integration/easypanel/index.md)|
|Explo|<ExploSVG style={{width: '3rem'}} />|Data visualization|Explo helps companies build real-time analytics dashboard by providing flexible components.|[Website](https://www.explo.co/integrations/clickhouse)|
|Flink|<img src={require('./images/logos/logo_flink.png').default} className="image" alt="Flink logo" style={{width: '3rem', 'backgroundColor': 'transparent'}}/>|Data ingestion|Flink sink for ClickHouse database, powered by Async Http Client|[GitHub](https://github.com/itinycheng/flink-connector-clickhouse)|
|Goose|<img src={require('./images/logos/goose_logo.png').default} className="image" alt="Goose logo" style={{width: '3rem', 'backgroundColor': 'transparent'}}/>|Data migration|A database migration tool that supports SQL migrations and Go functions.|[GitHub](https://github.com/pressly/goose),<br/>[Documentation](https://pressly.github.io/goose/)|
Expand Down
8 changes: 0 additions & 8 deletions docs/en/integrations/no-code/_category_.yml

This file was deleted.

5 changes: 3 additions & 2 deletions sidebars.js
Original file line number Diff line number Diff line change
Expand Up @@ -907,8 +907,9 @@ const sidebars = {
"en/integrations/data-visualization/tableau/tableau-and-clickhouse",
"en/integrations/data-visualization/tableau/tableau-online-and-clickhouse",
"en/integrations/data-visualization/zingdata-and-clickhouse",
"en/integrations/no-code/retool",
"en/integrations/deployment/easypanel/index"
"en/integrations/data-integration/retool/index",
"en/integrations/data-integration/easypanel/index",
"en/integrations/data-integration/splunk/index"
],
},{
type: "category",
Expand Down
Loading