Skip to content

Commit

Permalink
upgraded dependecies, docker images, added dedot option (#20)
Browse files Browse the repository at this point in the history
* upgraded dependecies, docker images, added dedot option

* fixes git secrets workfow, added auto release workflow
  • Loading branch information
tamir-michaeli authored Jan 22, 2023
1 parent 9446cb9 commit b33e18b
Show file tree
Hide file tree
Showing 13 changed files with 189 additions and 100 deletions.
91 changes: 91 additions & 0 deletions .github/workflows/auto-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
name: Docker Image CI

on:
release:
types: [published]


jobs:
build:
runs-on: ubuntu-latest
steps:
# Get the repository's code
- name: Checkout
uses: actions/checkout@v3
# https://github.com/docker/setup-qemu-action
- name: Set up QEMU
uses: docker/setup-qemu-action@v1
# https://github.com/docker/setup-buildx-action
- name: Set up Docker Buildx
id: buildx
uses: docker/setup-buildx-action@v1
- name: Login to Docker Hub
if: github.event_name != 'pull_request'
uses: docker/login-action@v1
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}
- name: Docker meta
id: dockeraction # you'll use this in the next step
uses: docker/metadata-action@v3
with:
# list of Docker images to use as base name for tags
images: |
logzio/fluent-bit-output
flavor: |
latest=false
# Docker tags based on the following events/attributes

- name: Build and push amd64
uses: docker/build-push-action@v2
with:
context: .
file: ./test/Dockerfile.arm
platforms: linux/amd64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.dockeraction.outputs.tags }}-amd
labels: ${{ steps.dockeraction.outputs.labels }}


- name: Build and push amd64 latest
uses: docker/build-push-action@v2
with:
context: .
file: ./test/Dockerfile.arm
platforms: linux/amd64
push: ${{ github.event_name != 'pull_request' }}
tags: logzio/fluent-bit-output:latest-amd
labels: ${{ steps.dockeraction.outputs.labels }}

- name: Build and push arm64 latest
uses: docker/build-push-action@v2
with:
context: .
file: ./test/Dockerfile.arm
platforms: linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: logzio/fluent-bit-output:latest-arm
labels: ${{ steps.dockeraction.outputs.labels }}

- name: Build and push arm64
uses: docker/build-push-action@v2
with:
context: .
file: ./test/Dockerfile.arm
platforms: linux/arm64
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.dockeraction.outputs.tags }}-arm
labels: ${{ steps.dockeraction.outputs.labels }}

- name: Create manifest version
run: |
docker manifest create ${{ steps.dockeraction.outputs.tags }} --amend ${{ steps.dockeraction.outputs.tags }}-arm --amend ${{ steps.dockeraction.outputs.tags }}-amd
- name: Create manifest latest
run: |
docker manifest create logzio/fluent-bit-output:latest --amend logzio/fluent-bit-output:latest-arm --amend logzio/fluent-bit-output:latest-amd
- name: Push manifest latest
run: |
docker manifest push logzio/fluent-bit-output:latest
- name: Push manifest version
run: |
docker manifest push ${{ steps.dockeraction.outputs.tags }}
17 changes: 8 additions & 9 deletions .github/workflows/git-secrets.yml
Original file line number Diff line number Diff line change
@@ -1,33 +1,32 @@
name: git-secrets

# Controls when the workflow will run
# Triggers the workflow on push or pull request events but only for the main branch
on: [push]

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "main"
git-secrets:
# The type of runner that the job will run on
runs-on: ubuntu-18.04

runs-on: ubuntu-22.04
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Check Out Source Code
uses: actions/checkout@v2

uses: actions/checkout@v3
- name: Set up Python 3.8
uses: actions/setup-python@v2
uses: actions/setup-python@v4
with:
python-version: 3.8
- name: Installing dependencies
run:
sudo apt-get install git less openssh-server
sudo apt-get install less openssh-server
- name: Installing scanning tool
run: |
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
ln -s "$(which echo)" /usr/local/bin/say
brew install git-secrets
git secrets --install
git secrets --register-aws
git secrets --register-aws
- name: Running scanning tool
run:
eval "$(/home/linuxbrew/.linuxbrew/bin/brew shellenv)"
git secrets --scan
32 changes: 18 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -63,12 +63,10 @@ For a list of options, see the configuration parameters below the code block.
| Parameter | Description |
|--------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| logzio_token | **Required**. Replace `<<SHIPPING-TOKEN>>` with the [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship to. |
| logzio_url | **Default**: `https://listener.logz.io:8071` <br> Listener URL and port. <br> Replace `<<LISTENER-HOST>>` with your region's listener host (for example, `listener.logz.io`). For more information on finding your account's region, see [Account region](https://docs.logz.io/user-guide/accounts/account-region.html). |
| logzio_type | **Default**: `logzio-fluent-bit` <br> The [log type](https://docs.logz.io/user-guide/log-shipping/built-in-log-types.html), shipped as `type` field. Used by Logz.io for consistent parsing. Can't contain spaces. |
| logzio_debug | **Default**: `false` <br> Set to `true` to print debug messages to stdout. |
| id | **Default**: `logzio_output_1` <br> Output id. Mandatory when using multiple outputs. |
| workers | Enables dedicated thread(s) for this output. Default value is 1. To support more traffic, adjust this value accordingly. |

| logzio_url | **Default**: `https://listener.logz.io:8071` Listener URL and port. Replace `<<LISTENER-HOST>>` with your region's listener host (for example, `listener.logz.io`). For more information on finding your account's region, see [Account region](https://docs.logz.io/user-guide/accounts/account-region.html). |
| logzio_type | **Default**: `logzio-fluent-bit` The [log type](https://docs.logz.io/user-guide/log-shipping/built-in-log-types.html), shipped as `type` field. Used by Logz.io for consistent parsing. Can't contain spaces. |
| logzio_debug | **Default**: `false` Set to `true` to print debug messages to stdout. |
| id | **Default**: `logzio_output_1` Output id. Mandatory when using multiple outputs. |

#### 3. Run Fluent Bit with the Logz.io plugin

Expand Down Expand Up @@ -120,14 +118,16 @@ For a list of options, see the configuration parameters below the code block.

**Parameters**

| Parameter | Description |
|---|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| logzio_token | **Required**. Replace `<<SHIPPING-TOKEN>>` with the [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship to. |
| logzio_url | **Default**: `https://listener.logz.io:8071` <br> Listener URL and port. <br> Replace `<<LISTENER-HOST>>` with your region's listener host (for example, `listener.logz.io`). For more information on finding your account's region, see [Account region](https://docs.logz.io/user-guide/accounts/account-region.html). |
| logzio_type | **Default**: `logzio-fluent-bit` <br> The [log type](https://docs.logz.io/user-guide/log-shipping/built-in-log-types.html), shipped as `type` field. Used by Logz.io for consistent parsing. Can't contain spaces. |
| logzio_debug | **Default**: `false` <br> Set to `true` to print debug messages to stdout. |
| id | **Default**: `logzio_output_1` <br> Output id. Mandatory when using multiple outputs. |

| Parameter | Description |
|---------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| logzio_token | **Required**. Replace `<<SHIPPING-TOKEN>>` with the [token](https://app.logz.io/#/dashboard/settings/general) of the account you want to ship to. |
| logzio_url | **Default**: `https://listener.logz.io:8071` Listener URL and port. <br> Replace `<<LISTENER-HOST>>` with your region's listener host (for example, `listener.logz.io`). For more information on finding your account's region, see [Account region](https://docs.logz.io/user-guide/accounts/account-region.html). |
| logzio_type | **Default**: `logzio-fluent-bit` The [log type](https://docs.logz.io/user-guide/log-shipping/built-in-log-types.html), shipped as `type` field. Used by Logz.io for consistent parsing. Can't contain spaces. |
| logzio_debug | **Default**: `false` Set to `true` to print debug messages to stdout. |
| id | **Default**: `logzio_output_1` < Output id. Mandatory when using multiple outputs. |
| dedot_enabled | **Default**: `false` Enabled dedot processing. |
| dedot_nested | **Default**: `false` Enables nesting dedot processing. |
| dedot_new_seperator | **Default**: `"_"` Seperator character to use when applying dedot processing. |
#### 2. Run the Docker image

Run the Docker image
Expand Down Expand Up @@ -167,6 +167,10 @@ Always confirm your logs are arriving at your Logz.io account.
## Change log
- **0.3.0**:
- Added an optional dedot processing.
- Upgraded to golang `1.19.1.` in docker image.
- Update to fluent-bit `2.0.8` in docker image.
- **0.2.0**:
- Added `id` parameter to support multiple outputs.
- **0.1.0**:
Expand Down
Binary file modified build/out_logzio-linux-arm64.so
Binary file not shown.
Binary file modified build/out_logzio-linux.so
Binary file not shown.
Binary file modified build/out_logzio-macOS.so
Binary file not shown.
Binary file modified build/out_logzio-windows.so
Binary file not shown.
2 changes: 1 addition & 1 deletion debug/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM golang:1.15.3 as gobuilder
FROM golang:1.18.4 as gobuilder

ENV GOOS=linux\
GOARCH=amd64
Expand Down
9 changes: 8 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
module github.com/logzio/fluent-bit-logzio-output

go 1.15
go 1.18

require (
github.com/fluent/fluent-bit-go v0.0.0-20200729034236-b9c0d6a20853
github.com/stretchr/testify v1.8.0
)

require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/ugorji/go/codec v1.1.7 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
Loading

0 comments on commit b33e18b

Please sign in to comment.