Introduce and use Pipeline resources

[calebhailey]

Suggested Improvement

Sensu Go version 6.5 introduced a new Pipeline resource as a first-class compilation tool for configuring pipelines. Pipelines make it much easier to reason about event processing in Sensu. By decoupling handler functions (e.g. "send an alert via Mattermost") from pipeline configuration (i.e. "only process events matching this filter, then enrich the event using this mutator, then process the data using this handler"), Handler resources become reusable building blocks that can be composed in a number of different workflows.

Example:

---
type: Pipeline
api_version: core/v2
metadata:
  name: alerts
spec:
  workflows:
  - name: mattermost
    filters:
    - api_version: core/v2
      type: EventFilter
      name: is_incident
    - api_version: core/v2
      type: EventFilter
      name: not_silenced
    handler:
      api_version: core/v2
      type: Handler
      name: mattermost
---
type: Handler
api_version: core/v2
metadata:
  name: mattermost
spec:
  type: pipe
  command: >-
    sensu-slack-handler
    --channel "#alerts"
    --username SensuGo
    --description-template "{{ .Check.Output }}\n\n[namespace:{{.Entity.Namespace}}]"
    --webhook-url ${MATTERMOST_WEBHOOK_URL}
  runtime_assets:
  - sensu/sensu-slack-handler:1.4.0
  timeout: 10
  secrets:
  - name: MATTERMOST_WEBHOOK_URL
    secret: mattermost_webhook_url
---
type: Secret
api_version: secrets/v1
metadata:
  name: mattermost_webhook_url
spec:
  provider: env
  id: MATTERMOST_WEBHOOK_URL

Checks can now reference pipelines:[] instead of handlers:[]:

---
type: CheckConfig
api_version: core/v2
metadata:
  name: disk-usage
spec:
  command: >-
    check-disk-usage
    --metrics
    --warning {{ .annotations.disk_usage_warning_threshold | default "80.0" }}
    --critical {{ .annotations.disk_usage_critical_threshold | default "90.0" }}
  runtime_assets:
  - sensu/check-disk-usage:0.6.0
  publish: true
  interval: 30
  subscriptions:
  - linux
  - windows
  - darwin
  timeout: 10
  pipelines:
  - api_version: pipelines/v1
    type: Pipeline
    name: mattermost
  - api_version: pipelines/v1
    type: Pipeline
    name: sumologic
  output_metric_format: prometheus_text
  output_metric_tags:
  - name: entity
    value: "{{ .name }}"
  - name: namespace
    value: "{{ .namespace }}"

Applicable Lessons

Introducing Pipelines may require the insertion of an additional lesson, likely before introducing Handlers.

• Lesson 01
• Lesson 02
• Lesson 03
• Lesson 04
• Lesson 05
• Lesson 06
• Lesson 07
• Lesson 08
• Lesson 09
• Lesson 10
• Lesson 11
• Lesson 12
• Lesson 13
• Lesson 14
• Lesson 15
• Lesson 16