Re-structure CI pipeline

[paullatzelsperger]

WHAT

This proposal is about re-structuring our CI pipelines as shown below to avoid spurious builds. Note that this merely deals with the test workflows (verify.yaml, deployment-test.yaml and business-tests.yaml) as well as the image- and package build (build.yaml).

We should restructure, such that every test stage triggers the subsequent one, and the last test stage triggers the build and publish.

WHY

CI workflows have a natural dependency onto one another, i.e. we should not publish Docker images or Maven packages unless all tests are green.

SOLUTION PROPOSAL

• rename build.yaml -> publish.yaml
• remove the build-extensions step, this is already covered by the unit-test job in verify.yaml
• allow the verify.yaml, deployment-test.yaml and business-tests.yaml flows to run in parallel.
• trigger the publish.yaml flow once all tests are green
• publish.yaml triggers trivy.yaml (same as now, no change)
• prevent publish.yaml from running on PRs and in forks (check via secrets presence)

FURTHER NOTES

• Waiting for multiple workflows is currently not supported, but a workaround could be to introduce an "all-in-one" workflow, that triggers the others within its jobs (which allows waiting).
• It should be noted that serializing/sequentializing workflows will result in overall longer CI runs. While this might be annoying, it is even more annoying to have "broken" Docker images or Maven packages.

[paullatzelsperger]

will be implemented in Issue #229

[stephanbcbauer]

@paullatzelsperger since #229 is closed ... can we also close this discussion? Thank you