Skip to content

Commit

Permalink
Update CLI Usage
Browse files Browse the repository at this point in the history
  • Loading branch information
jlandowner committed Dec 20, 2023
1 parent 828a427 commit 77f2ccc
Show file tree
Hide file tree
Showing 2 changed files with 45 additions and 45 deletions.
46 changes: 23 additions & 23 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# helm-chartsnap 📸 
# helm-chartsnap 📸
[![Go Reference](https://pkg.go.dev/badge/github.com/jlandowner/helm-chartsnap.svg)](https://pkg.go.dev/github.com/jlandowner/helm-chartsnap)
[![Go Report Card](https://goreportcard.com/badge/github.com/jlandowner/helm-chartsnap)](https://goreportcard.com/report/github.com/jlandowner/helm-chartsnap)
[![Workflow Status](https://github.com/jlandowner/helm-chartsnap/actions/workflows/release.yaml/badge.svg)](https://github.com/jlandowner/helm-chartsnap/actions/workflows/release.yaml)
Expand Down Expand Up @@ -29,44 +29,44 @@ Bring powerful UI testing capabilities for Helm chart with minimal configuration

* **Optimized for Helm chart**

- **Handling dynamic output generated by helm functions**: You can specify additional properties within the values.yaml file for each test case to providing flexibility in adjusting test behavior. For example, you can control dynamically generated values generated helm functions like [randAlphaNum](https://helm.sh/docs/chart_template_guide/function_list/).
- **Handling dynamic output generated by helm functions**: You can specify additional properties within the values.yaml file for each test case to provide flexibility in adjusting test behavior. For example, you can control the output dynamically generated by helm functions like [randAlphaNum](https://helm.sh/docs/chart_template_guide/function_list/).

* **Continuous Snapshot testing**

- **[GitHub Action](https://github.com/jlandowner/helm-chartsnap-action) is ready**: GitHub action which enables to do continuous snapshot testing for local or remote Helm charts in your CI is ready 🚀 -> [`helm-chart-action`](https://github.com/jlandowner/helm-chartsnap-action)

## Why Snapshot Testing? 📸

I believe that snapshot testing, popular testing approach in Web or UI testing with tools like Jest, is the best way to test Helm charts. Snapshot testing provides several benefits:
I believe that snapshot testing, a popular testing approach in Web or UI testing with tools like Jest, is the best way to test Helm charts. Snapshot testing provides several benefits:

- **Visual comparisons:** Snapshot tests create a baseline output and then compare it to subsequent test results. This allows for easy visual inspection of changes and regressions in Helm chart rendering.

- **Enhanced maintainability:** Snapshot testing simplifies the test setup process, making it more accessible and maintainable, especially when dealing with complex Helm charts. The input for the Helm chart is a combination of values. We need only maintain the assumed input combinations(=values.yaml), and their snapshots.

- **Continuous testing:** By continuously taking snapshots of the Helm chart in your CI process, you can quickly notice the changes in your whole chart and avoid regressions.

This advantages are not limited to the Helm chart you created, but also the third-party charts you have installed on your Kubernetes if you are a Kubernetes administrator. When upgrading cluster, you can notice any changes of the output manifests produced by your *values*, no matter how minor.
These advantages are not limited to the Helm chart you created, but also the third-party charts you have installed on your Kubernetes if you are a Kubernetes administrator. When upgrading a cluster, you can notice any changes in the output manifests produced by your *values*, no matter how minor.


### Motivation

There are existing Helm testing tools which can snapshot test.
- [helm-unittest](https://github.com/helm-unittest/helm-unittest), a popular unittest tool, it has option to do [snapshot testing](https://github.com/helm-unittest/helm-unittest#snapshot-testing).
There are existing Helm testing tools that support snapshot tests.
- [helm-unittest](https://github.com/helm-unittest/helm-unittest), a popular unit test tool, it has an option to do [snapshot testing](https://github.com/helm-unittest/helm-unittest#snapshot-testing).
- [terratest](https://terratest.gruntwork.io/), Automated tests for your infrastructure code. It can write tests as Go codes and it has the feature to test the output of Helm. [blog post](https://blog.gruntwork.io/automated-testing-for-kubernetes-and-helm-charts-using-terratest-a4ddc4e67344)

> Helm offitial test tool [`ct`](https://github.com/helm/chart-testing) is not for unit test or snapshot test but for lint and integration test
> Helm offitial test tool [`ct`](https://github.com/helm/chart-testing) is not for unit tests or snapshot tests but for lint and integration test
While they are valuable tools for testing Helm charts, it has certain limitations for snapshot testing.

- You need to learn and prepare the Test Suite files of the tools even if only for snapshot testing.

- Not snapshot whole chart by default, but designed to test on a per-file basis within the Helm chart.
- Not snapshot the whole chart by default, but designed to test on a per-file basis within the Helm chart.

- Helm function's dynamic outputs are not considered.

helm-unittest is designed that it conducts tests on a per-file basis within the Helm chart. This means that it evaluates each individual file independently, and when Helm chart values or conditions lead to an entire file being conditionally included or excluded, it can result in tests being unable to cover important scenarios.
helm-unittest is designed that conducts tests on a per-file basis within the Helm chart. This means that it evaluates each file independently, and when Helm chart values or conditions lead to an entire file being conditionally included or excluded, it can result in tests being unable to cover important scenarios.

Here is an example of test spec file for snapshot testing in helm-unittest.
Here is an example of a test spec file for snapshot testing in helm-unittest.

> https://github.com/helm-unittest/helm-unittest#snapshot-testing
>
Expand All @@ -88,7 +88,7 @@ For instance, consider a scenario where the Helm chart may create a `Certificate
In such cases, helm-unittest may not be able to adequately test the entire chart's behavior, as the output of these conditional files may be empty, leading to incomplete tests.
However all you have to do with `helm-chartsnap` is just to prepare some `values.yaml` files of the patterns you expect 😊
However, all you have to do with `helm-chartsnap` is just to prepare some `values.yaml` files of the patterns you expect 😊
## Installation 🚀
Expand All @@ -110,25 +110,25 @@ Other options:
## Usage
Just pass your chart for quickstart.
Just pass your chart for a quick start.
```
helm chartsnap -c YOUR_CHART
```
`__snapshot__` directory is created in the chart directory and `default.snap` is created.
Also you can create test cases as a variation of Values files of your chart.
Also, you can create test cases as a variation of the Values files of your chart.
Here is full usage of the command.
Here is the full usage of the command.
```sh
Usage:
chartsnap -c CHART [flags]
Examples:
# Snapshot with defualt values:
# Snapshot with default values:
chartsnap -c YOUR_CHART
# Update snapshot files:
Expand All @@ -140,13 +140,13 @@ Examples:
# Snapshot all test cases:
chartsnap -c YOUR_CHART -f YOUR_TEST_VALUES_FILES_DIRECTOY
# Set addtional args or flags for 'helm template' command:
# Set additional args or flags for the 'helm template' command:
chartsnap -c YOUR_CHART -f YOUR_TEST_VALUES_FILE -- --skip-tests
# Snapshot remote chart in Helm repository:
chartsnap -c CHART_NAME -f YOUR_VALUES_FILE -- --repo HELM_REPO_URL
# Snapshot [`ingress-nginx`](https://kubernetes.github.io/ingress-nginx/) helm chart for specific version with your own value file:
# Snapshot [`ingress-nginx`](https://kubernetes.github.io/ingress-nginx/) helm chart for a specific version with your value file:
chartsnap -c ingress-nginx -f YOUR_VALUES_FILE -- --repo https://kubernetes.github.io/ingress-nginx --namespace kube-system --version 4.8.3
# Snapshot [`cilium`](https://cilium.io) helm chart with default value and set flags:
Expand All @@ -170,7 +170,7 @@ Flags:
--parallelism int test concurrency if taking multiple snapshots for a test value file directory. default is unlimited (default -1)
--release-name string release name. this flag is passed to 'helm template RELEASE_NAME CHART --values VALUES' as 'RELEASE_NAME' (default "chartsnap")
-u, --update-snapshot update snapshot mode
-f, --values string path to a test values file or directory. if directroy is set, all test files are tested. if empty, default values are used. this flag is passed to 'helm template RELEASE_NAME CHART --values VALUES' as 'VALUES'
-f, --values string path to a test values file or directory. if the directory is set, all test files are tested. if empty, default values are used. this flag is passed to 'helm template RELEASE_NAME CHART --values VALUES' as 'VALUES'
-v, --version version for chartsnap
```
Expand Down Expand Up @@ -198,15 +198,15 @@ dynamicFields:

If you have a test values file directory, place it as a `.chartsnap.yaml` file within that directory. It is a common behavior in all snapshot tests in the directory.

For more example, see [example/app1](example/app1).
For more examples, see [example/app1](example/app1).

Alternatively, you can directly add it as a property in the test values file directly.
In this case you don't need to prepare `.chartsnap.yaml`. It is for taking snapshot of remote or thirdparty charts with your single values file.
In this case, you don't need to prepare `.chartsnap.yaml`. It is for taking snapshots of remote or third-party charts with your single values file.

Your values file can be like this:

```yaml
# Add this property on your values.
# Add this property to your values.
testSpec:
dynamicFields:
- apiVersion: v1
Expand All @@ -230,11 +230,11 @@ testSpec:
- /data/tls.crt
- /data/tls.key

# Others can be any your chart value.
# Others can be any of your chart values.
# ...
```

For more example, see [example/remote](example/remote).
For more examples, see [example/remote](example/remote).

## Showcase & Users ✨

Expand Down
44 changes: 22 additions & 22 deletions main.go
Original file line number Diff line number Diff line change
Expand Up @@ -82,34 +82,34 @@ Snapshot testing tool like Jest for Helm charts.
You can create test cases as a variation of Values files of your chart.
` + "`" + `__snapshot__` + "`" + ` directory is created in the same directory as test Values files.
In addition, Values files can have a ` + "`" + `testSpec` + "`" + ` property that can detail or control the test case.
In addition, chartsnap support preventing mismatched snapshots by Helm functions.
You can specify the paths of dynamic values in the generated YAML using [JSONPath](https://datatracker.ietf.org/doc/html/rfc6901).
` + "```" + `yaml
testSpec:
# dynamicFields defines values that are dynamically generated by Helm function like 'randAlphaNum'
# https://helm.sh/docs/chart_template_guide/function_list/#randalphanum-randalpha-randnumeric-and-randascii
# Replace outputs with fixed values to prevent unmatched outputs at each snapshot.
dynamicFields:
- apiVersion: v1
kind: Secret
name: cosmo-auth-env
jsonPath:
- /data/COOKIE_HASHKEY
- /data/COOKIE_BLOCKKEY
- /data/COOKIE_HASHKEY
- /data/COOKIE_SESSION_NAME
# Others can be any your chart value.
# ...
# dynamicFields defines values that are dynamically generated by Helm function like 'randAlphaNum'
# https://helm.sh/docs/chart_template_guide/function_list/#randalphanum-randalpha-randnumeric-and-randascii
# Replace outputs with fixed values to prevent unmatched outputs at each snapshot.
dynamicFields:
- apiVersion: v1
kind: Secret
name: cosmo-auth-env
jsonPath:
- /data/COOKIE_HASHKEY
- /data/COOKIE_BLOCKKEY
- /data/COOKIE_HASHKEY
- /data/COOKIE_SESSION_NAME
` + "```" + `
See the repository for full documentation.
Place it as a '.chartsnap.yaml' file within your test values file directory.
See the repository for the full documentation.
https://github.com/jlandowner/helm-chartsnap.git
MIT 2023 jlandowner/helm-chartsnap
`,
Example: `
# Snapshot with defualt values:
# Snapshot with default values:
chartsnap -c YOUR_CHART
# Update snapshot files:
Expand All @@ -121,13 +121,13 @@ MIT 2023 jlandowner/helm-chartsnap
# Snapshot all test cases:
chartsnap -c YOUR_CHART -f YOUR_TEST_VALUES_FILES_DIRECTOY
# Set addtional args or flags for 'helm template' command:
# Set additional args or flags for the 'helm template' command:
chartsnap -c YOUR_CHART -f YOUR_TEST_VALUES_FILE -- --skip-tests
# Snapshot remote chart in Helm repository:
chartsnap -c CHART_NAME -f YOUR_VALUES_FILE -- --repo HELM_REPO_URL
# Snapshot ingress-nginx (https://kubernetes.github.io/ingress-nginx/) helm chart for specific version with your own value file:
# Snapshot ingress-nginx (https://kubernetes.github.io/ingress-nginx/) helm chart for a specific version with your value file:
chartsnap -c ingress-nginx -f YOUR_VALUES_FILE -- --repo https://kubernetes.github.io/ingress-nginx --namespace kube-system --version 4.8.3
# Snapshot cilium (https://cilium.io) helm chart with default value and set flags:
Expand Down Expand Up @@ -155,7 +155,7 @@ MIT 2023 jlandowner/helm-chartsnap
}
rootCmd.PersistentFlags().StringVar(&o.ReleaseName, "release-name", "chartsnap", "release name. this flag is passed to 'helm template RELEASE_NAME CHART --values VALUES' as 'RELEASE_NAME'")
rootCmd.PersistentFlags().StringVarP(&o.NamespaceFlag, "namespace", "n", "default", "namespace. this flag is passed to 'helm template RELEASE_NAME CHART --values VALUES --namespace NAMESPACE' as 'NAMESPACE'")
rootCmd.PersistentFlags().StringVarP(&o.ValuesFile, "values", "f", "", "path to a test values file or directory. if directroy is set, all test files are tested. if empty, default values are used. this flag is passed to 'helm template RELEASE_NAME CHART --values VALUES' as 'VALUES'")
rootCmd.PersistentFlags().StringVarP(&o.ValuesFile, "values", "f", "", "path to a test values file or directory. if the directory is set, all test files are tested. if empty, default values are used. this flag is passed to 'helm template RELEASE_NAME CHART --values VALUES' as 'VALUES'")
rootCmd.PersistentFlags().StringVarP(&o.OutputDir, "output-dir", "o", "", "directory which is __snapshot__ directory is created. (default: values file directory if --values is set; chart directory if chart is local; else current directory)")
if err := rootCmd.MarkPersistentFlagDirname("output-dir"); err != nil {
panic(err)
Expand Down

0 comments on commit 77f2ccc

Please sign in to comment.