[fix] [testing] make TDD more productive #206
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This uses terratest's test-structure tools to make it easier to iterate on this code. The idea is that each test will be broken into stages like 'options', 'apply', 'validate' and 'cleanup' which can be skipped but also persist any state (like randomly generated options). So you can do something like: `make test TEST=./aws-s3-private-bucket SKIP_cleanup=1` Which will 1. run 'options' to generate a set of random variables as input for this test and perist them 2. run 'apply' which will use `terraform apply` to create resources and perist the state file 3. run 'validate' to make assertions about the resources created 4. skip 'cleanup' This will take 1-2min. Then if you run: `make test TEST=./aws-s3-private-bucket SKIP_cleanup=1 SKIP_options=1 SKIP_apply=1` It will only run the validate step, which takes < 10s. You can re run this, iterating on your tests until you are satisfied. At some point you might realize you need to update the terraform code. After you make those edits you can run: `make test TEST=./aws-s3-private-bucket SKIP_cleanup=1 SKIP_options=1` which will 1. skip options, using the existing options 2. run apply, using existing options and state file. This will mean that for some resources you can do an update-in-place, potentially much faster. 3. run validate 4. skip cleanup When all done, you can run cleanup and then re-run with no skips to make sure it works end-to-end.
ryanking
commented
Jun 25, 2020
| @@ -102,7 +102,10 @@ data "aws_iam_policy_document" "bucket_policy" { | |||
| statement { | |||
| sid = "EnforceTLS" | |||
| actions = ["*"] | |||
| resources = ["arn:aws:s3:::${var.bucket_name}/*"] | |||
| resources = [ | |||
There was a problem hiding this comment.
Note that writing these tests led me to realize we had a bug here!
ryanking
commented
Jun 25, 2020
| @@ -138,7 +138,10 @@ func Cleanup(t *testing.T, options *terraform.Options) { | |||
|
|
|||
| func ListEnvVar(name string) []string { | |||
| s := os.Getenv(name) | |||
| return strings.Split(s, ",") | |||
| splitFn := func(c rune) bool { | |||
There was a problem hiding this comment.
without this change an input of "" will give a slice with {""}
ryanking
changed the title
ryanking
changed the title
This was referenced Jul 15, 2020
czimergebot
pushed a commit
that referenced
this pull request
Jul 17, 2020
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This uses a technique inspired by terratest's test-structure tools to make it easier to iterate
on this code.
The idea is that each test will be broken into stages 'options',
'apply', 'validate' and 'cleanup' which can be skipped but also persist
any state (like randomly generated options).
So you can do something like:
make test TEST=./aws-s3-private-bucket SKIP=cleanupWhich will
terraform applyto create resources and persist the state fileThis time will be dominated by how long it takes to apply the changes (seconds to ~1hr).
Then if you run:
make test TEST=./aws-s3-private-bucket SKIP=cleanup,options,applyIt will only run the validate step, which takes < 10s. You can re run
this, iterating on your tests until you are satisfied.
At some point you might realize you need to update the terraform code.
After you make those edits you can run:
make test TEST=./aws-s3-private-bucket SKIP=cleanup,optionswhich will
for some resources you can do an update-in-place, potentially much
faster.
When all done, you can run cleanup and then re-run with no skips to make
sure it works end-to-end.