Merge pull request #52 from JFolberth/feature/terraform_aws

"Add Terraform build and apply stages for AWS environment"

README.md

@@ -39,6 +39,8 @@ I will try to keep this up to date; however, as software development goes someti
 | Bicep DotNet Build Stage | Run a `dotnet build` and a `dotnet publish` command against a project. Ability to include tests as an option. Will also include publishing a bicep artifact which will include running a `what-if` against all desired Azure environments. An example is contained in the repository [ToDo_AzureDeploymentEnvironment](https://github.com/JFolberth/ToDo_AzureDeploymentEnvironment/blob/main/YAML/demo_environment_template.yml)| [bicep_dotnet_build_stage.yml](stages/bicep_dotnet_build_stage.yml)
 | DotNet SQL Build Stage | Run a `dotnet build` against a SQL Project. Will also include a list of folders to produce as artifacts for future stage use An example is contained in the repository [cicd-adventureWorks](https://github.com/JFolberth/cicd-adventureWorks/blob/main/YAML/azure-pipeline.yml)| [dotnet_sql_build_stage.yml](stages/dotnet_sql_build_stage.yml)
 | Dacpac Deploy Stage | Leverages the [SqlAzureDacpacDeployment@1 task](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/sql-azure-dacpac-deployment-v1?view=azure-pipelines) to take a dacpac file and deploy it against an existing SQL Server and Database. An example is contained in the repository [cicd-adventureWorks](https://github.com/JFolberth/cicd-adventureWorks/blob/main/YAML/azure-pipeline.yml)| [dacpac_deploy_stage.yml](stages/dacpac_deploy_stage.yml)
+| Terraform Build AWS| Will install Terraform, init, plan in parallel  across environment and regions. Does require [Azure Pipelines Terraform Task](https://marketplace.visualstudio.com/items?itemName=JasonBJohnson.azure-pipelines-tasks-terraform&ssr=false#overview)  and the remote backend as configured in the [aws_terraform_dev2_variables.yml](variables/aws_terraform_dev2_variables.yml) By default Terraform files are expected to be in a folder called 'infrastructure' and leverage a variable file like dev.eus.variables.tfvars. This is easily changed be changed as needed for an organization by updating the 'TerraformWorkingDirectory' in the variables file and/or the 'additionalParameters' in the stage  | [terraform_build_aws_stage.yml](stages/terraform_build_aws_stage.yml)
+| Terraform Apply | Will install Terraform, init, plan, apply across environment and regions with a stage for each region/environment combination Does require [Azure Pipelines Terraform Task](https://marketplace.visualstudio.com/items?itemName=JasonBJohnson.azure-pipelines-tasks-terraform&ssr=false#overview) and the remote backend as configured in the [aws_terraform_dev2_variables.yml](variables/aws_terraform_dev_variables.yml). By default Terraform files are expected to be in a folder called 'infrastructure' and leverage a variable file like dev.eus.variables.tfvars. This is easily changed be changed as needed for an organization by updating the 'TerraformWorkingDirectory' in the variables file and/or the 'additionalParameters' in the stage. | [terraform_apply_aws_stage.yml](stages/terraform_apply_aws_stage.yml)
 
 # Additional Resources
 - [Series of Post on Microsoft's Health and Life Sciences Blogs covering YAML Pipelines](https://techcommunity.microsoft.com/t5/healthcare-and-life-sciences/bg-p/HealthcareAndLifeSciencesBlog/label-name/YAML%20Pipeline%20Series)

jobs/terraform_apply_aws_env_job.yml

@@ -0,0 +1,55 @@
+parameters:
+- name: environmentName
+  type: string
+  default: 'dev'
+- name: serviceName
+  type: string
+- name: terraformVersion
+  type: string
+  default: 'latest'
+- name: additionalParameters
+  type: object
+  default: []
+- name: templateDirectory
+  type: string 
+  default: 'infrastructure'
+
+jobs:
+- deployment: terraformApply${{ parameters.environmentName }}
+  displayName: Terraform Apply ${{ parameters.environmentName }}
+  variables: 
+  - template: ../variables/aws_terraform_${{ parameters.environmentName }}_variables.yml
+  - name: commandOptions
+    value: ''
+  environment: ${{ parameters.environmentName }}
+  strategy:
+    runOnce:
+      deploy:
+        steps:
+        - download: current
+          artifact: ${{ parameters.templateDirectory }}
+        - template: ../tasks/terraform_install_task.yml
+          parameters:
+            terraformVersion: ${{ parameters.terraformVersion }}
+
+        - template: ../tasks/terraform_init_aws_task.yml
+          parameters:
+           serviceName: ${{ parameters.serviceName }}
+           TerraformDirectory: $(Pipeline.Workspace)/${{ variables.workingDirectory  }}
+           backendServiceAws: ${{ variables.backendServiceAws }}
+           backendAwsRegion: ${{ variables.backendAwsRegion }}
+           backendAwsBucket: ${{ variables.backendAwsBucket }}
+
+        - template: ../tasks/terraform_plan_aws_task.yml
+          parameters:
+            TerraformDirectory: $(Pipeline.Workspace)/${{ variables.workingDirectory   }}
+            backendServiceAws: ${{ variables.backendServiceAws }}
+            providerAwsRegion: ${{ variables.backendAwsRegion }}
+            additionalParameters: ${{ parameters.additionalParameters }}
+            publishPlanResults: 'CI - ${{ parameters.environmentName }} - ${{ parameters.serviceName }}'
+
+        - template: ../tasks/terraform_apply_aws_task.yml
+          parameters:
+            TerraformDirectory: $(Pipeline.Workspace)/${{ variables.workingDirectory   }}
+            backendServiceAws: ${{ variables.backendServiceAws }}
+            providerAwsRegion: ${{ variables.backendAwsRegion }}

jobs/terraform_build_aws_env_job.yml

@@ -0,0 +1,41 @@
+parameters:
+- name: environmentName
+  type: string
+  default: 'dev'
+- name: serviceName
+  type: string
+- name: terraformVersion
+  type: string
+  default: 'latest'
+- name: additionalParameters
+  type: object
+  default: []
+- name: dependsOn
+  type: object
+  default: []
+jobs:
+- job: 'buildTerraform_${{ parameters.environmentName }}'
+  dependsOn: ${{parameters.dependsOn}}
+  displayName: Build Terraform ${{ parameters.environmentName }}
+  variables:
+  - template: ../variables/aws_terraform_${{ parameters.environmentName }}_variables.yml
+  steps:
+  - template: ../tasks/terraform_install_task.yml
+    parameters:
+      terraformVersion: ${{ parameters.terraformVersion }}
+
+  - template: ../tasks/terraform_init_aws_task.yml
+    parameters:
+     serviceName: ${{ parameters.serviceName }}
+     TerraformDirectory: ${{ variables.workingDirectory  }}
+     backendServiceAws: ${{ variables.backendServiceAws }}
+     backendAwsRegion: ${{ variables.backendAwsRegion }}
+     backendAwsBucket: ${{ variables.backendAwsBucket }}
+
+  - template: ../tasks/terraform_plan_aws_task.yml
+    parameters:
+      TerraformDirectory: ${{ variables.workingDirectory   }}
+      backendServiceAws: ${{ variables.backendServiceAws }}
+      providerAwsRegion: ${{ variables.backendAwsRegion }}
+      additionalParameters: ${{ parameters.additionalParameters }}
+      publishPlanResults: 'CI - ${{ parameters.environmentName }} - ${{ parameters.serviceName }}'

stages/terraform_apply_aws_stage.yml

@@ -0,0 +1,28 @@
+parameters:
+- name: environmentObjects
+  type: object
+  default:
+    - environmentName: 'dev'
+      regionAbrvs: ['cus']
+- name: templateFileName
+  type: string
+  default: 'main'
+- name: templateDirectory
+  type: string 
+  default: 'infrastructure'
+- name: serviceName
+  type: string
+  default: 'SampleApp'
+
+stages:
+  - ${{ each environmentObject in parameters.environmentObjects }} :
+    - ${{ each regionAbrv in environmentObject.regionAbrvs }} :
+        - stage: '${{ parameters.serviceName }}_${{ environmentObject.environmentName}}_${{regionAbrv}}_tf_apply'
+          jobs:
+          - template: ../jobs/terraform_apply_aws_env_job.yml
+            parameters:
+                environmentName: ${{ environmentObject.environmentName }}
+                serviceName: ${{ parameters.serviceName }}
+                additionalParameters: ['-var-file="variables/${{ environmentObject.environmentName }}.${{regionAbrv}}.variables.tfvars"']
+                templateDirectory: ${{ parameters.templateDirectory }}
+

stages/terraform_build_aws_stage.yml

@@ -0,0 +1,32 @@
+parameters:
+- name: environmentObjects
+  type: object
+  default:
+    - environmentName: 'dev'
+      regionAbrvs: ['cus']
+- name: templateFileName
+  type: string
+  default: 'main'
+- name: templateDirectory
+  type: string 
+  default: 'infrastructure'
+- name: serviceName
+  type: string
+  default: 'SampleApp'
+
+stages:
+- stage: '${{ parameters.serviceName }}_tf_build'
+  jobs:
+  - template: ../jobs/ado_publish_job.yml
+    parameters:
+      targetPath: ${{ parameters.templateDirectory }}
+      artifactname: ${{ parameters.templateDirectory }}
+  - ${{ each environmentObject in parameters.environmentObjects }} :
+    - ${{ each regionAbrv in environmentObject.regionAbrvs }} :
+      - template: ../jobs/terraform_build_aws_env_job.yml
+        parameters:
+          environmentName: ${{ environmentObject.environmentName }}
+          serviceName: ${{ parameters.serviceName }}
+          additionalParameters: ['-var-file="variables/${{ environmentObject.environmentName }}.${{regionAbrv}}.variables.tfvars"']
+
+

tasks/terraform_apply_aws_task.yml

@@ -0,0 +1,21 @@
+parameters:
+- name: TerraformDirectory
+  type: string
+- name: backendServiceAws
+  type: string
+- name: providerAwsRegion
+  type: string
+- name: additionalParameters
+  type: object
+  default: []
+
+steps:
+- task: TerraformCLI@1
+  displayName: 'Terraform : apply'
+  condition: and(succeeded(), eq(variables['TERRAFORM_PLAN_HAS_CHANGES'],'true'))
+  inputs:
+    command: apply
+    workingDirectory: ${{ parameters.TerraformDirectory }}
+    commandOptions: '$(System.DefaultWorkingDirectory)/terraform.tfplan'
+    providerServiceAws: ${{ parameters.backendServiceAws }}
+    providerAwsRegion: ${{ parameters.providerAwsRegion }}

tasks/terraform_apply_task.yml

@@ -1,7 +1,7 @@
 parameters:
 - name: TerraformDirectory
   type: string
-- name: AzureSubscriptionServiceConnectionName
+- name: backendServiceAws
   type: string
 - name: additionalParameters
   type: object

tasks/terraform_init_aws_task.yml

@@ -0,0 +1,24 @@
+parameters:
+- name: serviceName
+  type: string
+- name: TerraformDirectory
+  type: string
+- name: backendServiceAws
+  type: string
+- name: backendAwsRegion
+  type: string
+- name: backendAwsBucket
+  type: string
+
+
+steps:
+  - task: TerraformCLI@1
+    displayName: 'Terraform : init'
+    inputs:
+      command: init
+      backendType: aws
+      workingDirectory: ${{ parameters.TerraformDirectory }}
+      backendServiceAws: ${{ parameters.backendServiceAws }}
+      backendAwsRegion: ${{ parameters.backendAwsRegion }}
+      backendAwsBucket: ${{ parameters.backendAwsBucket }}
+      backendAwsKey: ${{ parameters.serviceName }}.tfstate

tasks/terraform_plan_aws_task.yml

@@ -0,0 +1,24 @@
+parameters:
+- name: TerraformDirectory
+  type: string
+- name: backendServiceAws 
+  type: string
+- name: outputFile
+  default: 'terraform.tfplan'
+- name: additionalParameters
+  type: object
+  default: []
+- name: providerAwsRegion
+  type: string
+- name: publishPlanResults
+  default: ''
+steps:
+    - task: TerraformCLI@1
+      displayName: 'Terraform : plan'
+      inputs:
+        command: plan
+        workingDirectory: ${{ parameters.TerraformDirectory }}
+        publishPlanResults: ${{ parameters.publishPlanResults }}
+        providerServiceAws: ${{ parameters.backendServiceAws  }}
+        providerAwsRegion: ${{ parameters.providerAwsRegion }}
+        commandOptions: "-lock=false ${{ join(' ', parameters.additionalParameters) }} -out=$(System.DefaultWorkingDirectory)/${{ parameters.outputFile }} -detailed-exitcode"

variables/aws_terraform_dev2_variables.yml

@@ -0,0 +1,5 @@
+variables:
+    backendServiceAws: awsdev2
+    workingDirectory: infrastructure
+    backendAwsBucket: jfmsterraformstate 
+    backendAwsRegion: us-east-1