diff --git a/README.md b/README.md index 9ef17ea..7132c9e 100644 --- a/README.md +++ b/README.md @@ -39,6 +39,8 @@ I will try to keep this up to date; however, as software development goes someti | Bicep DotNet Build Stage | Run a `dotnet build` and a `dotnet publish` command against a project. Ability to include tests as an option. Will also include publishing a bicep artifact which will include running a `what-if` against all desired Azure environments. An example is contained in the repository [ToDo_AzureDeploymentEnvironment](https://github.com/JFolberth/ToDo_AzureDeploymentEnvironment/blob/main/YAML/demo_environment_template.yml)| [bicep_dotnet_build_stage.yml](stages/bicep_dotnet_build_stage.yml) | DotNet SQL Build Stage | Run a `dotnet build` against a SQL Project. Will also include a list of folders to produce as artifacts for future stage use An example is contained in the repository [cicd-adventureWorks](https://github.com/JFolberth/cicd-adventureWorks/blob/main/YAML/azure-pipeline.yml)| [dotnet_sql_build_stage.yml](stages/dotnet_sql_build_stage.yml) | Dacpac Deploy Stage | Leverages the [SqlAzureDacpacDeployment@1 task](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/sql-azure-dacpac-deployment-v1?view=azure-pipelines) to take a dacpac file and deploy it against an existing SQL Server and Database. An example is contained in the repository [cicd-adventureWorks](https://github.com/JFolberth/cicd-adventureWorks/blob/main/YAML/azure-pipeline.yml)| [dacpac_deploy_stage.yml](stages/dacpac_deploy_stage.yml) +| Terraform Build AWS| Will install Terraform, init, plan in parallel across environment and regions. Does require [Azure Pipelines Terraform Task](https://marketplace.visualstudio.com/items?itemName=JasonBJohnson.azure-pipelines-tasks-terraform&ssr=false#overview) and the remote backend as configured in the [aws_terraform_dev2_variables.yml](variables/aws_terraform_dev2_variables.yml) By default Terraform files are expected to be in a folder called 'infrastructure' and leverage a variable file like dev.eus.variables.tfvars. This is easily changed be changed as needed for an organization by updating the 'TerraformWorkingDirectory' in the variables file and/or the 'additionalParameters' in the stage | [terraform_build_aws_stage.yml](stages/terraform_build_aws_stage.yml) +| Terraform Apply | Will install Terraform, init, plan, apply across environment and regions with a stage for each region/environment combination Does require [Azure Pipelines Terraform Task](https://marketplace.visualstudio.com/items?itemName=JasonBJohnson.azure-pipelines-tasks-terraform&ssr=false#overview) and the remote backend as configured in the [aws_terraform_dev2_variables.yml](variables/aws_terraform_dev_variables.yml). By default Terraform files are expected to be in a folder called 'infrastructure' and leverage a variable file like dev.eus.variables.tfvars. This is easily changed be changed as needed for an organization by updating the 'TerraformWorkingDirectory' in the variables file and/or the 'additionalParameters' in the stage. | [terraform_apply_aws_stage.yml](stages/terraform_apply_aws_stage.yml) # Additional Resources - [Series of Post on Microsoft's Health and Life Sciences Blogs covering YAML Pipelines](https://techcommunity.microsoft.com/t5/healthcare-and-life-sciences/bg-p/HealthcareAndLifeSciencesBlog/label-name/YAML%20Pipeline%20Series) diff --git a/jobs/terraform_apply_aws_env_job.yml b/jobs/terraform_apply_aws_env_job.yml new file mode 100644 index 0000000..1991542 --- /dev/null +++ b/jobs/terraform_apply_aws_env_job.yml @@ -0,0 +1,55 @@ +parameters: +- name: environmentName + type: string + default: 'dev' +- name: serviceName + type: string +- name: terraformVersion + type: string + default: 'latest' +- name: additionalParameters + type: object + default: [] +- name: templateDirectory + type: string + default: 'infrastructure' + +jobs: +- deployment: terraformApply${{ parameters.environmentName }} + displayName: Terraform Apply ${{ parameters.environmentName }} + variables: + - template: ../variables/aws_terraform_${{ parameters.environmentName }}_variables.yml + - name: commandOptions + value: '' + environment: ${{ parameters.environmentName }} + strategy: + runOnce: + deploy: + steps: + - download: current + artifact: ${{ parameters.templateDirectory }} + - template: ../tasks/terraform_install_task.yml + parameters: + terraformVersion: ${{ parameters.terraformVersion }} + + - template: ../tasks/terraform_init_aws_task.yml + parameters: + serviceName: ${{ parameters.serviceName }} + TerraformDirectory: $(Pipeline.Workspace)/${{ variables.workingDirectory }} + backendServiceAws: ${{ variables.backendServiceAws }} + backendAwsRegion: ${{ variables.backendAwsRegion }} + backendAwsBucket: ${{ variables.backendAwsBucket }} + + - template: ../tasks/terraform_plan_aws_task.yml + parameters: + TerraformDirectory: $(Pipeline.Workspace)/${{ variables.workingDirectory }} + backendServiceAws: ${{ variables.backendServiceAws }} + providerAwsRegion: ${{ variables.backendAwsRegion }} + additionalParameters: ${{ parameters.additionalParameters }} + publishPlanResults: 'CI - ${{ parameters.environmentName }} - ${{ parameters.serviceName }}' + + - template: ../tasks/terraform_apply_aws_task.yml + parameters: + TerraformDirectory: $(Pipeline.Workspace)/${{ variables.workingDirectory }} + backendServiceAws: ${{ variables.backendServiceAws }} + providerAwsRegion: ${{ variables.backendAwsRegion }} diff --git a/jobs/terraform_build_aws_env_job.yml b/jobs/terraform_build_aws_env_job.yml new file mode 100644 index 0000000..b6dbba4 --- /dev/null +++ b/jobs/terraform_build_aws_env_job.yml @@ -0,0 +1,41 @@ +parameters: +- name: environmentName + type: string + default: 'dev' +- name: serviceName + type: string +- name: terraformVersion + type: string + default: 'latest' +- name: additionalParameters + type: object + default: [] +- name: dependsOn + type: object + default: [] +jobs: +- job: 'buildTerraform_${{ parameters.environmentName }}' + dependsOn: ${{parameters.dependsOn}} + displayName: Build Terraform ${{ parameters.environmentName }} + variables: + - template: ../variables/aws_terraform_${{ parameters.environmentName }}_variables.yml + steps: + - template: ../tasks/terraform_install_task.yml + parameters: + terraformVersion: ${{ parameters.terraformVersion }} + + - template: ../tasks/terraform_init_aws_task.yml + parameters: + serviceName: ${{ parameters.serviceName }} + TerraformDirectory: ${{ variables.workingDirectory }} + backendServiceAws: ${{ variables.backendServiceAws }} + backendAwsRegion: ${{ variables.backendAwsRegion }} + backendAwsBucket: ${{ variables.backendAwsBucket }} + + - template: ../tasks/terraform_plan_aws_task.yml + parameters: + TerraformDirectory: ${{ variables.workingDirectory }} + backendServiceAws: ${{ variables.backendServiceAws }} + providerAwsRegion: ${{ variables.backendAwsRegion }} + additionalParameters: ${{ parameters.additionalParameters }} + publishPlanResults: 'CI - ${{ parameters.environmentName }} - ${{ parameters.serviceName }}' diff --git a/stages/terraform_apply_aws_stage.yml b/stages/terraform_apply_aws_stage.yml new file mode 100644 index 0000000..f1b70a8 --- /dev/null +++ b/stages/terraform_apply_aws_stage.yml @@ -0,0 +1,28 @@ +parameters: +- name: environmentObjects + type: object + default: + - environmentName: 'dev' + regionAbrvs: ['cus'] +- name: templateFileName + type: string + default: 'main' +- name: templateDirectory + type: string + default: 'infrastructure' +- name: serviceName + type: string + default: 'SampleApp' + +stages: + - ${{ each environmentObject in parameters.environmentObjects }} : + - ${{ each regionAbrv in environmentObject.regionAbrvs }} : + - stage: '${{ parameters.serviceName }}_${{ environmentObject.environmentName}}_${{regionAbrv}}_tf_apply' + jobs: + - template: ../jobs/terraform_apply_aws_env_job.yml + parameters: + environmentName: ${{ environmentObject.environmentName }} + serviceName: ${{ parameters.serviceName }} + additionalParameters: ['-var-file="variables/${{ environmentObject.environmentName }}.${{regionAbrv}}.variables.tfvars"'] + templateDirectory: ${{ parameters.templateDirectory }} + diff --git a/stages/terraform_build_aws_stage.yml b/stages/terraform_build_aws_stage.yml new file mode 100644 index 0000000..20a5854 --- /dev/null +++ b/stages/terraform_build_aws_stage.yml @@ -0,0 +1,32 @@ +parameters: +- name: environmentObjects + type: object + default: + - environmentName: 'dev' + regionAbrvs: ['cus'] +- name: templateFileName + type: string + default: 'main' +- name: templateDirectory + type: string + default: 'infrastructure' +- name: serviceName + type: string + default: 'SampleApp' + +stages: +- stage: '${{ parameters.serviceName }}_tf_build' + jobs: + - template: ../jobs/ado_publish_job.yml + parameters: + targetPath: ${{ parameters.templateDirectory }} + artifactname: ${{ parameters.templateDirectory }} + - ${{ each environmentObject in parameters.environmentObjects }} : + - ${{ each regionAbrv in environmentObject.regionAbrvs }} : + - template: ../jobs/terraform_build_aws_env_job.yml + parameters: + environmentName: ${{ environmentObject.environmentName }} + serviceName: ${{ parameters.serviceName }} + additionalParameters: ['-var-file="variables/${{ environmentObject.environmentName }}.${{regionAbrv}}.variables.tfvars"'] + + diff --git a/tasks/terraform_apply_aws_task.yml b/tasks/terraform_apply_aws_task.yml new file mode 100644 index 0000000..676d97f --- /dev/null +++ b/tasks/terraform_apply_aws_task.yml @@ -0,0 +1,21 @@ +parameters: +- name: TerraformDirectory + type: string +- name: backendServiceAws + type: string +- name: providerAwsRegion + type: string +- name: additionalParameters + type: object + default: [] + +steps: +- task: TerraformCLI@1 + displayName: 'Terraform : apply' + condition: and(succeeded(), eq(variables['TERRAFORM_PLAN_HAS_CHANGES'],'true')) + inputs: + command: apply + workingDirectory: ${{ parameters.TerraformDirectory }} + commandOptions: '$(System.DefaultWorkingDirectory)/terraform.tfplan' + providerServiceAws: ${{ parameters.backendServiceAws }} + providerAwsRegion: ${{ parameters.providerAwsRegion }} diff --git a/tasks/terraform_apply_task.yml b/tasks/terraform_apply_task.yml index eeb3fd0..2533317 100644 --- a/tasks/terraform_apply_task.yml +++ b/tasks/terraform_apply_task.yml @@ -1,7 +1,7 @@ parameters: - name: TerraformDirectory type: string -- name: AzureSubscriptionServiceConnectionName +- name: backendServiceAws type: string - name: additionalParameters type: object diff --git a/tasks/terraform_init_aws_task.yml b/tasks/terraform_init_aws_task.yml new file mode 100644 index 0000000..4c044f4 --- /dev/null +++ b/tasks/terraform_init_aws_task.yml @@ -0,0 +1,24 @@ +parameters: +- name: serviceName + type: string +- name: TerraformDirectory + type: string +- name: backendServiceAws + type: string +- name: backendAwsRegion + type: string +- name: backendAwsBucket + type: string + + +steps: + - task: TerraformCLI@1 + displayName: 'Terraform : init' + inputs: + command: init + backendType: aws + workingDirectory: ${{ parameters.TerraformDirectory }} + backendServiceAws: ${{ parameters.backendServiceAws }} + backendAwsRegion: ${{ parameters.backendAwsRegion }} + backendAwsBucket: ${{ parameters.backendAwsBucket }} + backendAwsKey: ${{ parameters.serviceName }}.tfstate diff --git a/tasks/terraform_plan_aws_task.yml b/tasks/terraform_plan_aws_task.yml new file mode 100644 index 0000000..41e9fec --- /dev/null +++ b/tasks/terraform_plan_aws_task.yml @@ -0,0 +1,24 @@ +parameters: +- name: TerraformDirectory + type: string +- name: backendServiceAws + type: string +- name: outputFile + default: 'terraform.tfplan' +- name: additionalParameters + type: object + default: [] +- name: providerAwsRegion + type: string +- name: publishPlanResults + default: '' +steps: + - task: TerraformCLI@1 + displayName: 'Terraform : plan' + inputs: + command: plan + workingDirectory: ${{ parameters.TerraformDirectory }} + publishPlanResults: ${{ parameters.publishPlanResults }} + providerServiceAws: ${{ parameters.backendServiceAws }} + providerAwsRegion: ${{ parameters.providerAwsRegion }} + commandOptions: "-lock=false ${{ join(' ', parameters.additionalParameters) }} -out=$(System.DefaultWorkingDirectory)/${{ parameters.outputFile }} -detailed-exitcode" diff --git a/variables/aws_terraform_dev2_variables.yml b/variables/aws_terraform_dev2_variables.yml new file mode 100644 index 0000000..d581c05 --- /dev/null +++ b/variables/aws_terraform_dev2_variables.yml @@ -0,0 +1,5 @@ +variables: + backendServiceAws: awsdev2 + workingDirectory: infrastructure + backendAwsBucket: jfmsterraformstate + backendAwsRegion: us-east-1