Skip to content

Commit

Permalink
Fix Public vs Private project Baselines and Rules #45 (#91)
Browse files Browse the repository at this point in the history
* Fix Public vs Private project Baselines and Rules #45

* Add missing rules
  • Loading branch information
webtonize authored Dec 28, 2023
1 parent bfc3fee commit bc6173a
Show file tree
Hide file tree
Showing 12 changed files with 301 additions and 3 deletions.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,6 +203,8 @@ in building the ruleset for this module.
- [Azure.DevOps.Pipelines.Settings.RequireCommentForPullRequestFromFork](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Pipelines.Settings.RequireCommentForPullRequestFromFork.md)
- [Azure.DevOps.Pipelines.Settings.RestrictSecretsForPullRequestFromFork](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Pipelines.Settings.RestrictSecretsForPullRequestFromFork.md)
- [Azure.DevOps.Pipelines.Settings.SanitizeShellTaskArguments](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Pipelines.Settings.SanitizeShellTaskArguments.md)
- [Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate.md)
- [Azure.DevOps.Project.Visibility](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Project.Visibility.md)
- [Azure.DevOps.Repos.Branch.BranchPolicyAllowSelfApproval](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Repos.Branch.BranchPolicyAllowSelfApproval.md)
- [Azure.DevOps.Repos.Branch.BranchPolicyCommentResolution](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Repos.Branch.BranchPolicyCommentResolution.md)
- [Azure.DevOps.Repos.Branch.BranchPolicyEnforceLinkedWorkItems](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Repos.Branch.BranchPolicyEnforceLinkedWorkItems.md)
Expand All @@ -226,6 +228,8 @@ in building the ruleset for this module.
- [Azure.DevOps.Repos.InheritedPermissions](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Repos.InheritedPermissions.md)
- [Azure.DevOps.Repos.License](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Repos.License.md)
- [Azure.DevOps.Repos.Readme](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Repos.Readme.md)
- [Azure.DevOps.RetentionSettings.ArtifactMinimumRetentionDays](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.RetentionSettings.ArtifactMinimumRetentionDays.md)
- [Azure.DevOps.RetentionSettings.PullRequestRunsMinimumRetentionDays](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.RetentionSettings.PullRequestRunsMinimumRetentionDays.md)
- [Azure.DevOps.ServiceConnections.ClassicAzure](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.ServiceConnections.ClassicAzure.md)
- [Azure.DevOps.ServiceConnections.Description](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.ServiceConnections.Description.md)
- [Azure.DevOps.ServiceConnections.GitHubPAT](./src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.ServiceConnections.GitHubPAT.md)
Expand Down
42 changes: 42 additions & 0 deletions src/PSRule.Rules.AzureDevOps/Functions/Common.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -149,3 +149,45 @@ function Get-AzDevOpsProject {
}
}
# End of Function Get-AzDevOpsProject

<#
.SYNOPSIS
Export the Azure DevOps Project
.DESCRIPTION
Export the Azure DevOps Project using Azure DevOps Rest API to a JSON file
.EXAMPLE
Export-AzDevOpsProject -Project $Project -OutputPath $OutputPath
.NOTES
The output file will be named $Project.prj.ado.json
#>
function Export-AzDevOpsProject {
[CmdletBinding()]
param (
[Parameter(Mandatory=$true)]
[string]
$Project,
[Parameter(Mandatory=$true)]
[string]
$OutputPath
)
if ($null -eq $script:connection) {
throw "Not connected to Azure DevOps. Run Connect-AzDevOps first"
}
$header = $script:connection.GetHeader()
$Organization = $script:connection.Organization
Write-Verbose "Getting project $Project for organization $Organization"
try {
$response = Get-AzDevOpsProject -Project $Project
$response | Add-Member -MemberType NoteProperty -Name ObjectType -Value "Azure.DevOps.Project"
$response | Add-Member -MemberType NoteProperty -Name ObjectName -Value "$Organization.$Project"
}
catch {
throw "Failed to get project $Project from Azure DevOps"
}
$response | ConvertTo-Json | Out-File -FilePath "$OutputPath/$Project.prj.ado.json"
}
# End of Function Export-AzDevOpsProject
3 changes: 3 additions & 0 deletions src/PSRule.Rules.AzureDevOps/PSRule.Rules.AzureDevOps.psm1
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,8 @@ Function Export-AzDevOpsRuleData {
$OutputPath
)
Write-Verbose "Exporting rule data for project $Project to $OutputPath"
Write-Verbose "Exporting project"
Export-AzDevOpsProject -Project $Project -OutputPath $OutputPath
Write-Verbose "Exporting repos and branch policies"
Export-AzDevOpsReposAndBranchPolicies -Project $Project -OutputPath $OutputPath
Write-Verbose "Exporting environment checks"
Expand Down Expand Up @@ -98,6 +100,7 @@ Export-ModuleMember -Function Export-AzDevOpsOrganizationRuleData
# End of Function Export-AzDevOpsOrganizationRuleData

Export-ModuleMember -Function Get-AzDevOpsProject
Export-ModuleMember -Function Export-AzDevOpsProject
Export-ModuleMember -Function Connect-AzDevOps
Export-ModuleMember -Function Disconnect-AzDevOps

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
---
category: Microsoft Azure DevOps Pipelines
severity: Severe
online version: https://github.com/cloudyspells/PSRule.Rules.AzureDevOps/blob/main/src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate.md
---

# Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate

## SYNOPSIS

Status badges should not be publicly accessible.

## DESCRIPTION

Status badges are publicly accessible by default. This means anyone with the URL can view
the status of a pipeline. Consider restricting access to status badges to prevent
unauthorized access.

Mininum TokenType: `ReadOnly`

## RECOMMENDATION

Consider restricting access to status badges to prevent unauthorized access.

## LINKS

- [Azure DevOps Security best practices](https://learn.microsoft.com/en-us/azure/devops/organizations/security/security-best-practices?view=azure-devops#tasks)
26 changes: 26 additions & 0 deletions src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Project.Visibility.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
---
category: Microsoft Azure DevOps Projects
severity: Critical
online version: https://github.com/cloudyspells/PSRule.Rules.AzureDevOps/blob/main/src/PSRule.Rules.AzureDevOps/en/Azure.DevOps.Project.Visibility.md
---

# Azure.DevOps.Project.Visibility

## SYNOPSIS

Projects should not be publicly accessible.

## DESCRIPTION

Projects can be configured to be publicly accessible. This means anyone with the URL can
view the project. Consider restricting access to projects to prevent unauthorized access.

Mininum TokenType: `ReadOnly`

## RECOMMENDATION

Consider restricting access to projects to prevent unauthorized access.

## LINKS

- [Azure DevOps Security best practices](https://learn.microsoft.com/en-us/azure/devops/organizations/security/security-best-practices?view=azure-devops#tasks)
Original file line number Diff line number Diff line change
Expand Up @@ -97,3 +97,17 @@ Rule 'Azure.DevOps.Pipelines.Settings.SanitizeShellTaskArguments' `
$Assert.HasField($TargetObject, "enableShellTasksArgsSanitizing", $true)
$Assert.HasFieldValue($TargetObject, "enableShellTasksArgsSanitizing", $true)
}

# Synopsis: Status badges should be private
Rule 'Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate' `
-Ref 'ADO-PLS-008' `
-Type 'Azure.DevOps.Pipelines.Settings' `
-Tag @{ release = 'GA'} `
-Level Warning {
# Description: Status badges should be private.
Reason 'Status badges are not private.'
Recommend 'Enable `Status badges should be private` in Project pipelines settings.'
# Links: https://learn.microsoft.com/en-us/azure/devops/organizations/security/security-best-practices?view=azure-devops#policies
$Assert.HasField($TargetObject, "statusBadgesArePrivate", $true)
$Assert.HasFieldValue($TargetObject, "statusBadgesArePrivate", $true)
}
17 changes: 17 additions & 0 deletions src/PSRule.Rules.AzureDevOps/rules/AzureDevOps.Projects.Rule.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# PSRule rule definitions for Azure DevOps Pipelines definitions

# Synopsis: Pipelines should use YAML definitions
Rule 'Azure.DevOps.Project.Visibility' `
-Ref 'ADO-PRJ-001' `
-Type 'Azure.DevOps.Project' `
-Tag @{ release = 'GA'} `
-Level Warning {
# Description "Projects should not be public"
Reason "The project is public"
Recommend "Consider making the project private"
# Links "https://learn.microsoft.com/en-us/azure/devops/organizations/security/security-best-practices?view=azure-devops#definitions"
AllOf {
$Assert.HasField($TargetObject, "visibility", $true)
$Assert.HasFieldValue($TargetObject, "visibility", "private")
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
apiVersion: github.com/microsoft/PSRule/v1
kind: Baseline
metadata:
name: Baseline.PublicProject
spec:
rule:
exclude:
- 'Azure.DevOps.Project.Visibility'
- 'Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate'
tag:
release: GA
configuration:
ghasEnabled: true
ghasBlockPushesEnabled: true
branchMinimumApproverCount: 1
releaseMinimumProductionApproverCount: 1
50 changes: 49 additions & 1 deletion tests/Common.Tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -235,8 +235,56 @@ Describe "Functions: Common.Tests" {
} | Should -Throw
}
}

Context " Export-AzDevOpsProject without a connection" {
It " should throw an error" {
{
Disconnect-AzDevOps
Export-AzDevOpsProject -Project $env:ADO_PROJECT -OutputPath $env:ADO_EXPORT_DIR
} | Should -Throw "Not connected to Azure DevOps. Run Connect-AzDevOps first"
}
}

Context " Export-AzDevOpsProject" {
BeforeAll {
Connect-AzDevOps -Organization $env:ADO_ORGANIZATION -PAT $env:ADO_PAT
$project = Get-AzDevOpsProject -Project $env:ADO_PROJECT
Export-AzDevOpsProject -Project $project.name -OutputPath $env:ADO_EXPORT_DIR
}

It " The output folder should contain a file named $env:ADO_PROJECT.prj.ado.json" {
$file = Join-Path -Path $env:ADO_EXPORT_DIR -ChildPath "$env:ADO_PROJECT.prj.ado.json"
Test-Path -Path $file | Should -Be $true
}

It " The output folder should contain a file named $env:ADO_PROJECT.prj.ado.json with a size greater than 0" {
$file = Join-Path -Path $env:ADO_EXPORT_DIR -ChildPath "$env:ADO_PROJECT.prj.ado.json"
(Get-Item $file).length | Should -BeGreaterThan 0
}

It " Should throw on a non-existing project" {
{
Export-AzDevOpsProject -Project 'wrong' -OutputPath $env:ADO_EXPORT_DIR
} | Should -Throw
}

It " The operation should fail with a wrong Organization" {
{
Disconnect-AzDevOps
Connect-AzDevOps -Organization 'wrong' -PAT $env:ADO_PAT
Export-AzDevOpsProject -Project $env:ADO_PROJECT -OutputPath $env:ADO_EXPORT_DIR -ErrorAction Stop
} | Should -Throw
}

It " The operation should fail with a wrong PAT" {
{
Disconnect-AzDevOps
Connect-AzDevOps -Organization $env:ADO_ORGANIZATION -PAT 'wrong'
Export-AzDevOpsProject -Project $env:ADO_PROJECT -OutputPath $env:ADO_EXPORT_DIR -ErrorAction Stop
} | Should -Throw
}
}
}

AfterAll {

}
4 changes: 2 additions & 2 deletions tests/Rules.Common.Tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,9 @@ BeforeAll {

Describe "PSRule.Rules.AzureDevOps Rules" {
Context ' Base rules' {
It ' should contain 59 rules' {
It ' should contain 61 rules' {
$rules = Get-PSRule -Module PSRule.Rules.AzureDevOps
$rules.Count | Should -Be 59
$rules.Count | Should -Be 61
}

It ' should contain a markdown help file for each rule' {
Expand Down
33 changes: 33 additions & 0 deletions tests/Rules.Pipelines.Settings.Tests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@ BeforeAll {
# Run rules with default token type
$ruleResult = Invoke-PSRule -InputPath "$($outPath)/" -Module PSRule.Rules.AzureDevOps -Format Detect -Culture en

# Run rules with the public baseline
$ruleResultPublic = Invoke-PSRule -InputPath "$($outPath)/" -Module PSRule.Rules.AzureDevOps -Format Detect -Culture en -Baseline Baseline.PublicProject

# Get temporary test output folder for tests with the ReadOnly TokenType
$outPathReadOnly = Get-Item -Path (Join-Path -Path $here -ChildPath 'outReadOnly')
$outPathReadOnly = $outPathReadOnly.FullName
Expand Down Expand Up @@ -212,6 +215,36 @@ Describe "Azure.DevOps.Pipelines.Settings rules" {
$fileExists | Should -Be $true;
}
}

Context ' Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate' {
It ' should Pass' {
$ruleHits = @($ruleResult | Where-Object { $_.RuleName -eq 'Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate' })
$ruleHits[0].Outcome | Should -Be 'Pass';
$ruleHits.Count | Should -Be 1;
}

It ' should be the same for ReadOnly TokenType' {
$ruleHits = @($ruleResultReadOnly | Where-Object { $_.RuleName -eq 'Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate' })
$ruleHits[0].Outcome | Should -Be 'Pass';
$ruleHits.Count | Should -Be 1;
}

It ' should be the same for the FineGrained TokenType' {
$ruleHits = @($ruleResultFineGrained | Where-Object { $_.RuleName -eq 'Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate' })
$ruleHits[0].Outcome | Should -Be 'Pass';
$ruleHits.Count | Should -Be 1;
}

It ' should have an English markdown help file' {
$fileExists = Test-Path -Path (Join-Path -Path $ourModule -ChildPath 'en/Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate.md');
$fileExists | Should -Be $true;
}

It ' should not be present in the PublicProject baseline' {
$ruleHits = @($ruleResultPublic | Where-Object { $_.RuleName -eq 'Azure.DevOps.Pipelines.Settings.StatusBadgesPrivate' })
$ruleHits.Count | Should -Be 0;
}
}
}

AfterAll {
Expand Down
68 changes: 68 additions & 0 deletions tests/Rules.Projects.Tests.ps1
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
BeforeAll {
# Setup error handling
$ErrorActionPreference = 'Stop';
Set-StrictMode -Version latest;

if ($Env:SYSTEM_DEBUG -eq 'true') {
$VerbosePreference = 'Continue';
}

# Setup tests paths
# $rootPath = $PWD;
$rootPath = $env:GITHUB_WORKSPACE
$ourModule = (Join-Path -Path $rootPath -ChildPath '/src/PSRule.Rules.AzureDevOps')

Import-Module -Name $ourModule -Force
$here = (Resolve-Path $PSScriptRoot).Path

# Get tempory test output folder and store path
$outPath = Get-Item -Path (Join-Path -Path $here -ChildPath 'out')
$outPath = $outPath.FullName

# Run rules with default token type
$ruleResult = Invoke-PSRule -InputPath "$($outPath)/" -Module PSRule.Rules.AzureDevOps -Format Detect -Culture en

# Run rules with the public baseline
$ruleResultPublic = Invoke-PSRule -InputPath "$($outPath)/" -Module PSRule.Rules.AzureDevOps -Format Detect -Culture en -Baseline Baseline.PublicProject

# Get temporary test output folder for tests with the ReadOnly TokenType
$outPathReadOnly = Get-Item -Path (Join-Path -Path $here -ChildPath 'outReadOnly')
$outPathReadOnly = $outPathReadOnly.FullName

# Run rules with ReadOnly token type
$ruleResultReadOnly = Invoke-PSRule -InputPath "$($outPathReadOnly)/" -Module PSRule.Rules.AzureDevOps -Format Detect -Culture en

# Get temporary test output folder for tests with the FineGrained TokenType
$outPathFineGrained = Get-Item -Path (Join-Path -Path $here -ChildPath 'outFineGrained')
$outPathFineGrained = $outPathFineGrained.FullName

# Run rules with FineGrained token type
$ruleResultFineGrained = Invoke-PSRule -InputPath "$($outPathFineGrained)/" -Module PSRule.Rules.AzureDevOps -Format Detect -Culture en
}

Describe "Azure.DevOps.Project rules" {
Context ' Azure.DevOps.Project.Visibility' {
It " should pass once" {
$ruleHits = @($ruleResult | Where-Object { $_.RuleName -eq 'Azure.DevOps.Project.Visibility' })
$ruleHits[0].Outcome | Should -Be 'Pass';
$ruleHits.Count | Should -Be 1;
}

It " should pass once for ReadOnly token type" {
$ruleHits = @($ruleResultReadOnly | Where-Object { $_.RuleName -eq 'Azure.DevOps.Project.Visibility' })
$ruleHits[0].Outcome | Should -Be 'Pass';
$ruleHits.Count | Should -Be 1;
}

It " should pass once for FineGrained token type" {
$ruleHits = @($ruleResultFineGrained | Where-Object { $_.RuleName -eq 'Azure.DevOps.Project.Visibility' })
$ruleHits[0].Outcome | Should -Be 'Pass';
$ruleHits.Count | Should -Be 1;
}

It " should not be present in the PublicProject baseline" {
$ruleHits = @($ruleResultPublic | Where-Object { $_.RuleName -eq 'Azure.DevOps.Project.Visibility' })
$ruleHits.Count | Should -Be 0;
}
}
}

0 comments on commit bc6173a

Please sign in to comment.