Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Auto-determine checks to run #106

Draft
wants to merge 3 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 9 additions & 63 deletions .github/workflows/build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -7,73 +7,19 @@ on:
pull_request:

jobs:
build:
name: 'PHP ${{ matrix.php.branch }}'
prepare-ci:
name: 'Prepare actions'
runs-on: ubuntu-latest
strategy:
matrix:
php:
- branch: '8.2'
- branch: '8.1'
- branch: '8.0'
- branch: '7.4'
- branch: '7.3'
- branch: '7.2'
- branch: '7.1'
- branch: '7.0'
- branch: '5.6'
# We want to fix failures individually.
fail-fast: false
steps:
- uses: actions/checkout@v3

- name: Install Nix
uses: cachix/install-nix-action@v18

- name: Set up Nix cache
uses: cachix/cachix-action@v12
with:
name: fossar
authToken: '${{ secrets.CACHIX_AUTH_TOKEN }}'

- name: Set job parameters
id: params
run: |
branch=${{ matrix.php.branch }}
major=${branch%%.*}
minor=${branch#*.}
attr=php$major$minor
echo "major=$major" >> $GITHUB_OUTPUT
echo "attr=$attr" >> $GITHUB_OUTPUT

- name: Build PHP
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-php

- name: Build Imagick extension
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-imagick

- name: Build Redis extension
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-redis

- name: Build Redis 3 extension
if: ${{ steps.params.outputs.major < 8 }}
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-redis3

- name: Build MySQL extension
if: ${{ steps.params.outputs.major < 7 }}
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-mysql

- name: Build Xdebug extension
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-xdebug

- name: Build Tidy extension
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-tidy

- name: Check that composer PHAR works
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-composer-phar

- name: Validate php.extensions.mysqli default unix socket path
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-mysqli-socket-path

- name: Validate php.extensions.pdo_mysql default unix socket path
run: nix-build -A outputs.checks.x86_64-linux.${{ steps.params.outputs.attr }}-pdo_mysql-socket-path
- name: Generate actions
run: nix-shell .github/workflows/workflow-from-flake-checks.nix
test:
needs:
- prepare-ci
uses: ./.github/workflows/test.yaml
secrets: inherit
Comment on lines +19 to +25
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not do anything. If we decide to go the way of generated CI file, we should probably drop this file, or replace it with a check that the generated workflow is in sync with the checks.

134 changes: 134 additions & 0 deletions .github/workflows/test.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
{
"jobs": {
"test": {
"name": "PHP ${{ matrix.php.branch }}",
"runs-on": "${{ matrix.os.image }}",
"steps": [
{
"uses": "actions/checkout@v3"
},
{
"name": "Install Nix",
"uses": "cachix/install-nix-action@v18"
},
{
"name": "Set up Nix cache",
"uses": "cachix/cachix-action@v10",
"with": {
"authToken": "${{ secrets.CACHIX_AUTH_TOKEN }}",
"name": "fossar"
}
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\",\"php70\",\"php71\",\"php72\",\"php73\",\"php74\",\"php80\",\"php81\",\"php82\"]'), matrix.php.attr) }}",
"name": "Check that composer PHAR works",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-composer-phar"
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\",\"php70\",\"php71\",\"php72\",\"php73\",\"php74\",\"php80\",\"php81\",\"php82\"]'), matrix.php.attr) }}",
"name": "Build Imagick extension",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-imagick"
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\"]'), matrix.php.attr) }}",
"name": "Build MySQL extension",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-mysql"
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\",\"php70\",\"php71\",\"php72\",\"php73\",\"php74\",\"php80\",\"php81\",\"php82\"]'), matrix.php.attr) }}",
"name": "Validate php.extensions.mysqli default unix socket path",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-mysqli-socket-path"
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\",\"php70\",\"php71\",\"php72\",\"php73\",\"php74\",\"php80\",\"php81\",\"php82\"]'), matrix.php.attr) }}",
"name": "Validate php.extensions.pdo_mysql default unix socket path",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-pdo_mysql-socket-path"
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\",\"php70\",\"php71\",\"php72\",\"php73\",\"php74\",\"php80\",\"php81\",\"php82\"]'), matrix.php.attr) }}",
"name": "Build PHP",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-php"
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\",\"php70\",\"php71\",\"php72\",\"php73\",\"php74\",\"php80\",\"php81\",\"php82\"]'), matrix.php.attr) }}",
"name": "Build Redis extension",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-redis"
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\",\"php70\",\"php71\",\"php72\",\"php73\",\"php74\"]'), matrix.php.attr) }}",
"name": "Build Redis 3 extension",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-redis3"
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\",\"php70\",\"php71\",\"php72\",\"php73\",\"php74\",\"php80\",\"php81\",\"php82\"]'), matrix.php.attr) }}",
"name": "Build Tidy extension",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-tidy"
},
{
"if": "${{ contains(fromJSON('[\"aarch64-darwin\",\"aarch64-linux\",\"x86_64-darwin\",\"x86_64-linux\"]'), matrix.os.platform) && contains(fromJSON('[\"php56\",\"php70\",\"php71\",\"php72\",\"php73\",\"php74\",\"php80\",\"php81\",\"php82\"]'), matrix.php.attr) }}",
"name": "Build Xdebug extension",
"run": "nix-build -A outputs.checks.${{ matrix.os.platform }}.${{ matrix.php.attr }}-xdebug"
}
],
"strategy": {
"fail-fast": false,
"matrix": {
"os": [
{
"image": "ubuntu-latest",
"platform": "x86_64-linux"
}
],
"php": [
{
"attr": "php56",
"branch": "5.6"
},
{
"attr": "php70",
"branch": "7.0"
},
{
"attr": "php71",
"branch": "7.1"
},
{
"attr": "php72",
"branch": "7.2"
},
{
"attr": "php73",
"branch": "7.3"
},
{
"attr": "php74",
"branch": "7.4"
},
{
"attr": "php80",
"branch": "8.0"
},
{
"attr": "php81",
"branch": "8.1"
},
{
"attr": "php82",
"branch": "8.2"
}
]
}
}
}
},
"name": "Test",
"on": {
"workflow_call": {
"secrets": {
"CACHIX_AUTH_TOKEN": {
"required": true
}
}
}
}
}
151 changes: 151 additions & 0 deletions .github/workflows/workflow-from-flake-checks.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# Runs Nix flake checks individually with group markers for GitHub Actions.
# Invoke with `nix-shell workflow-from-flake-checks.nix`

let
self = import ../..;

pkgs = self.inputs.nixpkgs.legacyPackages.${builtins.currentSystem};
inherit (self.inputs.nixpkgs) lib;

checkAttrs =
lib.foldl'
(
acc:
{
name,
value,
}:

acc // {
"${name}" = {
inherit (value) description;
platforms = acc.${name}.platforms or {} // { "${value.platform}" = null; };
phps = acc.${name}.phps or {} // { "${value.php}" = null; };
};
}
)
{}
(
builtins.concatLists (
lib.mapAttrsToList
(
platform:
checks:

lib.mapAttrsToList
(
attr:
check:

let
match = builtins.match "(php[0-9]+)-(.+)" attr;
in
{
name = builtins.head (builtins.tail match);
value = {
inherit (check) description;
inherit platform;
php = builtins.head match;
};
}
)
checks
)
self.outputs.checks
)
);

packages = self.outputs.packages.${builtins.currentSystem};
phpPackages = builtins.filter (name: builtins.match "php[0-9]+" name != null) (builtins.attrNames packages);

workflow = {
name = "Test";

on = {
workflow_call = {
secrets = {
CACHIX_AUTH_TOKEN = {
required = true;
};
};
};
};

jobs = {
test = {
name = "PHP \${{ matrix.php.branch }}";
runs-on = "\${{ matrix.os.image }}";
steps = [
{
uses = "actions/checkout@v3";
}
{
name = "Install Nix";
uses = "cachix/install-nix-action@v18";
}
{
name = "Set up Nix cache";
uses = "cachix/cachix-action@v10";
"with" = {
authToken = "\${{ secrets.CACHIX_AUTH_TOKEN }}";
name = "fossar";
};
}
]
++ lib.mapAttrsToList
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or maybe it would be enough to add sort here, that would ensure php test is first (not sure if there are any other dependencies – if yes, we should probably express them in checks.nix and use toposort).

(
attr:
check:

{
name = check.description;
"if" = "\${{ contains(fromJSON('${lib.escape [ "\\" "'" ] (builtins.toJSON (builtins.attrNames check.platforms))}'), matrix.os.platform) && contains(fromJSON('${lib.escape [ "\\" "'" ] (builtins.toJSON (builtins.attrNames check.phps))}'), matrix.php.attr) }}";
run = "nix-build -A outputs.checks.\${{ matrix.os.platform }}.\${{ matrix.php.attr }}-${attr}";
}
)
checkAttrs
;
strategy = {
# We want to fix failures individually.
fail-fast = false;
matrix = {
os = [
{
image = "ubuntu-latest";
platform = "x86_64-linux";
}
];
php =
builtins.map
(
phpAttr:

{
branch = lib.versions.majorMinor packages.${phpAttr}.version;
attr = phpAttr;
}
)
phpPackages;
};
};
};
};
};
in

pkgs.stdenv.mkDerivation {
name = "workflow-from-flake-checks";

buildCommand = ''
echo 'Please run `nix-shell .github/workflows/workflow-from-flake-checks.nix`, `nix-build` cannot be used.' > /dev/stderr
exit 1
'';

shellHook =
''
set -o errexit
echo ${lib.escapeShellArgs [ (builtins.toJSON workflow) ]} | ${pkgs.jq}/bin/jq . > .github/workflows/test.yaml
exit 0
''
;
}
Loading