Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: improve v8 snapshot cache action to be more resilient to windows slowness and have one PR per platform #30551

Merged
merged 13 commits into from
Nov 6, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 20 additions & 10 deletions .github/workflows/update_v8_snapshot_cache.yml
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ concurrency:
jobs:
update-v8-snapshot-cache:
strategy:
max-parallel: 1
max-parallel: 3
matrix:
platform: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.platform }}
Expand All @@ -57,14 +57,20 @@ jobs:
steps:
- name: Determine snapshot files - Windows
if: ${{ matrix.platform == 'windows-latest' }}
run: echo "SNAPSHOT_FILES='tooling\v8-snapshot\cache\win32\snapshot-meta.json'" >> $GITHUB_ENV
run: |
echo "SNAPSHOT_FILES='tooling\v8-snapshot\cache\win32\snapshot-meta.json'" >> $GITHUB_ENV
echo "PLATFORM=windows" >> $GITHUB_ENV
shell: bash
- name: Determine snapshot files - Linux
if: ${{ matrix.platform == 'ubuntu-latest' }}
run: echo "SNAPSHOT_FILES='tooling/v8-snapshot/cache/linux/snapshot-meta.json'" >> $GITHUB_ENV
run: |
echo "SNAPSHOT_FILES='tooling/v8-snapshot/cache/linux/snapshot-meta.json'" >> $GITHUB_ENV
echo "PLATFORM=linux" >> $GITHUB_ENV
- name: Determine snapshot files - Mac
if: ${{ matrix.platform == 'macos-latest' }}
run: echo "SNAPSHOT_FILES='tooling/v8-snapshot/cache/darwin/snapshot-meta.json'" >> $GITHUB_ENV
run: |
echo "SNAPSHOT_FILES='tooling/v8-snapshot/cache/darwin/snapshot-meta.json'" >> $GITHUB_ENV
echo "PLATFORM=darwin" >> $GITHUB_ENV
- name: Install setuptools - Mac
if: ${{ matrix.platform == 'macos-latest' }}
run: sudo -H pip install setuptools
Expand All @@ -85,7 +91,8 @@ jobs:
node-version: 18
cache: 'yarn'
- name: Run yarn
run: yarn
# set the timeout here to try and deal with Windows slowness
run: yarn --network-timeout 300000
- name: Run build
run: yarn build
- name: Generate prod snapshot from scratch
Expand All @@ -108,14 +115,17 @@ jobs:
- name: Determine branch name - commit to separate branch
if: ${{ inputs.commit_directly_to_branch != true }}
run: |
echo "BRANCH_NAME=update-v8-snapshot-cache-on-${{ env.BASE_BRANCH }}" >> $GITHUB_ENV
echo "BRANCH_EXISTS=$(git show-ref --verify --quiet refs/remotes/origin/update-v8-snapshot-cache-on-${{ env.BASE_BRANCH }} && echo 'true')" >> $GITHUB_ENV
echo "BRANCH_NAME=update-v8-snapshot-cache-on-${{ env.BASE_BRANCH }}-${{ env.PLATFORM }}" >> $GITHUB_ENV
echo "BRANCH_EXISTS=$(git show-ref --verify --quiet refs/remotes/origin/update-v8-snapshot-cache-on-${{ env.BASE_BRANCH }}-${{ env.PLATFORM }} && echo 'true')" >> $GITHUB_ENV
shell: bash
- name: Check need for PR or branch update
id: check-need-for-pr
run: |
echo "needs_pr=${{ steps.check-for-v8-snapshot-cache-changes.outputs.has_changes == 'true' && env.BRANCH_EXISTS != 'true' }}" >> $GITHUB_OUTPUT
echo "number_of_prs_for_branch=$(gh api '/repos/cypress-io/cypress/pulls?head=cypress-io:${{ env.BRANCH_NAME }}' --jq length)" >> $GITHUB_OUTPUT
echo "needs_pr=${{ steps.check-for-v8-snapshot-cache-changes.outputs.has_changes == 'true' && env.BRANCH_NAME != env.BASE_BRANCH && steps.check-need-for-pr.outputs.number_of_prs_for_branch == '0' }}" >> $GITHUB_OUTPUT
echo "needs_branch_update=${{ steps.check-for-v8-snapshot-cache-changes.outputs.has_changes == 'true' && env.BRANCH_EXISTS == 'true' }}" >> $GITHUB_OUTPUT
env:
GH_TOKEN: ${{ secrets.BOT_GITHUB_ACTION_TOKEN }}
shell: bash
## Update available and a branch/PR already exists
- name: Checkout existing branch
Expand All @@ -128,7 +138,7 @@ jobs:
git merge --squash -Xtheirs stash
## Update available and a PR doesn't already exist
- name: Checkout new branch
if: ${{ steps.check-need-for-pr.outputs.needs_pr == 'true' }}
if: ${{ steps.check-need-for-pr.outputs.needs_branch_update != 'true' }}
run: git checkout -b ${{ env.BRANCH_NAME }} ${{ env.BASE_BRANCH }}
## Commit changes if present
- name: Commit the changes
Expand All @@ -152,7 +162,7 @@ jobs:
github,
baseBranch: '${{ env.BASE_BRANCH }}',
branchName: '${{ env.BRANCH_NAME }}',
description: 'Update v8 snapshot cache',
description: 'Update v8 snapshot cache - ${{ env.PLATFORM }}',
body: 'This PR was automatically generated by the [update-v8-snapshot-cache](https://github.com/cypress-io/cypress/actions/workflows/update_v8_snapshot_cache.yml) github action.',
reviewers: ['ryanthemanuel']
})
42 changes: 27 additions & 15 deletions tooling/v8-snapshot/cache/darwin/snapshot-meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@
"./node_modules/@cypress/request/index.js",
"./node_modules/@cypress/request/lib/helpers.js",
"./node_modules/@cypress/request/lib/querystring.js",
"./node_modules/@cypress/request/node_modules/form-data/lib/form_data.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/canonicalDomain.js",
"./node_modules/@cypress/request/request.js",
"./node_modules/@ffmpeg-installer/ffmpeg/index.js",
"./node_modules/@graphql-tools/batch-execute/index.js",
Expand Down Expand Up @@ -280,6 +280,7 @@
"./node_modules/fluent-ffmpeg/lib/utils.js",
"./node_modules/follow-redirects/index.js",
"./node_modules/forever-agent/index.js",
"./node_modules/form-data/lib/form_data.js",
"./node_modules/front-matter/index.js",
"./node_modules/front-matter/node_modules/js-yaml/lib/js-yaml/loader.js",
"./node_modules/front-matter/node_modules/js-yaml/lib/js-yaml/schema/default_full.js",
Expand Down Expand Up @@ -1069,8 +1070,6 @@
"./node_modules/@cypress/commit-info/src/index.js",
"./node_modules/@cypress/commit-info/src/utils.js",
"./node_modules/@cypress/get-windows-proxy/node_modules/debug/src/common.js",
"./node_modules/@cypress/get-windows-proxy/node_modules/registry-js/dist/lib/index.js",
"./node_modules/@cypress/get-windows-proxy/node_modules/registry-js/dist/lib/registry.js",
"./node_modules/@cypress/get-windows-proxy/src/index.js",
"./node_modules/@cypress/parse-domain/build/tries/icann.complete.json",
"./node_modules/@cypress/parse-domain/build/tries/private.complete.json",
Expand All @@ -1091,7 +1090,24 @@
"./node_modules/@cypress/request/lib/multipart.js",
"./node_modules/@cypress/request/lib/redirect.js",
"./node_modules/@cypress/request/lib/tunnel.js",
"./node_modules/@cypress/request/node_modules/form-data/lib/populate.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/constants.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/cookie.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/cookieCompare.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/cookieJar.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/defaultPath.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/domainMatch.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/formatDate.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/index.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/parseDate.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/cookie/permutePath.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/getPublicSuffix.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/memstore.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/pathMatch.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/permuteDomain.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/store.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/utils.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/validators.js",
"./node_modules/@cypress/request/node_modules/tough-cookie/dist/version.js",
"./node_modules/@electron/fuses/dist/config.js",
"./node_modules/@electron/fuses/dist/constants.js",
"./node_modules/@electron/fuses/dist/index.js",
Expand Down Expand Up @@ -1577,6 +1593,7 @@
"./node_modules/binary-extensions/index.js",
"./node_modules/binaryextensions/edition-es5/index.js",
"./node_modules/bindings/bindings.js",
"./node_modules/brace-expansion/index.js",
"./node_modules/braces/index.js",
"./node_modules/braces/lib/compile.js",
"./node_modules/braces/lib/constants.js",
Expand Down Expand Up @@ -1624,6 +1641,8 @@
"./node_modules/chrome-remote-interface/node_modules/ws/lib/sender.js",
"./node_modules/chrome-remote-interface/node_modules/ws/lib/stream.js",
"./node_modules/chrome-remote-interface/node_modules/ws/lib/validation.js",
"./node_modules/ci-info/index.js",
"./node_modules/ci-info/vendors.json",
"./node_modules/cli-truncate/index.js",
"./node_modules/coffeescript/lib/coffeescript/lexer.js",
"./node_modules/coffeescript/lib/coffeescript/parser.js",
Expand Down Expand Up @@ -1907,6 +1926,7 @@
"./node_modules/fluent-ffmpeg/lib/processor.js",
"./node_modules/fluent-ffmpeg/lib/recipes.js",
"./node_modules/follow-redirects/debug.js",
"./node_modules/form-data/lib/populate.js",
"./node_modules/forwarded/index.js",
"./node_modules/fresh/index.js",
"./node_modules/front-matter/node_modules/js-yaml/index.js",
Expand Down Expand Up @@ -2186,9 +2206,6 @@
"./node_modules/ipaddr.js/lib/ipaddr.js",
"./node_modules/is-binary-path/index.js",
"./node_modules/is-buffer/index.js",
"./node_modules/is-ci/index.js",
"./node_modules/is-ci/node_modules/ci-info/index.js",
"./node_modules/is-ci/node_modules/ci-info/vendors.json",
"./node_modules/is-core-module/core.json",
"./node_modules/is-core-module/index.js",
"./node_modules/is-dotfile/index.js",
Expand All @@ -2197,6 +2214,7 @@
"./node_modules/is-fullwidth-code-point/index.js",
"./node_modules/is-glob/index.js",
"./node_modules/is-html/index.js",
"./node_modules/is-obj/index.js",
"./node_modules/is-plain-obj/index.js",
"./node_modules/is-regexp/index.js",
"./node_modules/is-stream/index.js",
Expand Down Expand Up @@ -2460,7 +2478,6 @@
"./node_modules/mime-types/index.js",
"./node_modules/mime/types.json",
"./node_modules/min-indent/index.js",
"./node_modules/minimatch/node_modules/brace-expansion/index.js",
"./node_modules/minimist/index.js",
"./node_modules/mocha-7.0.1/lib/browser/progress.js",
"./node_modules/mocha-7.0.1/lib/context.js",
Expand All @@ -2482,7 +2499,6 @@
"./node_modules/mocha-7.0.1/lib/reporters/markdown.js",
"./node_modules/mocha-7.0.1/lib/stats-collector.js",
"./node_modules/mocha-7.0.1/lib/utils.js",
"./node_modules/mocha-7.0.1/node_modules/brace-expansion/index.js",
"./node_modules/mocha-7.0.1/node_modules/debug/node_modules/ms/index.js",
"./node_modules/mocha-7.0.1/node_modules/debug/src/common.js",
"./node_modules/mocha-7.0.1/node_modules/glob/common.js",
Expand Down Expand Up @@ -3172,8 +3188,6 @@
"./node_modules/recast/parsers/babel.js",
"./node_modules/recast/parsers/esprima.js",
"./node_modules/recast/parsers/typescript.js",
"./node_modules/registry-js/dist/lib/index.js",
"./node_modules/registry-js/dist/lib/registry.js",
"./node_modules/request-promise-core/configure/request2.js",
"./node_modules/request-promise-core/errors.js",
"./node_modules/request-promise-core/lib/errors.js",
Expand Down Expand Up @@ -3313,7 +3327,6 @@
"./node_modules/stream-shift/index.js",
"./node_modules/string-width/index.js",
"./node_modules/stringify-object/index.js",
"./node_modules/stringify-object/node_modules/is-obj/index.js",
"./node_modules/strip-ansi/index.js",
"./node_modules/strip-eof/index.js",
"./node_modules/strip-final-newline/index.js",
Expand Down Expand Up @@ -3354,6 +3367,7 @@
"./node_modules/through/index.js",
"./node_modules/timm/lib/timm.js",
"./node_modules/tinycolor2/cjs/tinycolor.js",
"./node_modules/tldts/dist/cjs/index.js",
"./node_modules/to-fast-properties/index.js",
"./node_modules/to-regex-range/index.js",
"./node_modules/to-regex-range/node_modules/is-number/index.js",
Expand Down Expand Up @@ -3622,7 +3636,6 @@
"./packages/data-context/node_modules/ast-types/lib/shared.js",
"./packages/data-context/node_modules/ast-types/lib/types.js",
"./packages/data-context/node_modules/ast-types/main.js",
"./packages/data-context/node_modules/brace-expansion/index.js",
"./packages/data-context/node_modules/cross-spawn/index.js",
"./packages/data-context/node_modules/cross-spawn/lib/enoent.js",
"./packages/data-context/node_modules/cross-spawn/lib/parse.js",
Expand Down Expand Up @@ -4174,7 +4187,6 @@
"./packages/server/node_modules/p-queue/dist/priority-queue.js",
"./packages/server/node_modules/path-key/index.js",
"./packages/server/node_modules/path-to-regexp/index.js",
"./packages/server/node_modules/pump/index.js",
"./packages/server/node_modules/readable-stream/lib/internal/streams/async_iterator.js",
"./packages/server/node_modules/readable-stream/lib/internal/streams/destroy.js",
"./packages/server/node_modules/readable-stream/lib/internal/streams/end-of-stream.js",
Expand Down Expand Up @@ -4237,5 +4249,5 @@
"./tooling/v8-snapshot/cache/darwin/snapshot-entry.js"
],
"deferredHashFile": "yarn.lock",
"deferredHash": "4c1da762a851baab24ef5d1ba9d878ad207147c7703383f4796c0c9c7910240f"
"deferredHash": "48d21e834f4efdc929d52b19393d9aed9638636c4a64d18b75b70daf3c25511c"
}
2 changes: 1 addition & 1 deletion tooling/v8-snapshot/cache/linux/snapshot-meta.json
Original file line number Diff line number Diff line change
Expand Up @@ -4252,5 +4252,5 @@
"./tooling/v8-snapshot/cache/linux/snapshot-entry.js"
],
"deferredHashFile": "yarn.lock",
"deferredHash": "3f91247096ae79ca27c13f643c4d4f870ab6aa22f210ce68ddbcce8ea7050380"
"deferredHash": "48d21e834f4efdc929d52b19393d9aed9638636c4a64d18b75b70daf3c25511c"
}
Loading
Loading