You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The way this action is currently being tested is primarily through a GitHub Actions workflow that deploys several Worker fixtures and passes as long as the action doesn't fail to complete.
This approach has a few severe drawbacks. Because deploying these Workers to our test account requires our API token, PRs submitted by external contributors don't have access to our secrets and these always fail to run. These failures also mask those that might potentially occur during other checks like code formatting. This results in us merging sometimes-broken code due to having no reliable confidence signal before merging.
Second, the current E2E structure makes it difficult to assert against expected behaviors of the action in an automated way. It's also cumbersome to construct tests for each scenario (and tear them down after the test completes, etc.) These E2E tests end up mostly testing the underlying behavior of wrangler rather than the behavior of the action itself.
We can improve our test coverage by moving towards integration testing for much of the behavior currently being tested (to varying degrees) in an E2E manner. To do this, we'll invoke the action's entrypoint in a vitest and assert that the expected subprocesses are spawned appropriately.
We can limit E2E tests to a few basic smoke tests that only run if: ${{ github.repository_owner == 'cloudflare' }} while allowing the rest of the test suite to run on external PRs.
The text was updated successfully, but these errors were encountered:
The way this action is currently being tested is primarily through a GitHub Actions workflow that deploys several Worker fixtures and passes as long as the action doesn't fail to complete.
This approach has a few severe drawbacks. Because deploying these Workers to our test account requires our API token, PRs submitted by external contributors don't have access to our secrets and these always fail to run. These failures also mask those that might potentially occur during other checks like code formatting. This results in us merging sometimes-broken code due to having no reliable confidence signal before merging.
Second, the current E2E structure makes it difficult to assert against expected behaviors of the action in an automated way. It's also cumbersome to construct tests for each scenario (and tear them down after the test completes, etc.) These E2E tests end up mostly testing the underlying behavior of wrangler rather than the behavior of the action itself.
We can improve our test coverage by moving towards integration testing for much of the behavior currently being tested (to varying degrees) in an E2E manner. To do this, we'll invoke the action's entrypoint in a vitest and assert that the expected subprocesses are spawned appropriately.
We can limit E2E tests to a few basic smoke tests that only run
if: ${{ github.repository_owner == 'cloudflare' }}
while allowing the rest of the test suite to run on external PRs.The text was updated successfully, but these errors were encountered: