Skip to content

Commit

Permalink
v13.4.0 Release (#5551)
Browse files Browse the repository at this point in the history 
Co-authored-by: Cacie Prins <[email protected]>
Co-authored-by: Ryan Pei <[email protected]>
  • Loading branch information
@emilyrohrbough @cacieprins @ryanpei
3 people authored Oct 30, 2023
1 parent 0c6f4c6 commit 43f4fc3
Show file tree
Hide file tree
Showing 5 changed files with 195 additions and 46 deletions.
2 changes: 2 additions & 0 deletions docs/guides/cloud/flaky-test-management.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,8 @@ tests by Cypress Cloud. Therefore, **enabling
[test retries](/guides/guides/test-retries#Configure-Test-Retries) is required**
to take advantage of any flaky test management feature.

If you want more control over the pass/fail result of any test deemed flaky by test retries, you may want to use [experimental test retries](/guides/references/experiments#Experimental-Flake-Detection-Features), which is available as of Cypress `13.4.0`.

:::

### Flagging Flaky Tests
Expand Down
12 changes: 10 additions & 2 deletions docs/guides/guides/test-retries.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,21 @@ conditions that could result in unreliable tests include:
- Resource dependencies availability
- Network issues

With test retries, Cypress is able to retry failed tests to help reduce test
flakiness and continuous integration (CI) build failures. By doing so, this will
With test retries, Cypress is able to retry failed tests to help [detect test
flakiness](/guides/cloud/flaky-test-management#Flake-Detection) and continuous integration (CI) build failures. By doing so, this will
save your team valuable time and resources so you can focus on what matters most
to you.

## How It Works

:::info

You may want to use the [experimental retries](/guides/references/experiments#Experimental-Retries) feature which offers more options for [detecting flaky tests](/guides/cloud/flaky-test-management#Flake-Detection). This experimental feature is available as of Cypress `13.4.0`.

Test retries were originally intended to give failing tests more chances to "pass" (for instance, if CI environments for testing are unreliable), while still being determined as flaky if they do eventually pass. However this may not be the desired result in all cases. Experimental retries give you control over the conditions of the pass or fail result.

:::

By default, tests will not retry when they fail. You will need to
[enable test retries in your configuration](#Configure-Test-Retries) to use this
feature.
Expand Down
12 changes: 12 additions & 0 deletions docs/guides/references/changelog.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,18 @@
title: Changelog
---

## 13.4.0

_Released 10/30/2023_

**Features:**

- Introduced experimental configuration options for advanced retry logic: adds `experimentalStrategy` and `experimentalOptions` keys to the `retry` configuration key. See [Experimental Flake Detection Features](/guides/references/experiments/#Experimental-Flake-Detection-Features) for more information. Addressed in [#27930](https://github.com/cypress-io/cypress/pull/27930).

**Bugfixes:**

- Fixed a regression in [`13.3.2`](#13.3.2) where Cypress would crash with 'Inspected target navigated or closed' or 'Session with given id not found'. Fixes [#28141](https://github.com/cypress-io/cypress/issues/28141) and [#28148](https://github.com/cypress-io/cypress/issues/28148).

## 13.3.3

_Released 10/24/2023_
Expand Down
39 changes: 20 additions & 19 deletions docs/guides/references/configuration.mdx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ default values.
| `redirectionLimit` | `20` | The number of times that the application under test can redirect before erroring. |
| `reporter` | `spec` | The [reporter](/guides/tooling/reporters) used during `cypress run`. |
| `reporterOptions` | `null` | The [reporter options](/guides/tooling/reporters#Reporter-Options) used. Supported options depend on the reporter. |
| `retries` | `{ "runMode": 0, "openMode": 0 }` | The number of times to retry a failing test. Can be configured to apply to `cypress run` or `cypress open` separately. See [Test Retries](/guides/guides/test-retries) for more information. |
| `retries` | `{ "runMode": 0, "openMode": 0 }` | The number of times to retry a failing test. Can be configured to apply to `cypress run` or `cypress open` separately. As of Cypress `13.4.0`, the experimental Flake Detection strategy can also be configured. See [Test Retries](/guides/guides/test-retries) for more information. |
| `watchForFileChanges` | `true` | Whether Cypress will watch and restart tests on test file changes. |

### Timeouts
Expand Down Expand Up @@ -716,24 +716,25 @@ DEBUG=cypress:cli,cypress:server:specs

## History

| Version | Changes |
| --------------------------------------------- | ------------------------------------------------------------------------------------- |
| [13.0.0](/guides/references/changelog#13-0-0) | Removed `nodeVersion` option. |
| [13.0.0](/guides/references/changelog#13-0-0) | Removed `videoUploadOnPasses` option. |
| [11.0.0](/guides/references/changelog#11-0-0) | Removed `e2e.experimentalSessionAndOrigin` option. |
| [10.4.0](/guides/references/changelog#10-4-0) | Added `e2e.testIsolation` option. |
| [10.0.0](/guides/references/changelog#10-0-0) | Reworked page to support new `cypress.config.js` and deprecated `cypress.json` files. |
| [8.7.0](/guides/references/changelog#8-7-0) | Added `slowTestThreshold` option. |
| [8.0.0](/guides/references/changelog#8-0-0) | Added `clientCertificates` option and removed `firefoxGcInterval` configuration. |
| [7.0.0](/guides/references/changelog#7-0-0) | Added `e2e` and `component` options. |
| [7.0.0](/guides/references/changelog#7-0-0) | Added `redirectionLimit` option. |
| [6.1.0](/guides/references/changelog#6-1-0) | Added `scrollBehavior` option. |
| [5.2.0](/guides/references/changelog#5-2-0) | Added `includeShadowDom` option. |
| [5.0.0](/guides/references/changelog#5-0-0) | Added `retries` configuration. |
| [5.0.0](/guides/references/changelog#5-0-0) | Renamed `blacklistHosts` configuration to `blockHosts`. |
| [4.1.0](/guides/references/changelog#4-12-0) | Added `screenshotOnRunFailure` configuration. |
| [4.0.0](/guides/references/changelog#4-0-0) | Added `firefoxGcInterval` configuration. |
| [3.5.0](/guides/references/changelog#3-5-0) | Added `nodeVersion` configuration. |
| Version | Changes |
| --------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- |
| [13.4.0](/guides/references/changelog#13-4-0) | Added support for configuring the Experimental Flake Detection strategy via `retries.experimentalStrategy` and `retries.experimentalOptions`. |
| [13.0.0](/guides/references/changelog#13-0-0) | Removed `nodeVersion` option. |
| [13.0.0](/guides/references/changelog#13-0-0) | Removed `videoUploadOnPasses` option. |
| [11.0.0](/guides/references/changelog#11-0-0) | Removed `e2e.experimentalSessionAndOrigin` option. |
| [10.4.0](/guides/references/changelog#10-4-0) | Added `e2e.testIsolation` option. |
| [10.0.0](/guides/references/changelog#10-0-0) | Reworked page to support new `cypress.config.js` and deprecated `cypress.json` files. |
| [8.7.0](/guides/references/changelog#8-7-0) | Added `slowTestThreshold` option. |
| [8.0.0](/guides/references/changelog#8-0-0) | Added `clientCertificates` option and removed `firefoxGcInterval` configuration. |
| [7.0.0](/guides/references/changelog#7-0-0) | Added `e2e` and `component` options. |
| [7.0.0](/guides/references/changelog#7-0-0) | Added `redirectionLimit` option. |
| [6.1.0](/guides/references/changelog#6-1-0) | Added `scrollBehavior` option. |
| [5.2.0](/guides/references/changelog#5-2-0) | Added `includeShadowDom` option. |
| [5.0.0](/guides/references/changelog#5-0-0) | Added `retries` configuration. |
| [5.0.0](/guides/references/changelog#5-0-0) | Renamed `blacklistHosts` configuration to `blockHosts`. |
| [4.1.0](/guides/references/changelog#4-12-0) | Added `screenshotOnRunFailure` configuration. |
| [4.0.0](/guides/references/changelog#4-0-0) | Added `firefoxGcInterval` configuration. |
| [3.5.0](/guides/references/changelog#3-5-0) | Added `nodeVersion` configuration. |

## See also

Expand Down
Loading

0 comments on commit 43f4fc3

Please sign in to comment.