diff --git a/docs/guides/cloud/branch-review.mdx b/docs/guides/cloud/branch-review.mdx new file mode 100644 index 0000000000..cf5baa607b --- /dev/null +++ b/docs/guides/cloud/branch-review.mdx @@ -0,0 +1,184 @@ +--- +title: Branch Review +sidebar_position: 35 +sidebar_class_name: new_label +--- + +:::info + +## What you'll learn + +- What Branch Review is and why you should use it +- What test insights are available to review between branches +- How Branch Review works and what might impact available data + +::: + +:::info + +  Branch Review is currently in beta. +See [how to enable](/guides/cloud/branch-review#Public-Beta-Access) this feature below. + +::: + +## Overview + +Cypress Branch Review is designed to elevate your pull request review workflow. It allows you to quickly identify the impact a pull request might have on your test suite in a single view. Compare which tests are failing, flaky, pending, added, or modified between the source and base branches and prevent the merging of low-quality code. + +A common scenario throughout the software development lifecycle (SDLC) is an engineer's **feature** branch that will be merged into the repo's **develop** or **main** branch. + + + +Previously, pinpointing changes in your test suite's results required a manual side-by-side comparison between your newly-introduced branch's test runs and your base branch's test runs. This sub-optimal workflow often fails to answer the fundamental questions, _what changed_ and _why?_ You might be left wondering if the same tests are flaky between branches, when new failures were introduced, or if you added sufficient test coverage to your new branch. + +:::info + +Cypress Branch Review is currently only available for [GitHub integrated projects](/guides/cloud/integrations/source-control/github). We are working on adding support for GitLab and Bitbucket soon. + +::: + +## Public Beta Access + +:::tip + +To access the Branch Review public beta, an admin can simply enable it under "Cypress Labs" in your Organization Settings in Cypress Cloud. + + + +::: + +## Getting Started + +Branch Review works by leveraging the Cypress Cloud GitHub integration to query the GitHub API for branches with pull requests. Your project will first need to be connected to a GitHub repository. If you haven't already, follow the steps in [Install the Cypress GitHub app](/guides/cloud/integrations/source-control/github#Install-the-Cypress-GitHub-app) to connect your project to GitHub. + +### Accessing Branch Review + +To access Branch Review, navigate to the **Branches** tab in the left sidebar and select the branch you want to review. In order to see a comparison, the branch you select must have a pull request. + + + +If you do not see PR # tags associated with a branch, you may need to [pass PR numbers](/guides/continuous-integration/introduction#CI-Build-Information) to runs as environment variables. This helps ensure Cypress Cloud can map PR data correctly. + +#### Latest Runs + +If the selected branch has a PR associated and a PR # passed via CI, there is an additional callout on the top of the overview tab of the **Latest runs** page which directs to the Branch Review. + + + +## Branch Details + +### View Options + +"Review" is the default view of the **Branches** tab for projects with linked GitHub repositories via our GitHub integration. + +"Runs" provides a list filtered by the selected branch for projects without linked repositories or those with an unknown branch. An unkown branch is one in which Cypress does not have access to the local `.git` directory and cannot determine a branch for a run. + + + +### Review Header + +The header includes the Git commit message, PR selector, PR status, and base and feature branch labels with the Cypress Cloud test run ID number (#). + +If there are multiple pull requests open for the same branch, you can select the pull request you want to review from the dropdown to the right of the commit message. + +Clicking the branch label will link you directly to the appropriate [run overview](/guides/cloud/debugging/recorded-runs#Overview-tab) and hovering on these elements exposes additional run meta data. + + + +### Review Test Status + +When on the review screen, you will see the Failures, Flaky, Pending, Added, and Modified tabs. Each tab will show you the specs that fall into that category. You can click on a test to view the test details. + + + +:::note + +1. Review your PR information +2. Get a glance of changes in your branch across key statuses +3. See what has changed in a specific status, for example: + - total number introduced in your branch + - total number decreased or resolved in your branch + - total count, for example, 3 new and 1 existing +4. The test state (failed, flaky, pending) is also indicated at the spec level + - _new_ = the state was not previously captured, but now is captured
(newly pending tests can imply an `it.skip()` was not removed) + - _existing_ = the state was previously captured and now is still captured + - _resolved_ = the state was previously captured but now is no longer captured + +::: + +### Review Test Comparison + +Navigating into the test detail view reveals a side-by-side comparison of the test results on both branches, as well as, artifacts so you can compare before vs. after your PR. This can help narrow the focus on changes introduced or resolved on either branch and elevate the source of flake within tests throughout different attempts. Review the test definition panel for diff snapshots to help quickly determine changes in test code. + + + +## GitHub Pull Request Comments + +The Cypress Cloud GitHub Integration offers detailed test information via [PR comments](/guides/cloud/integrations/source-control/github#Pull-request-comments). Follow the "Review all test suite changes" link to load the Branch Review between the feature and base branches. + + + +## Troubleshooting + +### Note on Available Data + +Branch Review is a powerful tool to compare two branches with recorded runs to Cypress Cloud. There are factors that can impact what is available to review between a feature and base branch. For example, whether or not a branch at a particular commit has a run recorded to Cypress Cloud will affect what is displayed. + +The following captures this scenario across both branches: + +| Base Branch | Feature Branch | What Branch Review Displays | +| :-----------------------------------: | :-----------------------------------: | ---------------------------------------------------- | +| has run | has run | Comparison data using both found runs | +| has run | no run | Comparison data using last run on feature branch | +| no run | has run | Non-comparison data with found feature run | +| no run | no run | Non-comparison data using last run on feature branch | + +## Best Practices + +:::tip + +### Grouping Test Runs + +Cypress Cloud allows for [grouping recorded tests](/guides/cloud/smart-orchestration/parallelization#Grouping-test-runs) together under a single run. This means multiple `cypress run` calls can be labeled and associated to a single run in Cypress Cloud. + +Recording multiple test runs per commit without grouping will cause issues, as Branch Review relies on the _latest_ run. Therefore, recording one run per commit and utilizing grouping is essential to improving effectiveness and ensure an accurate comparison between branches. + +::: diff --git a/docs/guides/cloud/integrations/source-control/github.mdx b/docs/guides/cloud/integrations/source-control/github.mdx index 433ca9e071..b676b07c99 100644 --- a/docs/guides/cloud/integrations/source-control/github.mdx +++ b/docs/guides/cloud/integrations/source-control/github.mdx @@ -11,7 +11,7 @@ with your GitHub workflow via commit [status checks](#Status-checks) and to use GitHub integration. @@ -122,7 +122,7 @@ This installation method does not apply to GitHub Enterprise. 5. Click the **Enable GitHub** button. @@ -137,7 +137,7 @@ to GitHub.com to complete the installation: Cypress Cloud organization. @@ -153,7 +153,7 @@ you choose **All repositories**. ::: @@ -180,7 +180,7 @@ them into Cypress Cloud, and complete the activation process. App** button. @@ -192,7 +192,7 @@ them into Cypress Cloud, and complete the activation process. https://cloud.cypress.io/apps/github/callback @@ -203,7 +203,7 @@ them into Cypress Cloud, and complete the activation process. **Webhook secret (optional)**. @@ -214,12 +214,12 @@ them into Cypress Cloud, and complete the activation process. - **Commit statuses**: read & write @@ -244,19 +244,19 @@ them into Cypress Cloud, and complete the activation process. into the Cypress **GitHub Enterprise Private key** field. @@ -266,7 +266,7 @@ them into Cypress Cloud, and complete the activation process. app name\]** button. @@ -274,7 +274,7 @@ them into Cypress Cloud, and complete the activation process. 8. On your newly-authorized GitHub App, click the **Install** button. @@ -291,7 +291,7 @@ installation if you choose All repositories. ::: @@ -332,14 +332,14 @@ organization's Integrations page: ::: 3. Select a GitHub repository to associate with the project. @@ -347,7 +347,7 @@ Once a GitHub repository is associated with a Cypress project, the GitHub integration will be immediately enabled: @@ -356,7 +356,7 @@ You can also see all GitHub Integration-enabled Cypress projects within your organizations **Integrations** page: @@ -374,13 +374,13 @@ The Cypress GitHub App reports commit status checks in two distinct styles: [run group](/guides/cloud/smart-orchestration/parallelization#Grouping-test-runs).** - **Or one check per spec file.** @@ -390,7 +390,7 @@ Cypress Cloud to help you dig deeper into the problem via error messages, stack traces, screenshots, and video recordings: @@ -402,7 +402,7 @@ GitHub status checks are optional and can be disabled within a project's GitHub integration settings: @@ -436,7 +436,7 @@ requests via comments that include: An example of a Cypress pull-request comment can be seen below: @@ -446,7 +446,7 @@ PR comments are optional and can be disabled if not needed within a project's GitHub Integration settings: diff --git a/docs/guides/cloud/introduction.mdx b/docs/guides/cloud/introduction.mdx index 13551592e7..5471415613 100644 --- a/docs/guides/cloud/introduction.mdx +++ b/docs/guides/cloud/introduction.mdx @@ -100,7 +100,7 @@ and [GitLab](https://docs.gitlab.com/ee/subscriptions/self_managed/) instances too. diff --git a/docusaurus.config.js b/docusaurus.config.js index 27155967c9..be0c3ff31e 100644 --- a/docusaurus.config.js +++ b/docusaurus.config.js @@ -269,7 +269,7 @@ const config = { darkTheme: darkCodeTheme, }, zoom: { - selector: ':not(.mediaImage, .navbar__logo img, .logo)', // don't zoom these images + selector: ':not(.mediaImage, .navbar__logo img, .logo, .br-ui)', // don't zoom these images background: { light: 'rgb(50, 50, 50)', dark: 'rgb(50, 50, 50)', diff --git a/src/css/sidebar.scss b/src/css/sidebar.scss index efe6c0c0bb..9a5847e37e 100644 --- a/src/css/sidebar.scss +++ b/src/css/sidebar.scss @@ -34,4 +34,17 @@ // helps delineate different refs in sidebar, particularly in API section .theme-doc-sidebar-item-link.theme-doc-sidebar-item-link-level-2.menu__list-item.sidebar-guide-ref { font-weight: 600; +} + +// "new" feature label in sidebar +.theme-doc-sidebar-item-link.theme-doc-sidebar-item-link-level-2.menu__list-item.new_label { + a::after { + background-color: var(--cypress-color-indigo-400); + color: #fff; + content: "New"; + font-size: .7rem; + border-radius: 5px; + margin-left: 5px; + padding: 3px 5px 2px; + } } \ No newline at end of file diff --git a/static/img/dashboard/github-integration/pr-comment-fail.jpg b/static/img/dashboard/github-integration/pr-comment-fail.jpg deleted file mode 100644 index c58f3b281b..0000000000 Binary files a/static/img/dashboard/github-integration/pr-comment-fail.jpg and /dev/null differ diff --git a/static/img/dashboard/github-integration/pull-request-cypress-integration-comments-github-checks.jpg b/static/img/dashboard/github-integration/pull-request-cypress-integration-comments-github-checks.jpg deleted file mode 100644 index 996464044b..0000000000 Binary files a/static/img/dashboard/github-integration/pull-request-cypress-integration-comments-github-checks.jpg and /dev/null differ diff --git a/static/img/dashboard/v10/runs-list-in-cypress-app.png b/static/img/dashboard/v10/runs-list-in-cypress-app.png deleted file mode 100644 index 836de50f99..0000000000 Binary files a/static/img/dashboard/v10/runs-list-in-cypress-app.png and /dev/null differ diff --git a/static/img/guides/cloud/branch-review/branch-review-available.jpg b/static/img/guides/cloud/branch-review/branch-review-available.jpg new file mode 100644 index 0000000000..cce94d317e Binary files /dev/null and b/static/img/guides/cloud/branch-review/branch-review-available.jpg differ diff --git a/static/img/guides/cloud/branch-review/branch-review-details.jpg b/static/img/guides/cloud/branch-review/branch-review-details.jpg new file mode 100644 index 0000000000..54e27b44e7 Binary files /dev/null and b/static/img/guides/cloud/branch-review/branch-review-details.jpg differ diff --git a/static/img/guides/cloud/branch-review/branch-review-header.jpg b/static/img/guides/cloud/branch-review/branch-review-header.jpg new file mode 100644 index 0000000000..ff73635a46 Binary files /dev/null and b/static/img/guides/cloud/branch-review/branch-review-header.jpg differ diff --git a/static/img/guides/cloud/branch-review/branch-review-views.jpg b/static/img/guides/cloud/branch-review/branch-review-views.jpg new file mode 100644 index 0000000000..16d56ebdf5 Binary files /dev/null and b/static/img/guides/cloud/branch-review/branch-review-views.jpg differ diff --git a/static/img/guides/cloud/branch-review/branch-review.jpg b/static/img/guides/cloud/branch-review/branch-review.jpg new file mode 100644 index 0000000000..6ef23bf26c Binary files /dev/null and b/static/img/guides/cloud/branch-review/branch-review.jpg differ diff --git a/static/img/guides/cloud/branch-review/get-to-branch-review.jpg b/static/img/guides/cloud/branch-review/get-to-branch-review.jpg new file mode 100644 index 0000000000..ba6b8417cd Binary files /dev/null and b/static/img/guides/cloud/branch-review/get-to-branch-review.jpg differ diff --git a/static/img/guides/cloud/branch-review/labs-beta.png b/static/img/guides/cloud/branch-review/labs-beta.png new file mode 100644 index 0000000000..79b2619e4e Binary files /dev/null and b/static/img/guides/cloud/branch-review/labs-beta.png differ diff --git a/static/img/guides/cloud/branch-review/test-comparison.jpg b/static/img/guides/cloud/branch-review/test-comparison.jpg new file mode 100644 index 0000000000..44fdaa61b9 Binary files /dev/null and b/static/img/guides/cloud/branch-review/test-comparison.jpg differ diff --git a/static/img/guides/cloud/branch-review/test-status-down.png b/static/img/guides/cloud/branch-review/test-status-down.png new file mode 100644 index 0000000000..c14dc7a9cd Binary files /dev/null and b/static/img/guides/cloud/branch-review/test-status-down.png differ diff --git a/static/img/guides/cloud/branch-review/test-status-total.png b/static/img/guides/cloud/branch-review/test-status-total.png new file mode 100644 index 0000000000..e3aca0aac7 Binary files /dev/null and b/static/img/guides/cloud/branch-review/test-status-total.png differ diff --git a/static/img/guides/cloud/branch-review/test-status-up.png b/static/img/guides/cloud/branch-review/test-status-up.png new file mode 100644 index 0000000000..581be5f60e Binary files /dev/null and b/static/img/guides/cloud/branch-review/test-status-up.png differ diff --git a/static/img/dashboard/github-integration/cloud-details-specs-tab.jpg b/static/img/guides/cloud/integrations/github/cloud-details-specs-tab.jpg similarity index 100% rename from static/img/dashboard/github-integration/cloud-details-specs-tab.jpg rename to static/img/guides/cloud/integrations/github/cloud-details-specs-tab.jpg diff --git a/static/img/dashboard/github-integration/enable-github.jpg b/static/img/guides/cloud/integrations/github/enable-github.jpg similarity index 100% rename from static/img/dashboard/github-integration/enable-github.jpg rename to static/img/guides/cloud/integrations/github/enable-github.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-01.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-01.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-01.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-01.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-02.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-02.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-02.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-02.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-03.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-03.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-03.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-03.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-04-commit.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-04-commit.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-04-commit.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-04-commit.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-04-pr.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-04-pr.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-04-pr.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-04-pr.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-05.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-05.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-05.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-05.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-06.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-06.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-06.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-06.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-07.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-07.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-07.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-07.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-08.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-08.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-08.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-08.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-09.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-09.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-09.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-09.jpg diff --git a/static/img/dashboard/github-integration/ghe/ghe-10.jpg b/static/img/guides/cloud/integrations/github/ghe/ghe-10.jpg similarity index 100% rename from static/img/dashboard/github-integration/ghe/ghe-10.jpg rename to static/img/guides/cloud/integrations/github/ghe/ghe-10.jpg diff --git a/static/img/dashboard/github-integration/org-settings-with-no-enabled-projects.jpg b/static/img/guides/cloud/integrations/github/org-settings-with-no-enabled-projects.jpg similarity index 100% rename from static/img/dashboard/github-integration/org-settings-with-no-enabled-projects.jpg rename to static/img/guides/cloud/integrations/github/org-settings-with-no-enabled-projects.jpg diff --git a/static/img/dashboard/github-integration/org-settings-with-projects.jpg b/static/img/guides/cloud/integrations/github/org-settings-with-projects.jpg similarity index 100% rename from static/img/dashboard/github-integration/org-settings-with-projects.jpg rename to static/img/guides/cloud/integrations/github/org-settings-with-projects.jpg diff --git a/static/img/guides/cloud/integrations/github/pr-comment-fail.jpg b/static/img/guides/cloud/integrations/github/pr-comment-fail.jpg new file mode 100644 index 0000000000..0b13314013 Binary files /dev/null and b/static/img/guides/cloud/integrations/github/pr-comment-fail.jpg differ diff --git a/static/img/dashboard/github-integration/pr-comments-settings.jpg b/static/img/guides/cloud/integrations/github/pr-comments-settings.jpg similarity index 100% rename from static/img/dashboard/github-integration/pr-comments-settings.jpg rename to static/img/guides/cloud/integrations/github/pr-comments-settings.jpg diff --git a/static/img/dashboard/github-integration/project-settings-repo-selection.jpg b/static/img/guides/cloud/integrations/github/project-settings-repo-selection.jpg similarity index 100% rename from static/img/dashboard/github-integration/project-settings-repo-selection.jpg rename to static/img/guides/cloud/integrations/github/project-settings-repo-selection.jpg diff --git a/static/img/dashboard/github-integration/project-settings-selected-repo.jpg b/static/img/guides/cloud/integrations/github/project-settings-selected-repo.jpg similarity index 100% rename from static/img/dashboard/github-integration/project-settings-selected-repo.jpg rename to static/img/guides/cloud/integrations/github/project-settings-selected-repo.jpg diff --git a/static/img/guides/cloud/integrations/github/pull-request-cypress-integration-comments-github-checks.jpg b/static/img/guides/cloud/integrations/github/pull-request-cypress-integration-comments-github-checks.jpg new file mode 100644 index 0000000000..786aa12be1 Binary files /dev/null and b/static/img/guides/cloud/integrations/github/pull-request-cypress-integration-comments-github-checks.jpg differ diff --git a/static/img/dashboard/github-integration/select-gh-org.jpg b/static/img/guides/cloud/integrations/github/select-gh-org.jpg similarity index 100% rename from static/img/dashboard/github-integration/select-gh-org.jpg rename to static/img/guides/cloud/integrations/github/select-gh-org.jpg diff --git a/static/img/dashboard/github-integration/select-gh-repos.jpg b/static/img/guides/cloud/integrations/github/select-gh-repos.jpg similarity index 100% rename from static/img/dashboard/github-integration/select-gh-repos.jpg rename to static/img/guides/cloud/integrations/github/select-gh-repos.jpg diff --git a/static/img/dashboard/github-integration/status-check-settings.jpg b/static/img/guides/cloud/integrations/github/status-check-settings.jpg similarity index 100% rename from static/img/dashboard/github-integration/status-check-settings.jpg rename to static/img/guides/cloud/integrations/github/status-check-settings.jpg diff --git a/static/img/dashboard/github-integration/status-checks-per-group.jpg b/static/img/guides/cloud/integrations/github/status-checks-per-group.jpg similarity index 100% rename from static/img/dashboard/github-integration/status-checks-per-group.jpg rename to static/img/guides/cloud/integrations/github/status-checks-per-group.jpg diff --git a/static/img/dashboard/github-integration/status-checks-per-spec.jpg b/static/img/guides/cloud/integrations/github/status-checks-per-spec.jpg similarity index 100% rename from static/img/dashboard/github-integration/status-checks-per-spec.jpg rename to static/img/guides/cloud/integrations/github/status-checks-per-spec.jpg