diff --git a/docs/accessibility/changelog.mdx b/docs/accessibility/changelog.mdx index a61eafbb3c..3c56015ac6 100644 --- a/docs/accessibility/changelog.mdx +++ b/docs/accessibility/changelog.mdx @@ -7,10 +7,6 @@ sidebar_position: 200 # Changelog -## Week of 11/11/2024 - -- A "Share Issue" button has been added to the element details section for each reported rule violation. This allows for easily copying the key details and related links for a particular issue into your clipboard to add to a message or to a ticket in your issue tracking system. - ## Week of 10/28/2024 - Our [Axe Core® versioning policy](/accessibility/core-concepts/how-it-works#Axe-Core-updates) has been published, providing a 30-day buffer between any Axe Core® releases and the adoption of those releases in Cypress Cloud. diff --git a/docs/accessibility/configuration/axe-core-configuration.mdx b/docs/accessibility/configuration/axe-core-configuration.mdx index 45cddca7a9..b5f10646d3 100644 --- a/docs/accessibility/configuration/axe-core-configuration.mdx +++ b/docs/accessibility/configuration/axe-core-configuration.mdx @@ -8,7 +8,7 @@ sidebar_label: 'Axe Core® configuration' -Configuration for Axe-Core® rules is available through your Account Executive. We are happy to have a call with you to dial in your report config to make sure you are getting the most useful reports possible, and we find this onboarding very effective. +Configuration for Axe-Core® rules is available through your Account Executive. We we are happy to have a call with you to dial in your report config to make sure you are getting the most useful reports possible, and we find this onboarding very effective. In most cases, configuration of these rules as they run in Cypress Cloud isn't needed, because your implementation of any policies about what should "fail a build" is handled using the [Accessibility Results API](/accessibility/results-api), where you have full control over how to parse the results and what rules need to be reacted to. Keeping the results in Cypress Cloud broad helps you to still be able to see and understand all of the accessibility information, even if only a subset of the results would be considered blocking. @@ -24,7 +24,7 @@ In most cases, configuration of these rules as they run in Cypress Cloud isn't n ## Component Testing -Cypress Accessibility works great with component testing. In addition the rules that are off by default, "page-level" rules do not run for component tests, since a component is usually a fragment mounted in an empty page that would not be expected to have things like headings or a `main` element. +Cypress Accessibility works great with component testing. In addition the the rules that are off by default, "page-level" rules do not run for component tests, since a component is usually a fragment mounted in an empty page that would not be expected to have things like headings or a `main` element. ## Updates to the Axe Core® library diff --git a/docs/accessibility/core-concepts/how-it-works.mdx b/docs/accessibility/core-concepts/how-it-works.mdx index 637640ab96..d24b2d2b07 100644 --- a/docs/accessibility/core-concepts/how-it-works.mdx +++ b/docs/accessibility/core-concepts/how-it-works.mdx @@ -11,7 +11,7 @@ The stages of the Cypress Accessibility process are as follows: 1. As your run is recorded and specs are uploaded to Cypress Cloud, accessibility checks begin, based on the incoming [Test Replay](/cloud/features/test-replay) data, on the Cypress Cloud servers. 1. When your run completes, the final specs are processed for accessibility checks. All earlier specs have already entered processing and are likely completed by the end of the run. -1. Cypress Cloud organizes all the Views (pages and components) that were rendered during the run, merging and deduplicating across tests so that page-level results and live DOM snapshots can be grouped together. +1. Cypress Cloud organizes all the the Views (pages and components) that were rendered during the run, merging and deduplicating across tests so that page-level results and live DOM snapshots can be grouped together. 1. A combined "rule-level" report is created. 1. The completed results are displayed in Cypress Cloud and are flagged in integrations such Slack as GitHub/Gitlab comments, and available CI via the Results API so that you can optionally fail your pipeline if your standards are not met. diff --git a/docs/accessibility/get-started/introduction.mdx b/docs/accessibility/get-started/introduction.mdx index e4e1f1ae81..ce4dcd55c8 100644 --- a/docs/accessibility/get-started/introduction.mdx +++ b/docs/accessibility/get-started/introduction.mdx @@ -15,7 +15,7 @@ Looking for a general guide for Accessibility testing in Cypress App? [Go here]( - -## 2. Record a run - -Cypress Accessibility generates reports using [Test Replay](/cloud/features/test-replay) data and requires Cypress v13+. No additional configuration or instrumentation is required to get started. - -[Record a run](/cloud/get-started/setup) to the Cypress Cloud with Test Replay to start using Cypress Accessibility. - -## 3. View your Accessibility results - -After recording a test run, you can view your Accessibility results in the Cypress Cloud via the **Accessibility** tab. - - - -## 4. Customize your results - -Cypress Accessibility provides flexible configuration options to customize your reports. You can configure: - -- [Element filters](/accessibility/configuration/elementfilters): specify selectors for elements that should be excluded from Accessibility scans -- [View filters](/accessibility/configuration/viewfilters): specify URL patterns for URLs that should be excluded from Accessibility scans. Excluding a URL also excludes all links to that URL. -- [Views](/accessibility/configuration/views): specify URL patterns that represent views - -To add or modify the configuration for your project, navigate to the **App Quality** tab in your project settings. - - diff --git a/docs/accessibility/guides/_category_.json b/docs/accessibility/guides/_category_.json deleted file mode 100644 index 371eacc4b7..0000000000 --- a/docs/accessibility/guides/_category_.json +++ /dev/null @@ -1,4 +0,0 @@ -{ - "label": "Guides", - "position": 70 -} diff --git a/docs/accessibility/guides/accessibility-automation.mdx b/docs/accessibility/guides/accessibility-automation.mdx deleted file mode 100644 index c1b6565503..0000000000 --- a/docs/accessibility/guides/accessibility-automation.mdx +++ /dev/null @@ -1,47 +0,0 @@ ---- -title: Accessibility automation principles -sidebar_position: 80 ---- - -This guide describes some general ideas that are relevant to Cypress Accessibility, but are also somewhat shared among accessibility automation tools of any kind. - -## What it means to fix all Axe Core® violations {#what-it-means-to-fix-all-axe-core-violations} - -Cypress Accessibility runs Axe Core® checks on every significant state and variation of the applications you test with Cypress. This kind of automation can catch up to [57% of issues that would appear in a manual audit](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/), but it does not replace the human judgment that is applied during an audit. Auditors test a wide variety of scenarios and technologies in a way that would be impossible to fully automate. - -It helps to think of accessibility at three levels. The first is “basic usability”. Paying attention to issues reported by Cypress and Axe Core® should help you to this point. This is the threshold at which a disabled user is likely to be able to make substantial progress through the application, but may still hit major barriers or usability issues. If your contact forms and issue reporting systems are accessible, you will be able to learn about these problems from actual users. - -The second level of accessibility is full conformance with all Web Content Accessibility Guidelines (WCAG) success criteria. As mentioned above, more than half of issues reported in manual audits can be reported and caught early with Axe Core ®automation. Conformance with the WCAG is often the standard set by legislation and regulation related to web accessibility. If your goal is regulatory compliance, this is the bar. No generic automation tool can “prove” compliance with this standard, it **requires** human assessment. - -The third level, and the true goal of many accessibility programs, is a good user experience. This needs to be validated by actual disabled users who are daily users of assistive technology. It’s entirely possible to have a website that passes the first two levels, and still offers a poor user experience. - -Accessibility is about communicating the nature and structure of the application, and the available actions and information, in a way that allows disabled users to independently understand the interface and complete all tasks. The foundational [POUR principles](https://www.24a11y.com/2019/pour-the-foundation-first/) of the WCAG \- that software must be Perceivable, Operable, Understandable and Robust \- are bigger than any specific set of test plans or static checks. While Axe Core® tell us about many genuine accessibility barriers, it is important to leverage that information into making good decisions about accessibility and providing a truly equal experience. - -## False positives - -False positives are always a risk in any automated scans. Both Cypress Accessibility and Axe Core® are designed in a way that minimizes them, but it’s always possible that there will be some violations in the report that you recognize as invalid and would rather exclude, or are genuinely incorrect. - -While false positives are rare, if in doubt, use the “provide feedback” button to alert us to something you see as a false positive and we can work on diagnosing and solving the issue from our side. - -### Genuine false positives - -There are two main possible sources of genuine false positive results: - -1. The check was run on an **invalid** **state** \- for example a partially hydrated DOM \- that no user would ever see. These are scenarios that Cypress can account for in the product if they are reported. -2. The state tested was correct but Axe Core® reported a violation that it shouldn’t have, as a result of a bug or missing check. Since Axe Core® is an open-source product, if and when we encounter bugs, we can open issues in the project and even make pull requests if needed. - -### Loading states - -There’s another scenario that can present as a potential false positive: legitimate user-facing loading states. These are situations where some data is pending in your application. - -It might be tempting to say “this state shouldn’t be captured, it’s not fully loaded” \- but in reality, these types of “pending” states are valid, and often contain visual information like loading spinners, UI skeletons, and progress bars which should be accessible themselves, so the user knows what data is loading. It’s also possible that the data doesn’t come in at all, or is very slow, and a user needs to figure out what is happening and what they can do, so the application needs to remain in a fully accessible, usable condition even if data is pending. - -In general it’s a good idea in front-end development to “make impossible states impossible”, and in doing so you can follow the accessibility principle of Robustness (the last of the 4 POUR principles mentioned above), while also making it much less likely that an invalid state is possible to reach in Cypress in the first place. This can also have a beneficial effect on the overall stability of your tests. - -### Assistive technology support - -Certain accessibility techniques are not fully supported by all assistive technologies like screen readers or voice control software. Axe Core® accounts for this. The software and operating systems tested against are listed in their [Accessibility Supported document](https://github.com/dequelabs/axe-core/blob/develop/doc/accessibility-supported.md). For example: it can seem like a false positive from Axe Core® if a failing technique used in your code appears to work correctly in a certain screen reader, browser and operating system combination that you test with. But if it does not have sufficient accessibility support in commonly used technology configurations, Axe Core® will intentionally fail that technique, allowing you to switch to something with better support. - -## Summary - -Accessibility automation is a crucial part of an accessibility strategy in any fast-moving, modern codebase. When multiple code changes are merged on a daily or weekly basis, there is no possibility to run manual checks on every state and variation of the application prior to merge. Understanding some of the details of an "always on" set of accessibility checks in your test pipeline, where a lot of states are automatically detected, will help you develop a feel for how Cypress Accessibility behaves and what kinds of things it is best at finding, as well as what might be left over for other forms of testing. diff --git a/docs/accessibility/guides/accessibility-automation/_category_.json b/docs/accessibility/guides/accessibility-automation/_category_.json deleted file mode 100644 index cbe6ea57c2..0000000000 --- a/docs/accessibility/guides/accessibility-automation/_category_.json +++ /dev/null @@ -1,5 +0,0 @@ -{ - "label": "Accessibility automation", - "collapsible": true, - "collapsed": false -} diff --git a/docs/accessibility/guides/blocking-pull-requests.mdx b/docs/accessibility/guides/blocking-pull-requests.mdx deleted file mode 100644 index 2b4fd92b51..0000000000 --- a/docs/accessibility/guides/blocking-pull-requests.mdx +++ /dev/null @@ -1,60 +0,0 @@ ---- -title: Blocking pull requests and setting policies -sidebar_position: 40 ---- - -Cypress Accessibility reports are generated on the server in Cypress Cloud, based on artifacts uploaded by the test runs. Because of this, there is no impact to your actual Cypress test execution, in terms of performance or anything else - including failing the Cypress run due to accessibility violations. - -## Using the Results API - -The [Cypress Accessibility Results API](/accessibility/results-api) provides access to accessibility results after the run is completed, so that you can implement patterns like blocking a pull request or sending out alerts based on specific accessibility criteria. This is done by adding a separate accessibility verification step to your CI pipeline, and using a helper function provided by Cypress to automatically request the report for the appropriate Cypress run based on the build context. The example below shows how this can look in GitHub Actions when there is an accessibility failure based on your own chosen policy: - - - -## Implementing a status check - -The API gives you full flexibility to examine the results and react to it in whatever way you need to. It also provides a signal that can be used in status checks on Pull Requests. Here is an example where the accessibility verification step is passing: - - - -## Implementing a policy in the verification step - -The [Results API Documentation](/accessibility/results-api) describes the capabilities of the API in more detail, but here is a simplified example of how to use the API to implement the "no new failing accessibility rules" policy shown above. - -```js -const { getAccessibilityResults } = require('@cypress/extract-cloud-results') - -// define your known failing rules -const rulesWithExistingViolations = [ - 'button-name', - ... -] - -// call the API -getAccessibilityResults() - .then((results) => { - // compare the accessibility results with your acceptable standard - const newRuleViolations = results.rules.filter((rule) => { - return !rulesWithExistingViolations.includes(rule.name) - }) - - if (newRuleViolations.length > 0) { - // based on the standard, trigger any side effect you like, such as failing the step - throw new Error( - `${newRuleViolations.length} rule regressions were introduced and must be fixed.` - ) - } -}) -``` - -This process of examining the results and then deciding the appropriate response gives you the most flexibility in terms of how exactly you want to force a stopping point or specific reaction based on the contents. You have access to the results as well as any context needed from the CI environment, such as tags, to implement the correct response to any specific result values. diff --git a/docs/accessibility/guides/improving-accessibility.mdx b/docs/accessibility/guides/improving-accessibility.mdx deleted file mode 100644 index ecbf2b27c3..0000000000 --- a/docs/accessibility/guides/improving-accessibility.mdx +++ /dev/null @@ -1,60 +0,0 @@ ---- -title: Improving accessibility -sidebar_position: 20 ---- - -In this section we'll discuss how to break down the data in your accessibility report and make a plan for remediation. - - - -## Starting from Scratch - -When you first start to use Cypress Accessibility, you may be testing a legacy application with many existing accessibility violations. The [Accessibility tab for a run](/accessibility/core-concepts/run-level-reports) can show multiple violations detected on dozens or hundreds of pages in your application. The number of failing elements and page or component snapshots available might be overwhelming, making it difficult to know where to start. - -In this situation, the first step is to understand some of the key variables and how they are related within your organization. The kinds of things to consider are: - -- your goals for the accessibility of the application -- your timeline for making it accessible -- the available resources for remediation -- what code is likely to change in the near future, where accessibility can be included in designs and requirements -- the consequences, risks, and costs of remaining inaccessible - -These factors will help you find the right strategy for engaging with the results displayed in your accessibility reports. While incremental accessibility efforts by designers and engineers are never wasted, the best outcomes come from an overall organizational commitment to ship accessible software, which requires investment from all departments, so understanding your organization's stance on accessibility and who needs to be in the conversation is also helpful. - -If your organization or team is newer to accessibility, you are likely to find plenty of issues reported by Axe Core® in Cypress Cloud. Many of the issues in your first report may relate to concepts that are new, such as [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) or [invalid nesting of interactive controls](https://dequeuniversity.com/rules/axe/4.2/nested-interactive). Since it will be impossible to fix all issues overnight, we recommend finding some low-hanging fruit as a starting point. This can help your team build confidence by getting some wins on the board, and teach them the overall idea of how accessibility problems get fixed. - -### Good low-hanging fruit - -Some of the easiest, and most valuable issues to tackle first come from [WCAG SC 4.2.1 \- Name, role, value](https://www.w3.org/WAI/WCAG21/Understanding/name-role-value.html). This is the part of the Web Content Accessibility Guidelines that’s concerned with making sure interactive controls have appropriate labels for all users. In general, your app is likely to have some violations in this category, and Axe-Core® tends to flag these as Critical or Serious, because the lack of a label usually makes it difficult or impossible for some users to know what something does in a user interface. - -Within the “Name, role, value” success criterion, the [`button-name`](https://dequeuniversity.com/rules/axe/4.7/button-name) rule in Axe Core® (“Buttons must have discernible text”) is a great place to start. Icons being used as buttons are common in modern user interfaces and they are often missing descriptive text content for screen reader users, which directly impacts their ability to understand the purpose and effect of the button and independently complete tasks in your application. - -This rule, and others like it, are good starting points because the rule is easy to understand and easy to fix. Since the solution is purely a code update to add the missing label text to buttons, this category of rule requires little or no coordination across departments or teams to address. These types of issues are also straightforward to demonstrate and validate even for beginners with a screen reader. - -Existing code failing the button-name rule can usually be remediated pretty quickly, as long as it is clear what the correct text label is for a given control. And when it comes to new code, this text label content is also easy to add to discussions in the design and requirements stages of the development cycle, shifting the accessibility conversation left. - -Since every violation in Cypress is linked to a Deque University page with details about the nature of the problem and how to fix it, all the needed context is available, even to engineers who may be new to the topic of accessiblity in general, or to a specific violation. - -Even if your first rule is not “Buttons must have discernible text”, we suggest finding one with a similarly small amount of coordination or understanding needed to fix, so the team can experience success as quickly as possible. - -### Finding low-hanging fruit with Cypress - -Cypress Accessibility generates a separate report for each page and component tested in a run. It’s also possible to see all accessibility rules that failed across the entire run. The rules-first approach is useful when finding low-hanging fruit. The rules with the smallest element count are the closest to being fully fixed, and removed from the report. If, at the “All Views” level, there’s not a good rule with a low element count, the next step is to focus on just a single View in the report. You might not be able to fix all buttons across the application, for example, but maybe you can pick a single page and fix just the issues on that specific page. - -### Setting the goal - -Your goal is to get your target rule’s failing elements down to zero for your chosen scope. That scope can be a specific page, a set of pages, or the entire application. Seeing that rule disappear from a report provides a clear finish line for you and your team. It also provides time to assess the impact of the changes. - -Often, due to the componentized nature of modern web apps, fixing an issue in one place has knock-on benefits around the app. Even if you focused on a narrow scope, your changes may still have had a wider impact, so remember: it’s fine for these first goals to be small at first. In fact, it’s preferable. - -### Iterating towards success {#iterating-towards-success} - -By setting a reasonable goal to understanding and fix specific rules scoped to defined areas of the application being tested, your team can make incremental progress on accessibility without becoming overwhelmed. With support from the Deque University “Learn More” links embedded into the reports, your team can specialize in one rule at a time, learn the related context, and then go deeper on more complicated or debatable topics. - -Given enough capacity, as dictated by the urgency of accessibility remediation combined with other business constraints, you will eventually solve all issues that are reported with Cypress Accessibility, and be well on your way to a more usable, accessible experience. - -If you are interested in using Cypress to detect and prevent regressions in the rules that you already have passing, you can learn how to use the Results API to do this in our [Blocking Pull Requests Section](/accessibility/guides/blocking-pull-requests). diff --git a/docs/accessibility/guides/introduction.mdx b/docs/accessibility/guides/introduction.mdx deleted file mode 100644 index 0ed4f67cbd..0000000000 --- a/docs/accessibility/guides/introduction.mdx +++ /dev/null @@ -1,37 +0,0 @@ ---- -title: Introduction -sidebar_position: 10 ---- - -:::info - -##### What you'll learn - -- How to set goals and make progress using Cypress Accessibility -- How to generate reports during local development cycles -- Common issues and questions related to all kinds of accessibility automation -- How to use Cypress Accessibility to monitor a production website -- Switching between "improvement" and "maintenance" mindsets - ::: - -## Making the most of Cypress Accessibility - -In these guides, we'll describe how to leverage Cypress Accessibility to improve and maintain the accessibility of the websites and applications you test with Cypress. We'll also discuss how to work with your Cypress point-of-contact to set the product up to provide the most value to you based on the goals you have for each specific app you are testing. - -Every project will be in a different state when you start to use Cypress Accessibility. You might find you have thousands of issues on hundreds of different pages and components, or you might have just a few issues detected in an already-accessible experience. - -For projects that have a high volume of issues, we'll describe how to use Cypress Accessibility to drive the process of remediation and improvement, get some measurable wins on the board for your team, and support the gradual introduction of accessibility standards. - -For projects with few or no issues currently, we’ll lay out how Cypress supports a resilient, efficient, and timely approach to maintain your standards and avoid surprises over the long term. - -The end goal of this process is not to simply pass all your automated accessibility checks, but to deliver a truly usable, accessible, and equal experience to all your users. So we'll also discuss limitations related to static automatic checks and how to write explicit test coverage to prevent regressions and increase confidence. - -## Contents - -1. [Improving Accessibility](/accessibility/guides/improving-accessibility) -1. [Maintaining Accessibility](/accessibility/guides/maintaining-accessibility) -1. [Blocking pull requests and setting policies](/accessibility/guides/blocking-pull-requests) -1. [Getting feedback during local development](/accessibility/guides/local-development) -1. [Production monitoring](/accessibility/guides/production-monitoring) -1. [Maximizing coverage](/accessibility/guides/maximizing-coverage) -1. [Accessibility automation principles](/accessibility/guides/accessibility-automation) diff --git a/docs/accessibility/guides/local-development.mdx b/docs/accessibility/guides/local-development.mdx deleted file mode 100644 index ee0190c1c5..0000000000 --- a/docs/accessibility/guides/local-development.mdx +++ /dev/null @@ -1,24 +0,0 @@ ---- -title: Feedback during local development -sidebar_position: 50 ---- - -# Feedback during local development - -While there are some workarounds possible with browser extensions or Cypress plugins, the most reliable solution for local development is to record to Cypress Cloud from your local development machine. This allows you to run a small set of tests directly related to your changes and fully skip your build pipeline and CI processes. It can be run with a one line command: - -``` -npx cypress run --key --record --spec "cypress/e2e/my-spec.cy.js" -``` - -(You can skip the `--key` option by setting `CYPRESS_RECORD_KEY` as an environment variable. Learn more about recording in our [docs about recorded runs](/cloud/features/recorded-runs)). - -This has 3 key benefits: - -- Accuracy \- you will see exactly what Cypress does with your new code changes, taking into account your project's configuration, and using the Cloud's version of Axe Core®. There will be no violations reports for rules, pages, or elements your team has chosen to ignore, for example. -- Speed \- since the application is already built locally, while you are working on it, it is ready to test immediately without your build pipeline -- Scope \- you can run only the tests you need, which can make results even faster, and helps you exclude issues unrelated to the area you are working on - -One other reason to get early feedback this way is to quickly know if, in fixing one accessibility problem, you’ve revealed or introduced another. Recording small runs locally is a big quality-of-life improvement when you are specifically looking for this kind of feedback. - -If you expect your team to do a lot of runs this way and that these partial results would clutter your Cypress project, you can set up a dedicated Cloud project for “locally generated” runs to keep things organized. diff --git a/docs/accessibility/guides/maintaining-accessibility.mdx b/docs/accessibility/guides/maintaining-accessibility.mdx deleted file mode 100644 index 2f63756a53..0000000000 --- a/docs/accessibility/guides/maintaining-accessibility.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: Maintaining accessibility -sidebar_position: 30 ---- - -This section discusses the different mindset and success metrics you may have when a project moves from remediating known issues, to long term accessibility maintenance. - -## Switching from “improving” to “maintaining” - -As soon as a given accessibility rule reaches zero violations for a given page, component, or application, you enter “maintenance mode” for that rule. If the content being tested is stable, any time a failure of that rule shows back up in the reports for that area on a feature branch, it will be clear that changes on that feature branch are the reason for the failure. These changes could be changes in the application code that made it less accessible, **or** changes in the tests to cover new parts of the application. - -If a previously-passing rule starts failing on your main branch after not being fixed in a pull request, you can trace back to the re-introduction of the failure and understand what code or test change is responsible. This allows for follow-up and training with developers who introduce issues and gives you the needed information to know about and fix the problem itself. - -Your overall goal should be to reach a fully-passing state on every rule. Cypress provides you a lot of flexibility to manage large amounts of accessibility violations and make incremental progress towards well-defined targets. Through a combination of fixing accessibility problems, and configuring Cypress to focus on a tight scope of rules, standards, and application areas that matter most to you, you might be closer than you think to a “clean” main branch, from which you can expand your standards over time. - -## Maintaining accessibility - -Once you reach the state of a report that contains no violations, your organization’s usage of Cypress Accessibility will change dramatically, and start to focus mainly on detecting and preventing accessibility problems before they merge. This can be a big adjustment. Chipping away at a large backlog of known issues, and fixing things rule-by-rule, provides a clear measure of progress to your team and a sense of momentum. It’s likely you will have been able to measure success and set goals in relation to the accessibility violations present on your main branch up to now. - -When those violations are all fixed, what exactly is Cypress Accessibility “doing” in your pipeline? And how should you measure success? - -### What Cypress Accessibility is doing when a project is in maintenance mode {#what-cypress-accessibility-is-doing-when-a-project-is-in-maintenance-mode} - -The short answer is: providing **pre-merge insights**, and **post-merge monitoring**, that supports you in maintaining accessibility. Accessibility debt tends to accumulate silently over time, leading to large, problematic backlogs that are difficult to work through, and land on your desk all at once after an audit. As you will have learned by going through the remediation process to get to a clean report in the first place: it’s not always easy to fix an already-inaccessible experience. Sometimes you have to start a whole feature or component over, because you’ve built multiple structures on top of an inaccessible foundation. - -Early feedback about accessibility on feature branches will allow developers to review small, precise reports, and get all the context they need to either fix issues immediately, or raise problematic patterns for wider discussion. This is especially helpful when a new team member joins, who isn’t as familiar with accessibility practices, or when a well-meaning full-stack developer – somebody who can “get by” in the front-end – unintentionally makes a minor change with major accessibility implications. - -Detecting problems before code merges allows the training, awareness-building, discussion, and remediation to all happen in the same cycle as regular code review, when everybody has all the context they need, not weeks or months later when it will be difficult to inject a ticket into the backlog to untangle what might have become a “load-bearing” accessibility problem. - -### How to measure success in maintenance mode {#how-to-measure-success-in-maintenance-mode} - -The issues surfaced in Cypress Accessibility should follow certain patterns, allowing you to get a good “pulse” on your team’s effectiveness with accessibility and time spent dealing with issues. The best-case scenario is that, with a stable team of front-end developers working on the product, the total number of violations on the main branch, as well as the number of violations identified and corrected on feature branches, should trend downwards over time. This driven by a buildup of education as a result of the continuous feedback. The presence of the feedback should _change developer behavior_ and truly help shift accessibility left, into questions that get asked and answered even before development, when a ticket is created or a design is handed off. At that point, mistakes are avoided, instead of corrected. - -We don’t expect most teams to reach a state where they always get everything right in the first code commit: there will always be some issues surfacing in Cypress Cloud – but as teams mature, it’s reasonable to expect the reports to contain less and less violations. This state is ideal \- your development teams should be focused on adding value to your company by making changes to the product. - -For most of you reading this, accessibility problems are quality issues, just like bugs and UI issues, and they are important and need to be addressed. This is why optimizing and minimizing the effort related to accessibility is so valuable in a literal sense \- your team is going to spend time on accessibility anyway, every cycle saved on dealing with accessibility is freed up for feature work, quality improvements, and other important activities. - -### Ideal conditions never last forever - -The main thing that would disrupt the downward trend of pre-merge violations is plain old reality. Teams change, priorities change, people move around an organization, they get hired, they leave, they go on vacation. All of this can be disruptive to an accessibility program – sometimes it’s a specific person or small group that is keeping standards up and providing guidance to their peers. When that person moves out of their team, a previously stable accessibility situation can start to regress. - -Cypress Accessibility will help you see this kind of disruption in real time and help you take action as needed. One of the biggest pain points that has been described to us by customers is unexpectedly having a large backlog of high-priority accessibility issues surfaced in an annual audit, even when they had fixed everything from the previous audit. This happens for a few reasons: sometimes the fixes introduce new problems, sometimes net-new work is added that includes new accessibility violations, or design changes lead to front-end modifications with unexpected accessibility implications. - -With Cypress Accessibility, you have an early warning system for these situations. They will show up as an increase in violations reported during code reviews, and there will be no chance for this accessibility debt to accumulate silently. Your team has an opportunity to “self-correct” when this happens, or if accessibility is disregarded and violations do get merged, you will be able to track that on your main branch. - -## Summary - -Accessibility is an ongoing process that affects the whole organization, not a single project of fixing a known list of issues and “completing” accessibility. The automation layer of accessibility tests offered by Cypress Accessibility maximizes the amount of information you can have at your fingertips to improve your accessibility status and maintain an accessible application over the long term, in a software development environment where both expected and unexpected changes create constant challenges for quality. - -Cypress Accessibility provides a consolidated report with page- and component-level breakdowns and interactive, visual DOM snapshots to understand exactly what every violation relates to in your application. With filtering and configuration you can create a manageable series of goals to solve every accessibility issue that Axe Core® can detect. Once your accessibility report is fully passing for the rules and pages that make sense for your business and project, Cypress Accessibility supports continuous feedback on new code changes before they merge, and gives you an early warning system if things are getting off track. - -The clarity and efficiency from using Cypress Accessibility will optimize your team’s time spent managing and fixing accessibility issues that are detectable by automation, helping them focus more on building great things that create value for your organization, without sacrificing this critical aspect of quality software development and building up debt over time to do so. diff --git a/docs/accessibility/guides/maximizing-coverage.mdx b/docs/accessibility/guides/maximizing-coverage.mdx deleted file mode 100644 index cb41b402f5..0000000000 --- a/docs/accessibility/guides/maximizing-coverage.mdx +++ /dev/null @@ -1,18 +0,0 @@ ---- -title: Maximizing coverage -sidebar_position: 70 ---- - -As you gain more familiarity with the nature of accessibility violations and the underlying ideas behind them, you will see the connection between the generic, static checks offered by Axe Core®, and the specific user-facing behavior of your own applications. In many cases, you will be able to add new assertions to your tests, based on what you learned from Cypress Accessibility, but customized to your specific situation. - -An example of this is when Cypress Accessibility reports a button with a missing accessible name. The actual text that should go with the button depends upon your own application, and is unknown to Axe-Core, so a label check will pass for almost _any_ text that is placed in the button. After you’ve added the required text to the codebase, you can write an assertion about that specific text in your Cypress tests, to make clear to future developers that it is part of the spec of the application. That’s a small example, but it applies in many larger situations too. - -## Accessibility “hotspots” - -Another way to leverage the violations detected by Cypress Accessibility is to treat each one as a sign that accessibility in that area of the application may not have been considered when it was built. Because the whole DOM of the page or component is available for every violation, it’s faster to do a manual check of the patterns being used in the page around an Axe Core® violation. This can reveal accessibility “hotspots” containing usability issues that aren’t otherwise detectable by automation, but would appear in a manual audit. These can be cataloged and corrected, then regression tested with traditional Cypress automation. - -## Combining testing approaches - -Combining the always-on generic checks provided by Cypress Accessibility with regular Cypress assertions about specific details of your application can help accessibility quality stay high once things are fixed, regardless of whether problems were originally detected manually or automatically. This can take your accessibility confidence far beyond the 57% of issues detected, often cited when thinking about Axe Core® as a standalone library. - -In the testing context, you are already specifying user-facing aspects of your application, and that means almost any findings from manual regression checks or user feedback, can be explicitly covered by your standard testing approach. So the overall potential of your automation pipeline to provide good feedback about accessibility regressions is very high, if you leverage explicit assertions on top of the static check "error detection" approach to find mistakes. diff --git a/docs/accessibility/guides/production-monitoring.mdx b/docs/accessibility/guides/production-monitoring.mdx deleted file mode 100644 index 786002f7b4..0000000000 --- a/docs/accessibility/guides/production-monitoring.mdx +++ /dev/null @@ -1,33 +0,0 @@ ---- -title: Production monitoring -sidebar_position: 60 ---- - -Many organizations use Cypress automation in CI, and for local development, but also run scheduled tests against production or staging environments. This pattern works especially well for situations where a Content Management System is involved and content authors can add or modify things that affect the user interface, or anything else where the content being tested is not fully controlled by the testing environment. Since it can be impossible to write explicit assertions for ever-changing content like this, or for A/B tests of different UI variations implemented outside your codebase, production smoke tests tend to be more high-level health checks. - -Cypress Accessibility makes this kind of testing even more useful, as it will be able to run just fine against that dynamic set of content. Just by visiting a production URL in your Cypress tests, and taking one or two light actions in the UI, that page will be captured in your accessibility report, and alert you about content changes outside of your development lifecycle that introduce accessibility issues. - -One common approach to this kind of monitoring, if the project does not already have Cypress tests, is to use something like a site map, or just an array of target URLs, to quickly spin up a lot of coverage without writing new tests for every page, and then run that Cypress test on a schedule to get daily updated accessibility reports. Here is an example of doing this by requesting the `sitemap.xml` file from a website and scrolling to the footer on each page. - -```js -describe('accessibility scan', () => { - it('checks accessibility with the sitemap.xml', () => { - cy.request('https:///sitemap.xml').then((response) => { - const xmlString = response.body - const parser = new DOMParser() - const xmlDoc = parser.parseFromString(xmlString, 'application/xml') - const URLs = Array.from(xmlDoc.querySelectorAll('loc')).map( - (loc) => loc.textContent - ) - Cypress._.each(URLs, (URL) => { - it(`visits ${URL}`, () => { - cy.visit(URL) - cy.contains('').scrollIntoView() - }) - }) - }) - }) -}) -``` - -The outcome of this is a first-page-load accessibility report for every URL in the site. Any Cypress UI tests for specific workflows will increase the coverage area automatically to include the states and variations reached during the workflows. diff --git a/docs/accessibility/results-api.mdx b/docs/accessibility/results-api.mdx index 7003e059b6..f42d85cd59 100644 --- a/docs/accessibility/results-api.mdx +++ b/docs/accessibility/results-api.mdx @@ -52,24 +52,6 @@ The Cypress App [repository](https://github.com/cypress-io/cypress) uses the Res ```javascript title="scripts/verifyAccessibilityResults.js" const { getAccessibilityResults } = require('@cypress/extract-cloud-results') -/** - * The list of rules that currently have 1+ elements that have been flagged with - * violations within the Cypress Accessibility report that need to be addressed. - * - * Once the violation is fixed in the Accessibility report, - * the fixed rule should be removed from this list. - * - * View the Accessibility report for the Cypress run in the Cloud - * for more details on how to address these failures. - */ -const rulesWithExistingViolations = [ - 'aria-required-children', - 'empty-heading', - 'aria-dialog-name', - 'link-in-text-block', - 'list', -] - getAccessibilityResults({ projectId: '...', // optional if set from env recordKey: '...', // optional if set from env @@ -78,10 +60,11 @@ getAccessibilityResults({ const { runNumber, accessibilityReportUrl, summary, rules } = results const { total } = summary.violationCounts - console.log( - `Received ${summary.isPartialReport ? 'partial' : ''} results for run #${runNumber}.` - ) - console.log(`See full report at ${accessibilityReportUrl}.`) + console + .log( + `Received ${summary.isPartialReport ? 'partial' : ''} results for run #${runNumber}.` + ) + .console.log(`See full report at ${accessibilityReportUrl}.`) // write your logic to conditionally fail based on the results if (total === 0) { @@ -118,6 +101,24 @@ getAccessibilityResults({ } console.log('No new Accessibility violations detected!') + + /** + * The list of rules that currently have 1+ elements that have been flagged with + * violations within the Cypress Accessibility report that need to be addressed. + * + * Once the violation is fixed in the Accessibility report, + * the fixed rule should be removed from this list. + * + * View the Accessibility report for the Cypress run in the Cloud + * for more details on how to address these failures. + */ + const rulesWithExistingViolations = [ + 'aria-required-children', + 'empty-heading', + 'aria-dialog-name', + 'link-in-text-block', + 'list', + ] }) ``` @@ -242,10 +243,10 @@ env: jobs: run-cypress: - runs-on: ubuntu-24.04 + runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v4 + uses: actions/checkout@v3 - name: install run: npm install - name: Run @@ -288,7 +289,7 @@ run-cypress: pipeline { agent { docker { - image 'cypress/base:22.12.0' + image 'cypress/base:latest' } } @@ -359,7 +360,7 @@ version: 2.1 jobs: linux-test: docker: - - image: cypress/base:22.12.0 + - image: cypress/base:latest working_directory: ~/repo steps: diff --git a/docs/api/commands/origin.mdx b/docs/api/commands/origin.mdx index d98a40bd00..9d8f5f89e9 100644 --- a/docs/api/commands/origin.mdx +++ b/docs/api/commands/origin.mdx @@ -3,8 +3,8 @@ title: origin e2eSpecific: true --- -Visit multiple domains of different -[origin](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy#definition_of_an_origin) +Visit multiple different +[origins](https://developer.mozilla.org/en-US/docs/Web/Security/Same-origin_policy#definition_of_an_origin) in a single test. In normal use, a single Cypress test may only run commands in a single origin, a @@ -28,6 +28,18 @@ doc. ::: +:::info + +Changes in Cypress [v14.0.0](/app/references/changelog#14-0-0) + +Cypress no longer injects `document.domain` by default, which means `cy.origin()` +must now be used to navigate between any two origins in the same test, even if +the two origins are in the same superdomain. This behavior can be disabled by setting +the `injectDocumentDomain` configuration option to `true`, to allow a smooth transition +of tests to the new behavior. In Cypress 15, this configuration option will be removed. + +::: + ## Syntax ```js @@ -63,26 +75,28 @@ cy.get('h1').contains('My cool site under test') const hits = getHits() cy.visit('https://example.cypress.io/history/founder') // To interact with cross-origin content, move this inside cy.origin() callback -cy.get('h1').contains('About our Founder') -// Domain must be a precise match including subdomain, i.e. example.cypress.io -cy.origin('cypress.io', () => { - cy.visit('/history/founder') - cy.get('h1').contains('About our Founder') - // Fails because hits is not passed in via args - cy.get('#hitcounter').contains(hits) +cy.get('h1').contains('Kitchen Sink') +// Origin must be a precise match including scheme, subdomain and port, i.e. https://www.cypress.io +cy.origin('https://www.cypress.io', () => { + cy.visit('/about-us') + cy.get('h1').contains('About us') + // Fails because doanloads is not passed in via args + cy.contains(downloads) }) -// Won't work because still on cypress.io -cy.get('h1').contains('My cool site under test') +// Won't work because still on www.cypress.io +cy.get('h1').contains('Kitchen Sink') ``` ### Arguments - **url _(String)_** + **origin _(String)_** -A URL specifying the secondary origin in which the callback is to be executed. -This should at the very least contain a hostname, and may also include the -protocol, port number & path. The hostname must precisely match that of the -secondary origin, including all subdomains. Query params are not supported. +A string specifying the origin in which the callback is to be executed. +This should at the very least contain a hostname. It may also include the +scheme and port number. The path may be included, but is not necessary. +The hostname must precisely match that of the secondary origin, including +all subdomains. Query params are not supported. If no scheme is provided, +the scheme defaults to `https`. This argument will be used in two ways: @@ -542,6 +556,7 @@ inclusion in a future version of Cypress. | Version | Changes | | ------------------------------------------- | ----------------------------------------------------------------------------------------------------------- | +| [14.0.0](/app/references/changelog#14-0-0) | `cy.origin()` is now required when navigating between origins in the same test, rather than superdomains. | | [12.6.0](/app/references/changelog#10-7-0) | Support for `Cypress.require()` added and support for CommonJS `require()` and ES module `import()` removed | | [10.11.0](/app/references/changelog#10-7-0) | Support for CommonJS `require()` and ES module `import()` added and support for `Cypress.require()` removed | | [10.7.0](/app/references/changelog#10-7-0) | Support for `Cypress.require()` added | diff --git a/docs/api/commands/window.mdx b/docs/api/commands/window.mdx index 6afc4bf2e8..e2a16100e9 100644 --- a/docs/api/commands/window.mdx +++ b/docs/api/commands/window.mdx @@ -174,7 +174,7 @@ It fails. But the interesting thing is that the type of `event` is `KeyboardEvent` when you `console.log(event)`. It's because Cypress uses an `iframe` to load the application under test. In -other words, the `KeyboardEvent` used in the code above and the +other words, the `KeyboardEvent` used in the the code above and the `KeyboardEvent` class from which the `event` variable is constructed are different `KeyboardEvent`s. diff --git a/docs/api/cypress-api/require.mdx b/docs/api/cypress-api/require.mdx index d12ca9b14e..a67f2bf852 100644 --- a/docs/api/cypress-api/require.mdx +++ b/docs/api/cypress-api/require.mdx @@ -15,7 +15,7 @@ in browser-targeted code. :::caution -Using `Cypress.require()` within the [`cy.origin()`](/api/commands/origin) +Using `Cypress.require()` within the [`cy.origin()`](https://docs.cypress.io/api/commands/origin) callback requires enabling the [`experimentalOriginDependencies`](/app/references/experiments#End-to-End-Testing) option in the Cypress configuration. diff --git a/docs/api/node-events/overview.mdx b/docs/api/node-events/overview.mdx index acd7b1e02a..57fc7dde69 100644 --- a/docs/api/node-events/overview.mdx +++ b/docs/api/node-events/overview.mdx @@ -234,7 +234,7 @@ Cypress does this by spawning an independent `child_process` which then `requires` the [Cypress configuration file](/app/references/configuration). This is similar to the way Visual Studio Code or Atom works. -This code will be executed using the Node version that launched Cypress. +This code will be executed using the the Node version that launched Cypress. ### npm modules diff --git a/docs/app/continuous-integration/aws-codebuild.mdx b/docs/app/continuous-integration/aws-codebuild.mdx index 4a54c7cd3f..423cc4c502 100644 --- a/docs/app/continuous-integration/aws-codebuild.mdx +++ b/docs/app/continuous-integration/aws-codebuild.mdx @@ -132,14 +132,14 @@ version: 0.2 ## https://docs.aws.amazon.com/codebuild/latest/userguide/batch-build-buildspec.html ## Define build to run using the -## "cypress/browsers:22.12.0" image +## "cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1" image ## from the Cypress Amazon ECR Public Gallery batch: fast-fail: false build-list: - identifier: cypress-e2e-tests env: - image: public.ecr.aws/cypress-io/cypress/browsers:22.12.0 + image: public.ecr.aws/cypress-io/cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1 phases: install: diff --git a/docs/app/continuous-integration/bitbucket-pipelines.mdx b/docs/app/continuous-integration/bitbucket-pipelines.mdx index 1f1d85296c..052610f2c8 100644 --- a/docs/app/continuous-integration/bitbucket-pipelines.mdx +++ b/docs/app/continuous-integration/bitbucket-pipelines.mdx @@ -18,13 +18,35 @@ title: Bitbucket Pipelines Detailed documentation is available in the [Bitbucket Pipelines Documentation](https://support.atlassian.com/bitbucket-cloud/docs/get-started-with-bitbucket-pipelines/). -Bitbucket runs most builds in Docker containers as described in [Docker image options](https://support.atlassian.com/bitbucket-cloud/docs/docker-image-options/). +The example below shows a basic setup and job to use +[Bitbucket Pipelines](https://bitbucket.org/product/features/pipelines) to run +end-to-end tests with Cypress and Electron. -If you use the currently available default Bitbucket / Atlassian Linux images listed in the -"Default build environment" of the Bitbucket Cloud documentation -[Use Docker images as build environments](https://support.atlassian.com/bitbucket-cloud/docs/use-docker-images-as-build-environments/) you must additionally install [Cypress Linux prerequisites](/app/get-started/install-cypress#Linux-Prerequisites). +```yaml title="bitbucket-pipelines.yml" +image: node:latest + +pipelines: + default: + - step: + script: + # install dependencies + - npm ci + # start the server in the background + - npm run start & + # run Cypress tests + - npm run e2e +``` + +**How this `bitbucket-pipelines.yml` works:** -For a simpler setup, use a Cypress Docker image, as described in the following section. +- On _push_ to this repository, this job will provision and start Bitbucket + Pipelines-hosted Linux instance for running the pipelines defined in the + `pipelines` section of the configuration. +- The code is checked out from our GitHub/Bitbucket repository. +- Finally, our scripts will: + - Install npm dependencies + - Start the project web server (`npm start`) + - Run the Cypress tests within our GitHub/Bitbucket repository within Electron ## Testing with Cypress Docker Images @@ -34,10 +56,10 @@ Cypress locally and in CI, with some images including Chrome, Firefox and Edge. example, this allows us to run the tests in Firefox by passing the `--browser firefox` attribute to `cypress run`. -Read about [Cypress Docker variants](/app/continuous-integration/overview#Cypress-Docker-variants) to decide which image is best for your project. +Read about [Cypress docker variants](/app/continuous-integration/overview#Cypress-Docker-variants) to decide which image is best for your project. ```yaml title="bitbucket-pipelines.yml" -image: cypress/browsers:22.12.0 +image: cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1 pipelines: default: @@ -51,17 +73,6 @@ pipelines: - npx cypress run --browser firefox ``` -**How this `bitbucket-pipelines.yml` works:** - -- On _push_ to this repository, this job will provision and start Bitbucket - Pipelines using the Cypress Docker image. It will run the pipelines defined in the - `pipelines` section of the configuration. -- The code is checked out from the Bitbucket repository. -- Finally, our scripts will: - - Install npm dependencies - - Start the project web server (`npm start`) - - Run the Cypress tests within the Bitbucket repository using Firefox - ## Caching Dependencies and Build Artifacts Per the @@ -76,7 +87,7 @@ Artifacts from a job can be defined by providing paths to the `artifacts` attribute. ```yaml title="bitbucket-pipelines.yml" -image: cypress/browsers:22.12.0 +image: cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1 pipelines: default: @@ -140,7 +151,7 @@ recording test results to [Cypress Cloud](/cloud/get-started/introduction). ::: ```yaml title="bitbucket-pipelines.yml" -image: cypress/base:22.12.0 +image: cypress/base:22.11.0 ## job definition for running E2E tests in parallel e2e: &e2e @@ -199,7 +210,7 @@ definitions: The complete `bitbucket-pipelines.yml` is below: ```yaml title="bitbucket-pipelines.yml" -image: cypress/base:22.12.0 +image: cypress/base:22.11.0 ## job definition for running E2E tests in parallel e2e: &e2e diff --git a/docs/app/continuous-integration/circleci.mdx b/docs/app/continuous-integration/circleci.mdx index b4af9ed1bd..3e180b455f 100644 --- a/docs/app/continuous-integration/circleci.mdx +++ b/docs/app/continuous-integration/circleci.mdx @@ -42,7 +42,7 @@ orbs: workflows: build: jobs: - - cypress/run: # "run" job comes from "cypress" orb + - cypress/run # "run" job comes from "cypress" orb start-command: 'npm run start' ``` diff --git a/docs/app/continuous-integration/github-actions.mdx b/docs/app/continuous-integration/github-actions.mdx index c5199cd9ef..fa921cbefa 100644 --- a/docs/app/continuous-integration/github-actions.mdx +++ b/docs/app/continuous-integration/github-actions.mdx @@ -98,7 +98,7 @@ on: push jobs: cypress-run: - runs-on: ubuntu-24.04 + runs-on: ubuntu-22.04 steps: - name: Checkout uses: actions/checkout@v4 @@ -180,9 +180,9 @@ on: push jobs: cypress-run: - runs-on: ubuntu-24.04 + runs-on: ubuntu-22.04 container: - image: cypress/browsers:22.12.0 + image: cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1 options: --user 1001 steps: - name: Checkout @@ -229,7 +229,7 @@ on: push jobs: install: - runs-on: ubuntu-24.04 + runs-on: ubuntu-22.04 steps: - name: Checkout uses: actions/checkout@v4 @@ -249,7 +249,7 @@ jobs: path: build cypress-run: - runs-on: ubuntu-24.04 + runs-on: ubuntu-22.04 needs: install steps: - name: Checkout @@ -325,7 +325,7 @@ on: push jobs: install: - runs-on: ubuntu-24.04 + runs-on: ubuntu-22.04 steps: - name: Checkout uses: actions/checkout@v4 @@ -363,7 +363,7 @@ jobs: # ... omitted install job from above cypress-run: - runs-on: ubuntu-24.04 + runs-on: ubuntu-22.04 needs: install strategy: # don't fail the entire matrix on failure @@ -494,7 +494,7 @@ on: push jobs: cypress-run: name: Cypress run - runs-on: ubuntu-24.04 + runs-on: ubuntu-22.04 steps: - name: Checkout uses: actions/checkout@v4 @@ -521,7 +521,7 @@ on: push jobs: cypress-run: name: Cypress run - runs-on: ubuntu-24.04 + runs-on: ubuntu-22.04 steps: - name: Checkout uses: actions/checkout@v4 diff --git a/docs/app/continuous-integration/gitlab-ci.mdx b/docs/app/continuous-integration/gitlab-ci.mdx index e5f5a916ec..40f6a977e3 100644 --- a/docs/app/continuous-integration/gitlab-ci.mdx +++ b/docs/app/continuous-integration/gitlab-ci.mdx @@ -68,7 +68,7 @@ stages: - test test: - image: cypress/browsers:22.12.0 + image: cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1 stage: test script: # install dependencies @@ -99,7 +99,7 @@ cache: - .npm/ test: - image: cypress/browsers:22.12.0 + image: cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1 stage: test script: # install dependencies @@ -166,7 +166,7 @@ cache: ## Install npm dependencies and Cypress install: - image: cypress/browsers:22.12.0 + image: cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1 stage: build script: - npm ci @@ -211,13 +211,13 @@ cache: ## Install npm dependencies and Cypress install: - image: cypress/browsers:22.12.0 + image: cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1 stage: build script: - npm ci ui-chrome-tests: - image: cypress/browsers:22.12.0 + image: cypress/browsers:node-22.11.0-chrome-130.0.6723.69-1-ff-132.0-edge-130.0.2849.56-1 stage: test parallel: 5 script: diff --git a/docs/app/continuous-integration/overview.mdx b/docs/app/continuous-integration/overview.mdx index 82c0ea1deb..0b6ade0a37 100644 --- a/docs/app/continuous-integration/overview.mdx +++ b/docs/app/continuous-integration/overview.mdx @@ -212,9 +212,9 @@ cypress run --record --key=abc123 --parallel CI providers, such as [GitHub Actions](https://docs.github.com/en/actions/using-jobs/running-jobs-in-a-container) and [CircleCI](https://circleci.com/docs/executor-intro/#docker), allow workflows to run using -[Docker container images](https://docs.docker.com/get-started/docker-concepts/the-basics/what-is-a-container/). +[Docker container images](https://docs.docker.com/app/docker-concepts/the-basics/what-is-a-container/). -Cypress supports the use of [Docker](https://docs.docker.com/get-started/docker-overview/) +Cypress supports the use of [Docker](https://docs.docker.com/app/docker-overview/) through the provisioning of official [Cypress Docker images](https://github.com/cypress-io/cypress-docker-images). Images are Linux-based and support the following platforms: @@ -235,7 +235,7 @@ Cypress Docker images are available from: - [cypress/base](https://github.com/cypress-io/cypress-docker-images/tree/master/base) is the entry-level Cypress Docker image, allowing you to test in the Electron browser, built in to Cypress. It contains a complete Linux (Debian) operating system, together with the - [prerequisite operating system packages](/app/get-started/install-cypress#UbuntuDebian) for Cypress, + [prerequisite operating system packages](https://docs.cypress.io/app/get-started/install-cypress#UbuntuDebian) for Cypress, Node.js, npm and Yarn v1 Classic. An image `` gives you the choice of Node.js version. diff --git a/docs/app/core-concepts/best-practices.mdx b/docs/app/core-concepts/best-practices.mdx index b73c4c7eeb..d6dbedf45e 100644 --- a/docs/app/core-concepts/best-practices.mdx +++ b/docs/app/core-concepts/best-practices.mdx @@ -236,7 +236,7 @@ recommend you approach testing your components, look to: [Cypress Component Test Selecting elements with data attributes, text content, or Testing Library locators can each have some different implications for accessibility, but none of these approaches is a "complete" accessibility test, and you will always need additional, accessibility-specific testing (including automated and manual tests) to confirm -your application is working as expected for people with disabilities and the technology they use. See [our accessibility testing guide](/app/guides/accessibility-testing) for more details and comparisons of approaches. +your application is working as expected for people with disabilities and the technology they use. See[our accessibility testing guide](/app/guides/accessibility-testing) for more details and comparisons of approaches. ## Assigning Return Values @@ -312,7 +312,7 @@ or interact with sites or servers you do not control. that you control. Try to avoid visiting or requiring a 3rd party server. If you choose, you may use [`cy.request()`](/api/commands/request) to talk to 3rd party servers via their APIs. If possible, cache results via [`cy.session()`](/api/commands/session) -to avoid repeat visits. See also reasons against [Testing Apps You Don't Control](/app/end-to-end-testing/writing-your-first-end-to-end-test#Testing-Apps-You-Dont-Control). +to avoid repeat visits. ::: diff --git a/docs/app/end-to-end-testing/writing-your-first-end-to-end-test.mdx b/docs/app/end-to-end-testing/writing-your-first-end-to-end-test.mdx index da5226f2e4..9bcb2297a0 100644 --- a/docs/app/end-to-end-testing/writing-your-first-end-to-end-test.mdx +++ b/docs/app/end-to-end-testing/writing-your-first-end-to-end-test.mdx @@ -225,6 +225,35 @@ have failed. title="First test with cy.visit()" /> +:::danger + +Testing Apps You Don't Control + +In this guide we are testing our example application: +[`https://example.cypress.io`](https://example.cypress.io). However you should +think carefully about testing applications you **don't control**. Why? + +- They have the potential to change at any moment which will break tests. +- They may do A/B testing which makes it impossible to get consistent results. +- They may detect you are a script and block your access. +- They may have security features enabled which prevent Cypress from working. + +Generally speaking, the point of Cypress is to be a tool you use every day to +build and test your own applications, not a general purpose web automation tool. +However, this is a guideline rather than a hard-and-fast rule and there are a +number of good reasons to make exceptions for certain kinds of application: + +- They are specifically designed to integrate with third parties, e.g. SSO + providers. +- They provide you with a complementary service, e.g. SaaS control panels or + analytics. +- They reuse your content or provide plugins for an app you control. + +The key here is to carefully weigh the benefits of the tests in question against +the possible disruption and flake these sorts of tests can introduce. + +::: + ### Step 2: Query for an element Now that we've got a page loaded, we need to take some action on it. Why don't @@ -482,45 +511,6 @@ If you want a minimal code approach to creating tests, you can use interactions and generate tests. Visit our [guide](/app/guides/cypress-studio) for more information. -## Testing Apps You Don't Control - -:::danger - - **Anti-Pattern:** Trying to visit -or interact with sites or servers you do not control. - -::: - -In this guide we are testing our example application: -[https://example.cypress.io](https://example.cypress.io). -However you should think carefully about testing applications you **don't control** -or you haven't been invited to test by the owner. -Why? - -- They may have security features enabled which prevent Cypress from working, - such as detecting Cypress script usage. This can block your access and make it - appear that the application website is unresponsive. -- They have the potential to change at any moment which will break tests. -- They may do A/B testing which makes it impossible to get consistent results. - -Generally speaking, the point of Cypress is to be a tool you use every day to -build and test your own applications, not a general purpose web automation tool. -However, this is a guideline rather than a hard-and-fast rule and there are a -number of good reasons to make exceptions for certain kinds of application: - -- They are specifically designed to integrate with third parties, e.g. SSO - providers. -- They provide you with a complementary service, e.g. SaaS control panels or - analytics. -- They reuse your content or provide plugins for an app you control. - -The key here is to carefully weigh the benefits of the tests in question against -the possible disruption and flake these sorts of tests can introduce. - -See also [Visiting External Sites](/app/core-concepts/best-practices#Visiting-External-Sites), -on our [Best Practices](/app/core-concepts/best-practices) page, -which discusses strategies when this is necessary. - ## Next steps - Take our free diff --git a/docs/app/faq.mdx b/docs/app/faq.mdx index 28a0798a30..092736d97d 100644 --- a/docs/app/faq.mdx +++ b/docs/app/faq.mdx @@ -452,7 +452,7 @@ a remote page and does not resolve until all of the external resources complete their loading phase. Because we expect your applications to observe differing load times, this command's default timeout is set to 60000ms. If you visit an invalid url or a -[second unique domain](/app/guides/cross-origin-testing#Different-superdomain-per-test-requires-cyorigin), +[second unique domain](/app/guides/cross-origin-testing#Different-origins-per-test-require-cyorigin), Cypress will log a verbose yet friendly error message. **_In CI, how do I make sure my server has started?_** @@ -1253,10 +1253,24 @@ steps, Cypress adds its own overhead. Thus, the performance numbers you get from Cypress tests are slower than "normal" use. Still, you can access the native `window.performance` object and grab the page time measurements, see the [Evaluate performance metrics](https://github.com/cypress-io/cypress-example-recipes#testing-the-dom) -recipe. +recipe. You can also +[run Lighthouse audit straight from Cypress](https://www.testingwithmarie.com/post/web-performance-testing-with-google-lighthouse) +via [cypress-audit](https://www.npmjs.com/package/cypress-audit) community +plugin. ## Integrations with Other Tools/Frameworks/Libraries +### Can I test Gatsby.js sites using Cypress? + +For end-to-end tests, yes, as you can read in the official +[Gatsby docs](https://www.gatsbyjs.com/docs/end-to-end-testing/). You can also +watch the "Cypress + Gatsby webinar" +[recording](https://www.youtube.com/watch?v=Tx6Lg9mwcCE). + +For component testing, Gatsby is not currently supported out of the box, but it +might be possible by +[configuring a custom devServer](/app/references/configuration#devServer). + ### Can I test React applications using Cypress? For end-to-end testing, yes, absolutely. A good example of a fully tested React @@ -1484,5 +1498,5 @@ You will need to add a `tsconfig.json` to the folder and specify the types the files inside that folder should use. Don't forget to update your -[`specPattern`](/app/references/configuration#component) +[`specPattern`](https://docs.cypress.io/app/references/configuration#component) to include the new file location. diff --git a/docs/app/get-started/install-cypress.mdx b/docs/app/get-started/install-cypress.mdx index 68ff6af50f..0729817929 100644 --- a/docs/app/get-started/install-cypress.mdx +++ b/docs/app/get-started/install-cypress.mdx @@ -106,15 +106,15 @@ need prebuilt. ### Operating System -Cypress supports running under these operating systems: +Cypress is a desktop application that is installed on your computer. The desktop +application supports these operating systems: - **macOS** 10.15 and above _(Intel or Apple Silicon 64-bit (x64 or arm64))_. -- **Linux** Ubuntu 20.04 and above, Fedora 40 and above, and Debian 11 and above _(x64 or arm64)_ (see [Linux Prerequisites](#Linux-Prerequisites) down +- **Linux** Ubuntu 20.04 and above, Fedora 39 and above, and Debian 11 and above _(x64 or arm64)_ (see [Linux Prerequisites](#Linux-Prerequisites) down below). - Cypress deprecated the use of Node.js `16.x` in Cypress [`13.0.0`](/app/references/changelog#13-0-0). We recommend that users update to at least Node.js `18.x`. For related reasons, Cypress deprecates the use of Linux operating systems with library [`glibc`](https://www.gnu.org/software/libc/) versions `2.17` - `2.27`. The Linux CLI command `ldd --version` displays your glibc version. - **Windows** 10 and above _(x64)_. -- **Windows Server** 2019 and 2022 _(x64)_. ### Node.js @@ -189,7 +189,7 @@ It is worth noting that Cypress on Linux requires a minimum [`glibc`](https://ww apt-get install libgtk2.0-0 libgtk-3-0 libgbm-dev libnotify-dev libnss3 libxss1 libasound2 libxtst6 xauth xvfb ``` -For Ubuntu 24.04, and above, use the following command: +For Ubuntu 24.04 use the following command: ```shell apt-get install libgtk2.0-0t64 libgtk-3-0t64 libgbm-dev libnotify-dev libnss3 libxss1 libasound2t64 libxtst6 xauth xvfb diff --git a/docs/app/guides/cross-origin-testing.mdx b/docs/app/guides/cross-origin-testing.mdx index 5f67abeadc..ce0c74e719 100644 --- a/docs/app/guides/cross-origin-testing.mdx +++ b/docs/app/guides/cross-origin-testing.mdx @@ -31,9 +31,6 @@ and we are _mostly_ able to do this. ### What Cypress does under the hood -- Injects - [`document.domain`](https://developer.mozilla.org/en-US/docs/Web/API/Document/domain) - into `text/html` pages. - Proxies all HTTP / HTTPS traffic. - Changes the hosted URL to match that of the application under test. - Uses the browser's internal APIs for network level traffic. @@ -55,7 +52,7 @@ Cypress must assign and manage browser certificates to be able to modify the traffic in real time. You'll notice Chrome display a warning that the 'SSL certificate does not match'. This is expected behavior. Under the hood we act as our own CA authority and issue certificates dynamically in order to intercept requests -otherwise impossible to access. We only do this for the superdomain currently +otherwise impossible to access. We only do this for the origin currently under test, and bypass other traffic. Note, that Cypress allows you to optionally specify CA / client certificate @@ -72,39 +69,37 @@ It's important to note that although we do our **very best** to ensure your application works normally inside of Cypress, there _are_ some limitations you need to be aware of. -- [Different superdomain per test requires cy.origin command](#Different-superdomain-per-test-requires-cyorigin) +- [Different origins per test require cy.origin()](#Different-origins-per-test-require-cyorigin) - [Cross-origin iframes are not supported](#Cross-origin-iframes) - [Navigating from HTTPS to HTTP will error](#Insecure-Content) - [Cypress requires that the URLs navigated to have the same port](#Same-port-per-test) -### Different superdomain per test requires cy.origin +### Different origins per test require `cy.origin()` Cypress changes its own host URL to match that of your applications. With the exception of `cy.origin`, Cypress requires that the URLs navigated to have the -[same superdomain](/app/guides/cross-origin-testing#Parts-of-a-URL) for the +[origin](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Origin) for the entirety of a single test. -If you attempt to visit two different superdomains, the `cy.origin` command must +If you attempt to visit two different origins, the `cy.origin` command must be used to wrap Cypress commands of the second visited domain. Otherwise, Cypress commands will timeout after the navigation and will eventually error. This is because the commands that were expected to run on the second domain are actually being run on the first domain. -Without `cy.origin`, you can visit different superdomains in _different_ tests, -but not in the _same_ test. Please read our -[Cross Origin Testing Guide](/app/guides/cross-origin-testing) for more -information. +Without `cy.origin()`, you can visit different origins in _different_ tests, +but not in the _same_ test. #### Examples of test cases that will error without the use of `cy.origin` 1. [`.click()`](/api/commands/click) an `` with an `href` to a different - superdomain with subsequent Cypress commands being run. + origin with subsequent Cypress commands being run. 2. [`.submit()`](/api/commands/submit) a `
` that causes your web server to - redirect you a different superdomain where additional Cypress commands are + redirect you to a different origin where additional Cypress commands are run. 3. Issue a JavaScript redirect in your application, such as - `window.location.href = '...'`, to a different superdomain where additional - Cypress commands are run. + `window.location.href = '...'`, which navigates to a different origin + where additional Cypress commands are run. In each of these situations, Cypress will lose the ability to automate your application and will error via command timeout unless the `cy.origin` command is @@ -113,17 +108,6 @@ used. Read on to learn about [working around these common problems](#Common-Workarounds). -#### What is a superdomain? - -But what is same superdomain? It's actually very similar to that of same -origin! Two URLs have the same origin if the **protocol**, **port** (if specified), and -**host** match. Cypress automatically handles hosts of the same superdomain by -injecting the -[`document.domain`](https://developer.mozilla.org/en-US/docs/Web/API/Document/domain) -property into the visited `text/html` pages. This is why navigations without the -use of the [`cy.origin()`](/api/commands/origin) command are solely scoped to the -same superdomain. - #### Parts of a URL @@ -136,38 +120,33 @@ chart to help clarify the differences! alt="Diagram showing the parts of a URL including href broken into protocol, host, pathname, hash and smaller components of hostname, port, path, and search with and example URL" /> -Given the URLs below, all have the same superdomain compared to -`https://www.cypress.io`. - -- `https://cypress.io` -- `https://docs.cypress.io` -- `https://example.cypress.io/commands/querying` +##### Origin -The URLs below, however, will have different superdomains/origins compared to -`https://www.cypress.io`. +An `origin` is comprised of a URL's `scheme`, `hostname`, _and_ `port`. Given the URLs +below, all have the same origin compared to `https://www.cypress.io`: -- `http://www.cypress.io` (Different protocol) -- `https://docs.cypress.io:81` (Different port) -- `https://www.auth0.com/` (Different host of different superdomain) +- `https://www.cypress.io/cloud` +- `https://www.cypress.io/app` -The `http://localhost` URLs differ if their ports are different. For example, -the `http://localhost:3000` URL is considered to be a different origin from the -`http://localhost:8080` URL. +But the following have different origins compared to `https://www.cypress.io`: -The rules are: +- `http://www.cypress.io` (different `scheme`) +- `https://docs.cypress.io` (different `hostname` due to the subdomain) +- `https://www.auth0.com` (different `hostname`) +- `https://www.cypress.io:81` (different `port`) :::danger You **cannot** [visit](/api/commands/visit) -two domains of different superdomains in the same test and continue to interact with -the page without the use of the [`cy.origin()`](/api/commands/origin) command. +two different origins in the same test and continue to interact with the page without +the use of the [`cy.origin()`](/api/commands/origin) command. ::: :::tip You **can** [visit](/api/commands/visit) -two or more domains of different origin in **different** tests without needing [`cy.origin()`](/api/commands/origin). +two or more origins in **different** tests without needing [`cy.origin()`](/api/commands/origin). ::: For practical purposes, this means the following: @@ -176,16 +155,16 @@ For practical purposes, this means the following: // This test will run without error it('navigates', () => { cy.visit('https://www.cypress.io') - cy.visit('https://docs.cypress.io') + cy.visit('https://www.cypress.io/app') cy.get('selector') // yup all good }) ``` ```javascript -// this will error because cypress-dx.com doesn't match the cypress.io superdomain +// this will error because the origin https://docs.cypress.io doesn't match the origin https://www.cypress.io it('navigates', () => { cy.visit('https://www.cypress.io') - cy.visit('https://www.cypress-dx.com') + cy.visit('https://docs.cypress.io') cy.get('selector') }) ``` @@ -196,8 +175,8 @@ the sequential command should run against: ```javascript it('navigates', () => { cy.visit('https://example.cypress.io') - cy.visit('https://www.cypress-dx.com') - cy.origin('https://www.cypress-dx.com', () => { + cy.visit('https://docs.cypress.io') + cy.origin('https://docs.cypress.io', () => { cy.get('selector') // yup all good }) }) @@ -306,7 +285,7 @@ and break down how to work around them in Cypress. ### External Navigation The most common situation where you might encounter this error is when you click -on an `` that navigates to another superdomain. +on an `` that navigates to another origin. ```html title="index.html" @@ -322,14 +301,14 @@ cy.get('selector').should('exist') // Cypress errors :::warning -We don't recommend visiting a superdomain that you don't control in your tests +We don't recommend visiting an origin that you don't control in your tests which you can read more about [here](/app/core-concepts/best-practices#Visiting-External-Sites) ::: -If you control this superdomain, either by owning the hosted instance -or by other means, we recommend testing this superdomain with `cy.origin`. +If you control this origin, either by owning the hosted instance +or by other means, we recommend testing this origin with `cy.origin`. ```javascript title="test.cy.js" cy.visit('http://localhost:8080') // where your web server + HTML is hosted @@ -340,7 +319,7 @@ cy.origin('https://example.cypress.io', () => { }) ``` -If you're not in control of this superdomain, we recommend you test that the `href` +If you're not in control of this origin, we recommend you test that the `href` property is correct instead of performing the navigation. This will help lead to more deterministic tests. @@ -385,7 +364,7 @@ cy.get('form').submit() // submit the form! ``` If your back end server handling the `/submit` route does a `30x` redirect to a -different superdomain, you'll need to use the `cy.origin` command if running +different origin, you'll need to use the `cy.origin` command if running additional Cypress commands after submitting the form. ```javascript title="routes.js" @@ -533,7 +512,7 @@ Setting `chromeWebSecurity` to `false` in Chrome-based browsers allows you to do the following: - Display insecure content -- Navigate to any superdomain without cross-origin errors with or without +- Navigate to any origin without cross-origin errors with or without `cy.origin` - Access cross-origin iframes that are embedded in your application @@ -586,7 +565,7 @@ Want to enable `experimentalModifyObstructiveThirdPartyCode`? Let's do it! ## Other workarounds -There are other ways of testing the interaction between two superdomains. The +There are other ways of testing the interaction between two origins. The browser has a natural security barrier called `origin policy` this means that state like `localStorage`, `cookies`, `service workers` and many other APIs are not shared between them anyways. Cypress does offer APIs around `localStorage`, @@ -596,7 +575,7 @@ As a best practice, you should not visit or interact with any website not under your control. If your organization uses Single Sign On (SSO) or OAuth then you might choose to -test a 3rd party service other than your superdomain, which can be tested with +test a 3rd party service other than your origin, which can be tested with [`cy.origin()`](/api/commands/origin). We've written several other guides specifically about handling this situation. diff --git a/docs/app/guides/migration/protractor-to-cypress.mdx b/docs/app/guides/migration/protractor-to-cypress.mdx index 2ba1e27c0d..cc69425530 100644 --- a/docs/app/guides/migration/protractor-to-cypress.mdx +++ b/docs/app/guides/migration/protractor-to-cypress.mdx @@ -88,7 +88,7 @@ With [Cypress](/app/core-concepts/open-mode), your tests run in an interactive browser environment in real time. Cypress's [Command Log](/app/core-concepts/open-mode#Command-Log) displays the tests from your test suite and their assertions. When you -[click on a command or assertion](/app/core-concepts/open-mode#Command-Log) +[click on a command or assertion](https://docs.cypress.io/app/core-concepts/open-mode#Clicking-on-Commands) in the command log, a DOM snapshot displays so you can see what the application under test looked like at the time of the test's execution. This allows you to see the **real rendered UI** and the behavior of the app under **real user @@ -98,7 +98,7 @@ scenario. Cypress also helps you to write your tests by making it as easy as possible to find the right CSS selectors for the DOM elements in your application with its -[Selector Playground](/app/core-concepts/open-mode#Selector-Playground) and [Cypress Studio](/app/guides/cypress-studio). +[Selector Playground](https://docs.cypress.io/app/core-concepts/open-mode#Selector-Playground) and [Cypress Studio](/app/guides/cypress-studio). These tools help you cut down on time spent finding the right selectors so you can focus on what's important: writing tests that verify your app's logic. @@ -216,7 +216,7 @@ Check out the documentation for more details like how to [configure your tests to run in a specific browser](#Running-the-builder-with-a-specific-browser) or [record test results](#Recording-test-results-to-Cypress-Cloud) to -[Cypress Cloud](/cloud/get-started/introduction). +[Cypress Cloud](https://docs.cypress.io/cloud/get-started/introduction). ::: @@ -707,7 +707,7 @@ cy.get('#example-input') Cypress has one additional feature that can make a critical difference in the reliability of your tests' assertions: -[retry-ability](/app/core-concepts/retry-ability). +[retry-ability](https://docs.cypress.io/app/core-concepts/retry-ability). When your test fails an assertion or command, Cypress will mimic a real user with build-in wait times and multiple attempts at asserting your tests in order to minimize the amount of false negatives / positives. @@ -1077,7 +1077,7 @@ environment. parallel across multiple CI machines. With Cypress, your tests can be -[parallelized on a per spec file basis](/cloud/features/smart-orchestration/parallelization). +[parallelized on a per spec file basis](https://docs.cypress.io/cloud/features/smart-orchestration/parallelization). This is an important distinction between Protractor and Cypress parallelization. One of the reasons why Cypress parallelizes tests per file is because some users may write tests that build up state that subsequent tests, although we @@ -1136,7 +1136,7 @@ to learn more. ### Recording test results to Cypress Cloud We recommend setting your -[Cypress Cloud](/cloud/get-started/introduction) recording key +[Cypress Cloud](https://docs.cypress.io/cloud/get-started/introduction) recording key as an environment variable and _NOT_ as a builder option when running it in CI. ```json diff --git a/docs/app/references/advanced-installation.mdx b/docs/app/references/advanced-installation.mdx index 805c0b5f3f..840e46395d 100644 --- a/docs/app/references/advanced-installation.mdx +++ b/docs/app/references/advanced-installation.mdx @@ -25,7 +25,7 @@ title: Advanced Installation | `CYPRESS_CACHE_FOLDER` | [Changes the Cypress binary cache location](#Binary-cache) | | `CYPRESS_RUN_BINARY` | [Location of Cypress binary at run-time](#Run-binary) | | `CYPRESS_VERIFY_TIMEOUT` | Overrides the timeout duration for the `verify` command. The default value is 30000. | -| `CYPRESS_SKIP_VERIFY` | Skips the [`verify` command](command-line#cypress-verify) when `true` | +| `CYPRESS_SKIP_VERIFY` | Skips the `verify` command (for stable CI environments) | | ~~CYPRESS_SKIP_BINARY_INSTALL~~ | removed use `CYPRESS_INSTALL_BINARY=0` instead | | ~~CYPRESS_BINARY_VERSION~~ | removed use `CYPRESS_INSTALL_BINARY` instead | @@ -186,56 +186,32 @@ ensure this, consider exporting this environment variable. For example, in a Setting the environment variable `CYPRESS_RUN_BINARY` overrides where the npm module finds the Cypress binary. -`CYPRESS_RUN_BINARY` should be a path to an already unzipped Cypress binary executable. -The Cypress commands [open](./command-line#cypress-open), [run](./command-line#cypress-run) and [verify](command-line#cypress-verify) -will then launch the provided binary. - -The following example [cypress run](./command-line#cypress-run) commands assume that you have first -downloaded the Cypress binary to the default `Downloads` directory of your operating system. - -Depending on how you then unzip the downloaded Cypress binary `cypress.zip` file, -using a CLI command or alternatively a GUI interface, -the directory structure may include one additional top-level directory named `cypress`, -which you may need to add to the path defined by `CYPRESS_RUN_BINARY`. - -If available, use the following to avoid the additional top-level directory level: - -```shell -unzip -q cypress -``` - -:::note - -The examples below are for npm. -If you are using Yarn or pnpm as package manager, replace `npx` with `yarn` or `pnpm` as appropriate. -See [How to run commands](./command-line.mdx#How-to-run-commands). - -::: +`CYPRESS_RUN_BINARY` should be a path to an already unzipped binary executable. +The Cypress commands `open`, `run`, and `verify` will then launch the provided +binary. ### Mac ```shell -CYPRESS_RUN_BINARY=~/Downloads/Cypress.app/Contents/MacOS/Cypress npx cypress run +CYPRESS_RUN_BINARY=~/Downloads/Cypress.app/Contents/MacOS/Cypress cypress run ``` ### Linux ```shell -CYPRESS_RUN_BINARY=~/Downloads/Cypress/Cypress npx cypress run +CYPRESS_RUN_BINARY=~/Downloads/Cypress/Cypress cypress run ``` ### Windows ```shell -CYPRESS_RUN_BINARY=~/Downloads/Cypress/Cypress.exe npx cypress run +CYPRESS_RUN_BINARY=~/Downloads/Cypress/Cypress.exe cypress run ``` -:::tip +:::caution -Cypress assumes that `CYPRESS_RUN_BINARY` points to a writeable directory structure so that it can save and re-use -the results of verifying the Cypress binary. -If you encounter a `permission denied` failure message from [cypress verify](./command-line.mdx#cypress-verify), -you may be able to work around the failure by setting the environment variable `CYPRESS_SKIP_VERIFY` to `true`. +We recommend **not exporting** the `CYPRESS_RUN_BINARY` environment variable, +since it will affect every cypress module installed on your file system. ::: diff --git a/docs/app/references/changelog.mdx b/docs/app/references/changelog.mdx index 6e40caaf43..e7fea889ef 100644 --- a/docs/app/references/changelog.mdx +++ b/docs/app/references/changelog.mdx @@ -847,7 +847,7 @@ Refer to the [v13 Migration Guide](/app/references/migration-guide#Migrating-to- **Features:** -- Added [`--runner-ui`](/app/references/command-line#cypress-run-runner-ui) and [`--no-runner-ui`](/app/references/command-line#cypress-run-no-runner-ui) CLI flags to control whether the Cypress Runner UI is visible during a run. Addressed in [#27582](https://github.com/cypress-io/cypress/pull/27582). +- Added [`--runner-ui`](/app/references/command-line#cypress-run-runner-ui) and [`--no-runner-ui`](https://docs.cypress.io/app/references/command-line#cypress-run-no-runner-ui) CLI flags to control whether the Cypress Runner UI is visible during a run. Addressed in [#27582](https://github.com/cypress-io/cypress/pull/27582). - Consolidates and improves terminal output when uploading test artifacts to Cypress Cloud. Addressed in [#27402](https://github.com/cypress-io/cypress/pull/27402) **Bugfixes:** @@ -1542,11 +1542,11 @@ _Released 1/24/2023_ will help us make decisions to improve memory issues. Addresses [#23391](https://github.com/cypress-io/cypress/issues/23391). - Added new - [`experimentalSkipDomainInjection`](/app/references/experiments#Experimental-Skip-Domain-Injection) + `experimentalSkipDomainInjection` configuration option to disable Cypress from setting `document.domain` on injection, allowing users to test Salesforce domains. If you believe you are having `document.domain` issues, please see the - [`experimentalSkipDomainInjection`](/app/references/experiments#Experimental-Skip-Domain-Injection) + `experimentalSkipDomainInjection` guide. This config option is end-to-end only. Addresses [#2367](https://github.com/cypress-io/cypress/issues/2367), [#23958](https://github.com/cypress-io/cypress/issues/23958), @@ -1599,7 +1599,7 @@ _Released 1/03/2023_ - Added support for mapping the `CYPRESS_PULL_REQUEST_ID`, `CYPRESS_PULL_REQUEST_URL`, and/or `CYPRESS_CI_BUILD_URL` environment - variables to the corresponding Cloud run. This provides workarounds when + variables to the the corresponding Cloud run. This provides workarounds when supported CI provider mappings are incorrect or unsupported CI providers are used. Addressed in [#25036](https://github.com/cypress-io/cypress/pull/25036). - Added new Cypress API, diff --git a/docs/app/references/command-line.mdx b/docs/app/references/command-line.mdx index cf3e93bfe1..d7f8f34ef8 100644 --- a/docs/app/references/command-line.mdx +++ b/docs/app/references/command-line.mdx @@ -204,7 +204,7 @@ cypress run [options] | `--parallel` | [Run recorded specs in parallel across multiple machines](#cypress-run-parallel) | | `--port`,`-p` | [Override default port](#cypress-run-port-lt-port-gt) | | `--project`, `-P` | [Path to a specific project](#cypress-run-project-lt-project-path-gt) | -| `--quiet`, `-q` | [Reduce output to `stdout`](#cypress-run-quiet) | +| `--quiet`, `-q` | If passed, Cypress output will not be printed to `stdout`. Only output from the configured [Mocha reporter](/app/tooling/reporters) will print. | | `--record` | [Whether to record the test run](#cypress-run-record-key-lt-record-key-gt) | | `--reporter`, `-r` | [Specify a Mocha reporter](#cypress-run-reporter-lt-reporter-gt) | | `--reporter-options`, `-o` | [Specify Mocha reporter options](#cypress-run-reporter-lt-reporter-gt) | @@ -434,17 +434,6 @@ To see this in action we've set up an cypress run --project ./some/nested/folder ``` -#### `cypress run --quiet` {#cypress-run-quiet} - -To reduce the output from Cypress printed to `stdout`, use -`--quiet`. - -```shell -cypress run --quiet -``` - -If passed, Cypress will only print output to `stdout` from the [built-in](/app/tooling/reporters#Built-in-reporters) default [Mocha spec reporter](https://mochajs.org/#spec), or from any other configured [Mocha reporter](/app/tooling/reporters). - #### `cypress run --record --key ` {#cypress-run-record-key-lt-record-key-gt} Record your test results to [Cypress Cloud](/cloud/get-started/introduction). For @@ -834,8 +823,6 @@ and `cypress run` commands. The `CYPRESS_VERIFY_TIMEOUT` environment variable should be provided for those commands if you wish to modify the timeout duration. -You can disable verification by setting the `CYPRESS_SKIP_VERIFY` environment variable to `true`. - ### `cypress version` Prints the installed Cypress binary version, the Cypress package version, the diff --git a/docs/app/references/configuration.mdx b/docs/app/references/configuration.mdx index f19767b929..e30962b3ed 100644 --- a/docs/app/references/configuration.mdx +++ b/docs/app/references/configuration.mdx @@ -146,7 +146,6 @@ For more options regarding screenshots, view the | Option | Default | Description | | ----------------------- | ------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `defaultBrowser` | `null` | The default browser to launch if the "--browser" command line option is not provided. This option only affects the first browser launch; changing this option after the browser is already launched will have no effect. | | `chromeWebSecurity` | `true` | Whether to enable Chromium-based browser's Web Security for same-origin policy and insecure mixed content. | | `blockHosts` | `null` | A String or Array of hosts that you wish to block traffic for. [Please read the notes for examples on using this.](#blockHosts) | | `modifyObstructiveCode` | `true` | Whether Cypress will search for and replace obstructive JS code in `.js` or `.html` files. [Please read the notes for more information on this setting.](#modifyObstructiveCode) | @@ -185,16 +184,17 @@ creating `e2e` and `component` objects inside your Cypress configuration. These options are available to be specified inside the `e2e` configuration object: -| Option | Default | Description | -| ------------------------- | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -| `baseUrl` | `null` | URL used as prefix for [`cy.visit()`](/api/commands/visit) or [`cy.request()`](/api/commands/request) command's URL. | -| `setupNodeEvents` | `null` | Function in which node events can be registered and config can be modified. Takes the place of the (removed) pluginFile option. [Please read the notes for examples on using this.](#setupNodeEvents) | -| `supportFile` | `cypress/support/e2e.{js,jsx,ts,tsx}` | Path to file to load before spec files load. This file is compiled and bundled. (Pass `false` to disable) | -| `specPattern` | cypress/e2e/\*\*/\*.cy.\{js,jsx,ts,tsx\} | A String or Array of glob patterns of the test files to load. | -| `excludeSpecPattern` | `*.hot-update.js` | A String or Array of glob patterns used to ignore test files that would otherwise be shown in your list of tests. [Please read the notes on using this.](#excludeSpecPattern) | -| `experimentalRunAllSpecs` | `false` | Enables the "Run All Specs" UI feature, allowing the execution of multiple specs sequentially. | -| `slowTestThreshold` | `10000` | Time, in milliseconds, to consider a test "slow" during `cypress run`. A slow test will display in orange text in the default reporter. | -| `testIsolation` | `true` | Whether or not [test isolation](/app/core-concepts/writing-and-organizing-tests#Test-Isolation) is enabled to ensure a clean browser context between tests. | +| Option | Default | Description | +| --------------------------------------------------------------- | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| `baseUrl` | `null` | URL used as prefix for [`cy.visit()`](/api/commands/visit) or [`cy.request()`](/api/commands/request) command's URL. | +| `setupNodeEvents` | `null` | Function in which node events can be registered and config can be modified. Takes the place of the (removed) pluginFile option. [Please read the notes for examples on using this.](#setupNodeEvents) | +| `supportFile` | `cypress/support/e2e.{js,jsx,ts,tsx}` | Path to file to load before spec files load. This file is compiled and bundled. (Pass `false` to disable) | +| `specPattern` | cypress/e2e/\*\*/\*.cy.\{js,jsx,ts,tsx\} | A String or Array of glob patterns of the test files to load. | +| `excludeSpecPattern` | `*.hot-update.js` | A String or Array of glob patterns used to ignore test files that would otherwise be shown in your list of tests. [Please read the notes on using this.](#excludeSpecPattern) | +| `experimentalRunAllSpecs` | `false` | Enables the "Run All Specs" UI feature, allowing the execution of multiple specs sequentially. | +| `slowTestThreshold` | `10000` | Time, in milliseconds, to consider a test "slow" during `cypress run`. A slow test will display in orange text in the default reporter. | +| `testIsolation` | `true` | Whether or not [test isolation](/app/core-concepts/writing-and-organizing-tests#Test-Isolation) is enabled to ensure a clean browser context between tests. | +| `injectDocumentDomain` deprecated | `false` | Instructs Cypress to use the `document.domain` property to reduce the need for [`cy.origin`](/api/commands/origin). [Please read the notes on using this.](#injectDocumentDomain) | :::cypress-config-example @@ -590,6 +590,54 @@ The `**/node_modules/**` pattern is automatically added to `excludeSpecPattern` and does not need to be specified (and can't be overridden). See [`e2e`](#e2e) or [`component`](#component) testing-specific options. +### injectDocumentDomain + +This option is deprecated, and will be removed in Cypress. 15 + +Set this configuration option to `true` to instruct Cypress to +[inject document.domain](/app/guides/cross-origin-testing#What-Cypress-does-under-the-hood) +into your test application. This can reduce the need for `cy.origin()` when [navigating +between subdomains](/app/guides/cross-origin-testing), but comes with compatibility +caveats for some sites. + +This configuration option is provided to ease the transition between `cy.origin()`'s behavior +in Cypress 13 and the default behavior in Cypress 14. It will be removed in Cypress 15. +[Read the Cypress 14 migration guide](/app/references/migration-guide#Migrating-to-Cypress-140) to understand how to update your tests to remove +the need to set this flag. + +This configuration value _must_ be set to true if you are running tests with experimental +WebKit support, as `cy.origin` is not yet supported in WebKit. + +:::caution + +#### Known Incompatibilities + +Setting this configuration value to `true` can cause the application you are testing +to behave in unexpected ways. This is especially true if a site your tests visit +sets the [`Origin-Agent-Cluster`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Origin-Agent-Cluster) +header. + +At this point in time, we are aware of the following sites that cannot be tested properly +if this option is set to `true`: + +- Azure AD B2C authentication workflows +- Salesforce +- Google + +::: + +:::cypress-config-example + +```js +{ + e2e: { + injectDocumentDomain: true + } +} +``` + +::: + ### isInteractive You can open Cypress in the interactive mode via the `cypress open` command, and @@ -714,26 +762,27 @@ DEBUG=cypress:cli,cypress:data-context:sources:FileDataSource,cypress:data-conte ## History -| Version | Changes | -| -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------- | -| [13.16.0](/app/references/changelog#13-16-0) | Added `defaultBrowser` option. | -| [13.4.0](/app/references/changelog#13-4-0) | Added support for configuring the Experimental Flake Detection strategy via `retries.experimentalStrategy` and `retries.experimentalOptions`. | -| [13.0.0](/app/references/changelog#13-0-0) | Removed `nodeVersion` option. | -| [13.0.0](/app/references/changelog#13-0-0) | Removed `videoUploadOnPasses` option. | -| [11.0.0](/app/references/changelog#11-0-0) | Removed `e2e.experimentalSessionAndOrigin` option. | -| [10.4.0](/app/references/changelog#10-4-0) | Added `e2e.testIsolation` option. | -| [10.0.0](/app/references/changelog#10-0-0) | Reworked page to support new `cypress.config.js` and deprecated `cypress.json` files. | -| [8.7.0](/app/references/changelog#8-7-0) | Added `slowTestThreshold` option. | -| [8.0.0](/app/references/changelog#8-0-0) | Added `clientCertificates` option and removed `firefoxGcInterval` configuration. | -| [7.0.0](/app/references/changelog#7-0-0) | Added `e2e` and `component` options. | -| [7.0.0](/app/references/changelog#7-0-0) | Added `redirectionLimit` option. | -| [6.1.0](/app/references/changelog#6-1-0) | Added `scrollBehavior` option. | -| [5.2.0](/app/references/changelog#5-2-0) | Added `includeShadowDom` option. | -| [5.0.0](/app/references/changelog#5-0-0) | Added `retries` configuration. | -| [5.0.0](/app/references/changelog#5-0-0) | Renamed `blacklistHosts` configuration to `blockHosts`. | -| [4.1.0](/app/references/changelog#4-12-0) | Added `screenshotOnRunFailure` configuration. | -| [4.0.0](/app/references/changelog#4-0-0) | Added `firefoxGcInterval` configuration. | -| [3.5.0](/app/references/changelog#3-5-0) | Added `nodeVersion` configuration. | +| Version | Changes | +| ------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | +| [14.0.0](/app/references/changelog#14-0-0) | Added support for re-enabling superdomain navigation with the `injectDocumentDomain` configuration option | +| [13.16.0](/app/references/changelog#13-16-0) | Added `defaultBrowser` option. +| [13.4.0](/app/references/changelog#13-4-0) | Added support for configuring the Experimental Flake Detection strategy via `retries.experimentalStrategy` and `retries.experimentalOptions`. | +| [13.0.0](/app/references/changelog#13-0-0) | Removed `nodeVersion` option. | +| [13.0.0](/app/references/changelog#13-0-0) | Removed `videoUploadOnPasses` option. | +| [11.0.0](/app/references/changelog#11-0-0) | Removed `e2e.experimentalSessionAndOrigin` option. | +| [10.4.0](/app/references/changelog#10-4-0) | Added `e2e.testIsolation` option. | +| [10.0.0](/app/references/changelog#10-0-0) | Reworked page to support new `cypress.config.js` and deprecated `cypress.json` files. | +| [8.7.0](/app/references/changelog#8-7-0) | Added `slowTestThreshold` option. | +| [8.0.0](/app/references/changelog#8-0-0) | Added `clientCertificates` option and removed `firefoxGcInterval` configuration. | +| [7.0.0](/app/references/changelog#7-0-0) | Added `e2e` and `component` options. | +| [7.0.0](/app/references/changelog#7-0-0) | Added `redirectionLimit` option. | +| [6.1.0](/app/references/changelog#6-1-0) | Added `scrollBehavior` option. | +| [5.2.0](/app/references/changelog#5-2-0) | Added `includeShadowDom` option. | +| [5.0.0](/app/references/changelog#5-0-0) | Added `retries` configuration. | +| [5.0.0](/app/references/changelog#5-0-0) | Renamed `blacklistHosts` configuration to `blockHosts`. | +| [4.1.0](/app/references/changelog#4-12-0) | Added `screenshotOnRunFailure` configuration. | +| [4.0.0](/app/references/changelog#4-0-0) | Added `firefoxGcInterval` configuration. | +| [3.5.0](/app/references/changelog#3-5-0) | Added `nodeVersion` configuration. | ## See also diff --git a/docs/app/references/experiments.mdx b/docs/app/references/experiments.mdx index 71316b7774..5058186fef 100644 --- a/docs/app/references/experiments.mdx +++ b/docs/app/references/experiments.mdx @@ -257,63 +257,11 @@ creating `e2e` and `component` objects inside your Cypress configuration. These experiments are available to be specified inside the `e2e` configuration object: -| Option | Default | Description | -| --------------------------------- | ------- | ---------------------------------------------------------------------------------------------------------- | -| `experimentalStudio` | `false` | Generate and save commands directly to your test suite by interacting with your app as an end user would. | -| `experimentalRunAllSpecs` | `false` | Enables the "Run All Specs" UI feature, allowing the execution of multiple specs sequentially. | -| `experimentalOriginDependencies` | `false` | Enables support for `Cypress.require` within `cy.origin`. | -| `experimentalSkipDomainInjection` | `null` | Removes injecting `document.domain` into `text/html` pages for any sites that match the provided patterns. | - -#### Experimental Skip Domain Injection - -Under the hood, Cypress -[injects document.domain](/app/guides/cross-origin-testing#What-Cypress-does-under-the-hood) -into your test application to lessen the burden of navigation. This is well -described in our [Cross Origin Testing](/app/guides/cross-origin-testing) -guide. However, some sites have compatibility issues with this feature. - -The `experimentalSkipDomainInjection` option disables injecting -`document.domain` inside Cypress. When enabled, all cross-origin/subdomain -navigation must use `cy.origin()`, which may make tests a bit more verbose. We -only recommend including your site pattern if you are having issues running -Cypress out of the box and suspect setting `document.domain` is interfering with -your site's ability to render properly. - -Before enabling, verify your application is not implementing frame busting -techniques, which you can mitigate with the -[`modifyObstructiveCode`](/app/references/configuration#modifyObstructiveCode) -and -[`experimentalModifyObstructiveThirdPartyCode`](/app/guides/cross-origin-testing#Modifying-Obstructive-Third-Party-Code) -flags. - -At this point in time, we are aware of the following sites that require the -`experimentalSkipDomainInjection` option to be set to be tested properly: - -- Google -- Salesforce - -This flag can be enabled by passing an array of origin URLs or -[minimatch](https://github.com/isaacs/minimatch) glob patterns: - -:::cypress-config-example - -```js -{ - e2e: { - experimentalSkipDomainInjection: [ - '*.salesforce.com', - '*.force.com', - '*.google.com', - ] - } -} -``` - -::: - -If using other Salesforce domains, such as -[enhanced domains](https://help.salesforce.com/s/articleView?id=sf.domain_name_enhanced.htm&type=5), -you will need to add the correct matching glob pattern. +| Option | Default | Description | +| -------------------------------- | ------- | --------------------------------------------------------------------------------------------------------- | +| `experimentalStudio` | `false` | Generate and save commands directly to your test suite by interacting with your app as an end user would. | +| `experimentalRunAllSpecs` | `false` | Enables the "Run All Specs" UI feature, allowing the execution of multiple specs sequentially. | +| `experimentalOriginDependencies` | `false` | Enables support for `Cypress.require` within `cy.origin`. | ### Component Testing diff --git a/docs/app/references/launching-browsers.mdx b/docs/app/references/launching-browsers.mdx index c89b69b237..6037ed88aa 100644 --- a/docs/app/references/launching-browsers.mdx +++ b/docs/app/references/launching-browsers.mdx @@ -31,7 +31,6 @@ using number of browsers including: - [Chrome](https://www.google.com/chrome/) - [Chrome Beta](https://www.google.com/chrome/beta/) - [Chrome Canary](https://www.google.com/chrome/canary/) -- [Chrome for Testing](https://github.com/GoogleChromeLabs/chrome-for-testing/) - [Chromium](https://www.chromium.org/Home) - [Edge](https://www.microsoft.com/edge) - [Edge Beta](https://www.microsoftedgeinsider.com/download) @@ -67,7 +66,7 @@ See each browser's official release schedule for more information. The Chrome browser is evergreen - meaning it will automatically update itself, sometimes causing a breaking change in your automated tests. You can use the information in [Download Chromium](https://on.cypress.io/chromium-downloads) to download a -specific released version of Chrome for Testing or Chromium for every platform. +specific released version of Chromium for every platform. ### Electron Browser @@ -127,12 +126,6 @@ Or Chrome Canary: cypress run --browser chrome:canary ``` -Or Chrome for Testing: - -```shell -cypress run --browser chrome-for-testing -``` - ### Edge Browsers Microsoft Edge-family (Chromium-based) browsers are supported by Cypress. @@ -278,11 +271,11 @@ list of browsers you want available for selection during `cypress open`. // name: 'chrome', // channel: 'canary', // family: 'chromium', -// displayName: 'Chrome Canary', -// version: '133.0.6890.0', +// displayName: 'Canary', +// version: '80.0.3966.0', // path: -// '/Applications/Google Chrome Canary.app/Contents/MacOS/Canary', -// majorVersion: 133 +// '/Applications/Canary.app/Contents/MacOS/Canary', +// majorVersion: 80 // } return { browsers: config.browsers.filter( @@ -360,7 +353,8 @@ in the **Settings** tab. ### Unsupported Browsers -Some browsers such as Internet Explorer are not currently supported. +Some browsers such as Safari and Internet Explorer are not currently supported. +Support for more browsers is on our roadmap. ## Browser Environment diff --git a/docs/app/references/migration-guide.mdx b/docs/app/references/migration-guide.mdx index 26368544d9..808406b3f1 100644 --- a/docs/app/references/migration-guide.mdx +++ b/docs/app/references/migration-guide.mdx @@ -33,6 +33,52 @@ Starting in Cypress 14, Cypress will officially support [the latest 3 major vers Older browser versions may still work with Cypress, but we recommend keeping your browsers up to date to ensure compatibility with Cypress. +### Changes to `cy.origin()` + +To account for Chrome's [impending deprecation](https://developer.chrome.com/blog/document-domain-setter-deprecation) of setting `document.domain`, and to support sites that use [origin-keyed agent clusters](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Origin-Agent-Cluster), +Cypress no longer injects `document.domain` into `text/html` content by default. + +Because of this, tests that visit more than one origin (defined as a composite of the URL _scheme_, _hostname_, and _port_) must now use `cy.origin()`. +Without `cy.origin()`, interacting with a second origin in the same test will cause the test to fail, even if the two origins +are in the same superdomain. This means you must now use `cy.origin()` in more situations than before. + + **Failing Test** ```js cy.visit('https://www.cypress.io') +cy.visit('https://docs.cypress.io') cy.get('[role="banner"]').should('be.visible') +// Cypress will not be able to interact with the page, causing the test to fail ``` + + **Fixed Test** ```js cy.visit('https://www.cypress.io') +cy.visit('https://docs.cypress.io') cy.origin('https://docs.cypress.io', () => {cy + .get('[role="banner"]') + .should('be.visible')} +) ``` + +:::info + +To ease this transition, Cypress v14.0 introduced the ["injectDocumentDomain" configuration option](/app/references/configuration#injectDocumentDomain). When this option +is set to true, `cy.origin()` will not be required to navigate between origins, as long as the superdomain matches. + + If `injectDocumentDomain` is set to `true`, +Cypress will warn that this option is deprecated. + + `injectDocumentDomain` will be removed in Cypress +15. + + Setting `injectDocumentDomain` to `true` may +cause certain sites to stop working in Cypress. Please read the [configuration notes](/app/references/configuration#injectDocumentDomain) +before use. + + If your test suites require `experimentalWebKitSupport`, +`injectDocumentDomain` must be set to `true`. + + Chrome may remove support for `document.domain` +at any time; if this configuration option is enabled, Cypress may cease to work in +Chrome at any time. If this occurs, Chrome will raise an issue in its developer tools +indicating that the deprecated `document.domain` is in use. To resolve this issue, +set the `injectDocumentDomain` option to `false` and issue any newly necessary `cy.origin()` +commands.{' '} + +::: + ### Deprecation of `resourceType` on `cy.intercept` The `resourceType` option on [`cy.intercept`](/api/commands/intercept) has been deprecated in Cypress 14.0.0. We anticipate the types of the `resourceType` to change in the future or be completely removed @@ -68,7 +114,6 @@ If you would like to disable JIT compilation, you can do so by setting [`justInT For users with the existing `experimentalJustInTimeCompile` flag set, you can remove this flag from your configuration. - ### React `<18` CT no longer supported With [LTS ending](https://github.com/reactjs/react.dev/issues/1745#issuecomment-466767389) for React 16 and 17 several years ago, the minimum required React version for component testing is now `18.0.0`. diff --git a/docs/app/references/troubleshooting.mdx b/docs/app/references/troubleshooting.mdx index 373a6e00d1..5670ccd602 100644 --- a/docs/app/references/troubleshooting.mdx +++ b/docs/app/references/troubleshooting.mdx @@ -415,94 +415,58 @@ On Mac, Cypress attempts to find installed browsers by their bundle identifier. If this does not succeed, it will fall back to the Linux browser detection method. -| Browser Name | Expected Bundle Identifier | Expected Executable Path | -| -------------------- | ------------------------------------- | -------------------------------------------------------------------------------------- | -| `chrome` | `com.google.Chrome` | `/Applications/Google Chrome.app/Contents/MacOS/Google Chrome` | -| `chrome:beta` | `com.google.Chrome.beta` | `/Applications/Google Chrome Beta.app/Contents/MacOS/Google Chrome Beta` | -| `chrome:canary` | `com.google.Chrome.canary` | `/Applications/Google Chrome Canary.app/Contents/MacOS/Google Chrome Canary` | -| `chrome-for-testing` | `com.google.chrome.for.testing` | `/Applications/Google Chrome for Testing.app/Contents/MacOS/Google Chrome for Testing` | -| `chromium` | `org.chromium.Chromium` | `/Applications/Chromium.app/Contents/MacOS/Chromium` | -| `firefox` | `org.mozilla.firefox` | `/Applications/Firefox.app/Contents/MacOS/firefox` | -| `firefox:dev` | `org.mozilla.firefoxdeveloperedition` | `/Applications/Firefox Developer Edition.app/Contents/MacOS/firefox` | -| `firefox:nightly` | `org.mozilla.nightly` | `/Applications/Firefox Nightly.app/Contents/MacOS/firefox` | -| `edge` | `com.microsoft.Edge` | `/Applications/Microsoft Edge.app/Contents/MacOS/Microsoft Edge` | -| `edge:beta` | `com.microsoft.Edge.Beta` | `/Applications/Microsoft Edge Beta.app/Contents/MacOS/Microsoft Edge Beta` | -| `edge:canary` | `com.microsoft.Edge.Canary` | `/Applications/Microsoft Edge Canary.app/Contents/MacOS/Microsoft Edge Canary` | -| `edge:dev` | `com.microsoft.Edge.Dev` | `/Applications/Microsoft Edge Dev.app/Contents/MacOS/Microsoft Edge Dev` | +| Browser Name | Expected Bundle Identifier | Expected Executable | +| --------------- | -------------------------- | ------------------------------------- | +| `chrome` | `com.google.Chrome` | `Contents/MacOS/Google Chrome` | +| `chromium` | `org.chromium.Chromium` | `Contents/MacOS/Chromium` | +| `chrome:canary` | `com.google.Chrome.canary` | `Contents/MacOS/Google Chrome Canary` | For the current list, see [packages/launcher](https://github.com/cypress-io/cypress/blob/develop/packages/launcher/lib/darwin/index.ts) files. -**Note:** `chrome-for-testing` and `chromium` typically require the downloaded application file to be manually moved to the `/Applications` folder to be auto-detected. - #### Linux On Linux, Cypress scans your `PATH` for a number of different binary names. If the browser you are trying to use does not exist under one of the expected binary names, Cypress will not be able to find it. -| Browser Name | Expected Binary Name(s) | -| -------------------- | ---------------------------------------------------- | -| `chrome` | `google-chrome`, `chrome`, or `google-chrome-stable` | -| `chrome:beta` | `google-chrome-beta` | -| `chrome:canary` | `google-chrome-canary` | -| `chrome-for-testing` | `chrome` | -| `chromium` | `chromium-browser` or `chromium` | -| `firefox` | `firefox` | -| `firefox:dev` | `firefox-developer-edition`, `firefox` | -| `firefox:nightly` | `firefox-nightly`, `firefox-trunk` | -| `edge` | `edge`, `microsoft-edge` | -| `edge:beta` | `edge-beta`, `microsoft-edge-beta` | -| `edge:canary` | `edge-canary`, `microsoft-edge-canary` | -| `edge:dev` | `edge-dev`, `microsoft-edge-dev` | +| Browser Name | Expected Binary Name(s) | +| --------------- | ---------------------------------------------------- | +| `chrome` | `google-chrome`, `chrome`, or `google-chrome-stable` | +| `chromium` | `chromium-browser` or `chromium` | +| `chrome:canary` | `google-chrome-canary` | These binary names should work for most Linux distributions. If your distribution packages browsers under a different binary name, you can add a symlink using the expected binary name so that Cypress can detect it. -For example, to create a symlink for the `chrome-for-testing` browser: +For example, if your distribution packages Google Chrome as `chrome`, you could +add a symlink to `google-chrome` like this: ```shell -sudo ln -s /path/to/binary/chrome /usr/local/bin/chrome +sudo ln `which chrome` /usr/local/bin/google-chrome ``` -**Note:** `chrome-for-testing` and `chromium` typically require the binary to be manually added to the `PATH` or a symlink created to be auto-detected. - #### Windows On Windows, Cypress scans the following locations to try to find each browser: -| Browser Name | Expected Executable Path | -| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | -| `chrome` | `C:/Program Files (x86)/Google/Chrome/Application/chrome.exe`, `C:/Program Files/Google/Chrome/Application/chrome.exe` | -| `chrome:beta` | `C:/Program Files (x86)/Google/Chrome Beta/Application/chrome.exe`, `C:/Program Files/Google/Chrome Beta/Application/chrome.exe` | -| `chrome:canary` | `%APPDATA%/Local/Google/Chrome SxS/Application/chrome.exe` | -| `chrome-for-testing` | `C:/Program Files/Google/Chrome for Testing/chrome.exe`, `C:/Program Files (x86)/Google/Chrome for Testing/chrome.exe` | -| `chromium` | `C:/Program Files (x86)/Google/chrome-win32/chrome.exe`, `C:/Program Files/Google/chrome-win/chrome.exe`, `C:/Program Files/Google/Chromium/chrome.exe`, `C:/Program Files (x86)/Google/Chromium/chrome.exe` | -| `firefox` | `C:/Program Files/Mozilla Firefox/firefox.exe`, `C:/Program Files (x86)/Mozilla Firefox/firefox.exe`, `%APPDATA%/Local/Mozilla Firefox/firefox.exe` | -| `firefox:dev` | `C:/Program Files/Firefox Developer Edition/firefox.exe`, `C:/Program Files (x86)/Firefox Developer Edition/firefox.exe`, `%APPDATA%/Local/Firefox Developer Edition/firefox.exe` | -| `firefox:nightly` | `C:/Program Files/Firefox Nightly/firefox.exe`, `C:/Program Files (x86)/Firefox Nightly/firefox.exe`, `%APPDATA%/Local/Firefox Nightly/firefox.exe` | -| `edge` | `C:/Program Files (x86)/Microsoft/Edge/Application/msedge.exe` | -| `edge:beta` | `C:/Program Files (x86)/Microsoft/Edge Beta/Application/msedge.exe` | -| `edge:canary` | `%APPDATA%/Local/Microsoft/Edge SxS/Application/msedge.exe` | -| `edge:dev` | `C:/Program Files (x86)/Microsoft/Edge Dev/Application/msedge.exe` | +| Browser Name | Expected Path | +| --------------- | ------------------------------------------------------------- | +| `chrome` | `C:/Program Files (x86)/Google/Chrome/Application/chrome.exe` | +| `chromium` | `C:/Program Files (x86)/Google/chrome-win32/chrome.exe` | +| `chrome:canary` | `%APPDATA%/../Local/Google/Chrome SxS/Application/chrome.exe` | For the current list, see [packages/launcher](https://github.com/cypress-io/cypress/blob/develop/packages/launcher/lib/windows/index.ts) files. To make a browser installed at a different path be auto-detected, create a -symlink using `mklink` in the location that Cypress expects to find your +symbolic link using `mklink` in the location that Cypress expects to find your browser. -For example, to create a symlink for the `chrome-for-testing` browser: - -```shell -mklink /d "C:\Program Files\Google\Chrome for Testing" "C:\path\to\chrome\executable\folder" -``` - -**Note:** `chrome-for-testing` and `chromium` typically require the downloaded folder to either be moved and renamed to the location mentioned above or a symlink created as described above to be auto-detected. +[Read more about creating symbolic links on Windows](https://www.howtogeek.com/howto/16226/complete-guide-to-symbolic-links-symlinks-on-windows-or-linux/) Occasionally Cypress will have issues detecting the type of browser in Windows environments. To manually detect the browser type, append the browser type to diff --git a/docs/app/tooling/IDE-integration.mdx b/docs/app/tooling/IDE-integration.mdx index 3bab4954b4..17b1d3e5dc 100644 --- a/docs/app/tooling/IDE-integration.mdx +++ b/docs/app/tooling/IDE-integration.mdx @@ -73,8 +73,6 @@ with Cypress. - [Test Utils](https://marketplace.visualstudio.com/items?itemName=chrisbreiding.test-utils): Easily add or remove `.only` and `.skip` modifiers with keyboard shortcuts or the command palette. -- [Cypress Test Explorer](https://marketplace.visualstudio.com/items?itemName=dpanshug.cypress-test-explorer): - Helps you discover, navigate and run Cypress tests directly from the editor. ### IntelliJ Platform diff --git a/docs/app/tooling/code-coverage.mdx b/docs/app/tooling/code-coverage.mdx index e9dc8b858b..93d1c40773 100644 --- a/docs/app/tooling/code-coverage.mdx +++ b/docs/app/tooling/code-coverage.mdx @@ -20,22 +20,24 @@ As you write more and more end-to-end tests, you will find yourself wondering - do I need to write more tests? Are there parts of the application still untested? Are there parts of the application that perhaps are tested too much? -Cypress offers a couple of solutions to help answer these questions: +Cypress offers a couple of solutions to help answer these questions. -- **[Code coverage](#Code-Coverage)** - find out which lines of the application's source code were executed +- **[Code coverage](#Code-Coverage)** - find out which lines of the application's source code were - **[UI coverage](#UI-Coverage)** - find out which parts of the UI were tested + executed ### Code Coverage Code coverage is a metric that helps you understand how much of your application code is covered by your tests. If there are important sections of the application's logic that **were not** executed from the tests, -then a new test can be added to ensure that the application logic for these sections is -also tested. +then a new test can be added to ensure that part of our application logic is +tested. Computing the source code lines that were executed during the test is done through **code coverage**. Code coverage requires inserting additional counters into your source code before running it using **instrumentation**. +. This document will be focused on code coverage and the instrumentation required to set it up. ### UI Coverage @@ -147,7 +149,7 @@ achieving 100% code coverage requires multiple tests. Once the tests finish, the coverage object can be serialized and saved to disk so that a human-friendly report can be generated. The collected coverage -information can also be sent to external services and can help during pull request +information can also be sent to external services and help during pull request reviews. :::info @@ -162,14 +164,14 @@ and Cypress does not instrument your code - you need to do it yourself. The golden standard for JavaScript code instrumentation is the battle-hardened -[Istanbul](https://istanbul.js.org) and, luckily, it plays very nicely with Cypress. -You can instrument the code as a build step through one of two ways: +[Istanbul](https://istanbul.js.org) and, luckily, it plays very nicely with the +Cypress. You can instrument the code as a build step through one of two ways: - Using the [nyc](https://github.com/istanbuljs/nyc) module - a command-line interface for the [Istanbul](https://istanbul.js.org) library - As part of your code transpilation pipeline using the [`babel-plugin-istanbul`](https://github.com/istanbuljs/babel-plugin-istanbul) - tool + tool. ### Using NYC @@ -211,7 +213,7 @@ render( Notice the calls to `cov_18hmhptych.s[0]++` and `cov_18hmhptych.s[1]++` that increment the statement counters. All counters and additional book-keeping -information are stored in a single object attached to the browser's `window` +information is stored in a single object attached to the browser's `window` object. We can see the counters if we serve the `instrumented` folder instead of `src` and open the application. @@ -269,7 +271,7 @@ A really nice feature of both [nyc](https://github.com/istanbuljs/nyc) and [`babel-plugin-istanbul`](https://github.com/istanbuljs/babel-plugin-istanbul) is that the source maps are generated automatically, allowing us to collect code coverage information, but also interact with the original, non-instrumented code -in the Developer Tools. In the screenshot above, the bundle (green arrow) has +in the Developer Tools. In the screenshot above the bundle (green arrow) has coverage counters, but the source mapped files in the green rectangle show the original code. @@ -418,7 +420,7 @@ Notice how the **ADD_TODO** action was executed 3 times - because our test has added 3 todo items, and the **COMPLETE_TODO** action was executed just once - because our test has marked 1 todo item as completed. -The source lines not covered, marked in yellow (the switch cases the test missed) +The source lines not covered marked in yellow (the switch cases the test missed) and red (regular statements) are a great guide for writing more end-to-end tests. We need tests that delete todo items, edit them, mark all of them as completed at once and clear completed items. When we cover every switch @@ -440,7 +442,7 @@ The produced HTML report shows 99% code coverage alt="99 percent code coverage" /> -Every source file but one is covered at 100%. We can have great confidence in our +Every source file but 1 is covered at 100%. We can have great confidence in our application, and safely refactor the code knowing that we have a robust set of end-to-end tests. @@ -458,7 +460,7 @@ you. If you do want to merge the reports yourself: - on every machine running Cypress tests, copy the produced code coverage report into a common folder under a unique name to avoid overwriting it -- after all E2E tests finish, combine the reports yourself using the `nyc merge` +- after all E2E tests finish, combine the reports yourself using `nyc merge` command You can find an example of merging partial reports in our diff --git a/docs/app/tooling/reporters.mdx b/docs/app/tooling/reporters.mdx index eb0046bb78..78b26680a4 100644 --- a/docs/app/tooling/reporters.mdx +++ b/docs/app/tooling/reporters.mdx @@ -21,7 +21,7 @@ Cypress provides several options to review results of a test run. screenshots, videos, and Test Replay in Cypress Cloud. - Cypress App's open source built in and custom reporters. -This document covers how to use [built in](#Built-in-reporters) and [custom](#Custom-reporters) reporters in Cypress App. +This document covers how to use [built in](#Built in reporter) and [custom](#Custom-reporters) reporters in Cypress App. :::tip @@ -41,7 +41,7 @@ By default, Cypress uses the `spec` reporter to output information to `STDOUT`. We've also added the two most common 3rd party reporters for Mocha. These are built into Cypress and you can use them without installing anything. -- [`teamcity`](https://github.com/travisjeffery/mocha-teamcity-reporter) +- [`teamcity`](https://github.com/cypress-io/mocha-teamcity-reporter) - [`junit`](https://github.com/michaelleeallen/mocha-junit-reporter) ## Custom reporters @@ -188,7 +188,7 @@ additional feedback while the tests are running! The solution here is to use multiple reporters. You will have the benefit of both worlds. -We suggest using the npm module [cypress-multi-reporters](https://www.npmjs.com/package/cypress-multi-reporters). +We suggest using the npm module, ### Examples @@ -199,7 +199,7 @@ each spec file. We need to install additional dependencies: -- [`cypress-multi-reporters`](https://github.com/YOU54F/cypress-plugins/tree/master/cypress-multi-reporters): +- [`cypress-multi-reporters`](https://github.com/you54f/cypress-multi-reporters): enables multiple reporters - [`mocha-junit-reporter`](https://github.com/michaelleeallen/mocha-junit-reporter) the actual junit reporter, as we cannot use the `junit` reporter that comes diff --git a/docs/app/tooling/visual-testing.mdx b/docs/app/tooling/visual-testing.mdx index 5904ba43aa..312c3c1da1 100644 --- a/docs/app/tooling/visual-testing.mdx +++ b/docs/app/tooling/visual-testing.mdx @@ -173,13 +173,6 @@ With the Chromatic plugin installed, Chromatic captures an archive of your UI wh See [Chromatic's official docs](https://www.chromatic.com/docs/cypress/setup/?utm_source=cypress_docs) and their [blog](https://www.chromatic.com/blog/how-to-visual-test-with-cypress/?utm_source=cypress_docs). -### Argos - -Argos integrates seamlessly with your existing Cypress setup—configuration, mocking, and tests—to enable visual testing of your application’s UI. -With Argos set up, it captures and analyzes screenshots during your Cypress test runs to detect visual changes. - -Check out [Argos's official documentation](https://argos-ci.com/docs/quickstart/cypress) for setup instructions and best practices. - ### Wopee.io [Wopee.io](https://wopee.io) seamlessly integrates with Cypress to enhance test coverage, expedite maintenance, and ensure more resilient test runs. Aiming for autonomous visual testing, Wopee.io allows you to easily incorporate visual validation into your existing Cypress tests, adding an extra layer of quality assurance. diff --git a/docs/cloud/features/analytics/enterprise-reporting.mdx b/docs/cloud/features/analytics/enterprise-reporting.mdx index 355c29ed27..02b28cc1d9 100644 --- a/docs/cloud/features/analytics/enterprise-reporting.mdx +++ b/docs/cloud/features/analytics/enterprise-reporting.mdx @@ -56,9 +56,8 @@ The type of information and insights available on the various tabs are described These are a few of the questions that can be answered by the cards shown on the **Overall Usage** tab of Enterprise Reporting. Utilize the `Start Date` and `End Date` filters if you need to drill into a specific time period. The `Projects` -filter can be used to filter the data by one or multiple projects. The `Branch` filter can be used to select one or more -branches that exist in your projects. The data on this tab correlates to what Cypress Cloud -[counts as a test result](/cloud/faq#What-counts-as-a-test-result). +filter can be used to filter the data by one or multiple projects. The data on this tab correlates to what +Cypress Cloud [counts as a test result](/cloud/faq#What-counts-as-a-test-result). A couple of things to note: -- Data shown in Enterprise Reporting is not real-time. For aggregated data it is available as of midnight UTC of the current day. - I.e. - data will be populated through "end of day yesterday". However, individual result data is near real-time and is available - 30 minutes after the run has completed. Endpoints that return near real-time data are specified below. +- Data shown in Enterprise Reporting is not real-time. It is available as of midnight UTC of the current day. + I.e. - data will be populated through "end of day yesterday". - Historical data is available as specified by your data retention limit in Cypress Cloud. ## API endpoint details @@ -81,11 +80,6 @@ The following query parameters are available for use: For example, to request that the data be filtered for projects _Project e2e_ and _Phoenix_, use: `&projects=Project%20e2e&projects=Phoenix` -- branch. Optional, and accepts the name of your commit branch. The branch name should be URL encoded. You can pass as many branch - parameters as needed. - - For example, to request that the data be filtered for branches _develop_ and _commitBranch1_, use: `&branch=develop&branch=commitBranch1` - - export_format. Specify the type of data format that will be returned in the response file. Valid options include: `csv`, `json`, `xlsx` ### Project list @@ -428,8 +422,6 @@ This data allows you to extract all spec results regardless of spec result statu returned is 500,000. Make multiple requests with different start_date and end_date parameter values if your volume of records will exceed 500k for a single request. -Data is available near real-time, 30 minutes after the run has completed. - The report_id is: `spec-details` Data returned includes: @@ -463,8 +455,6 @@ This report allows you to extract all test result details for failed tests. The returned is 500,000. Make multiple requests with different start_date and end_date parameter values if your volume of records will exceed 500k for a single request. -Data is available near real-time, 30 minutes after the run has completed. - The report_id is: `failed-test-details` Data returned includes: @@ -482,8 +472,6 @@ Data returned includes: - error_name (string array) - error_message (string array) - test_replay_url -- spec (path and filename) -- test_name (the test name matches what is shown in Cypress Cloud and a '///' delimiter is used between fragments of the test name) Example: https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=failed-test-details&export_format=csv&start_date=2024-01-01 @@ -494,8 +482,6 @@ This report allows you to extract all test result details regardless of test res returned is 500,000. Make multiple requests with different start_date and end_date parameter values if your volume of records will exceed 500k for a single request. -Data is available near real-time, 30 minutes after the run has completed. - The report_id is: `test-details` Data returned includes: @@ -514,8 +500,6 @@ Data returned includes: - error_name (string array) - error_message (string array) - test_replay_url -- spec (path and filename) -- test_name (the test name matches what is shown in Cypress Cloud and a '///' delimiter is used between fragments of the test name) Example: https://cloud.cypress.io/enterprise-reporting/report?token=YOUR-API-KEY&report_id=test-details&export_format=csv&start_date=2024-01-01 @@ -626,8 +610,6 @@ This report allows you to extract all test result details of flaky tests. The ma returned is 500,000. Make multiple requests with different start_date and end_date parameter values if your volume of records will exceed 500k for a single request. -Data is available near real-time, 30 minutes after the run has completed. - The report_id is: `flaky-test-details` Data returned includes: diff --git a/docs/cloud/integrations/slack.mdx b/docs/cloud/integrations/slack.mdx index dce494c192..a2d63d50c0 100644 --- a/docs/cloud/integrations/slack.mdx +++ b/docs/cloud/integrations/slack.mdx @@ -118,24 +118,16 @@ To set notification preferences: 1. Navigate to Slack configuration page 2. Select the notification preferences for each channel -#### Filter notifications by tag or run group +#### Filter notifications by tag When configuring at the project-level, you can filter notifications posted to Slack based on a -[tag](/app/references/command-line#cypress-run-tag-lt-tag-gt) or -[group](/app/references/command-line#cypress-run-group-lt-name-gt) the run was +[tag](/app/references/command-line#cypress-run-tag-lt-tag-gt) the run was recorded with. This can be helpful to only post notifications on certain runs, like those to a staging or release environment. 1. Navigate to the **Slack** configuration page 2. Under **Tags**, select the **Tag** dropdown for the desired integration and - select which tag(s) to filter. If more than one tag is selected the Slack message will be sent - if any of the tags were associated with the run. -3. Under **Run Groups**, select the **Run Group** dropdown for the desired integration and - select which run group(s) to filter. If more than one run group is selected the Slack message will be - sent if any of the run groups were associated with the run. - -If both a tag and run group are selected, only a run associated with both criteria will -trigger a message to be sent to Slack. + select which tag to filter. ### Mute a channel @@ -145,7 +137,8 @@ and resume notifications for a specific channel without losing the configuration you've put in place. 1. Navigate to the **Slack** configuration page -2. Toggle the **Active** selector for each Slack channel you want muted. +2. Under **Actions**, select **Mute** for each Slack channel you want + muted. ### Remove a Slack channel diff --git a/docs/cloud/integrations/teams.mdx b/docs/cloud/integrations/teams.mdx index 59f31e9800..5844967c17 100644 --- a/docs/cloud/integrations/teams.mdx +++ b/docs/cloud/integrations/teams.mdx @@ -1,5 +1,5 @@ --- -title: 'Microsoft Teams Integration' +title: 'Microsoft Teams Integration | Cypress Documentation' description: 'Setup Microsoft Teams notifications for your Cypress Cloud organization and projects.' sidebar_position: 60 sidebar_label: Microsoft Teams @@ -15,6 +15,13 @@ sidebar_label: Microsoft Teams ::: +:::caution + + Microsoft Teams integration is currently in +beta + +::: + The Microsoft Teams integration allows you to see your Cypress test results directly in your Microsoft Teams channels. @@ -217,46 +224,3 @@ needs. You can have Cypress Cloud post run results for a specific project. Once you've configured and added your webhooks, the installation is complete! Cypress Cloud will post run results for **this** project to the specified Microsoft Teams channel. - -## Notification preferences - -By default, Cypress Cloud will post a Teams message to each configured channel -only for failing runs. However, you can select any combination of the followings preferences: - -- **Failed runs**: will notify on runs with a _failed_ status. -- **Passed runs**: will notify on runs with a _passed_ status. -- **Flaky tests**: will notify on runs that have an identified _flaky_ test. - -To set notification preferences: - -1. Navigate to Teams configuration page -2. Select the notification preferences for each channel - -#### Filter notifications by tag or run group - -When configuring at the project-level, you can filter notifications posted to Teams based on a -[tag](/app/references/command-line#cypress-run-tag-lt-tag-gt) or -[group](/app/references/command-line#cypress-run-group-lt-name-gt) the run was -recorded with. This can be helpful to only post notifications on certain runs, -like those to a staging or release environment. - -1. Navigate to the **Teams** configuration page -2. Under **Tags**, select the **Tag** dropdown for the desired integration and - select which tag(s) to filter. If more than one tag is selected the Teams message will be sent - if any of the tags were associated with the run. -3. Under **Run Groups**, select the **Run Group** dropdown for the desired integration and - select which run group(s) to filter. If more than one run group is selected the Teams message will be - sent if any of the run groups were associated with the run. - -If both a tag and run group are selected, only a run associated with both criteria will -trigger a message to be sent to Teams. - -### Mute a channel - -If you want Cypress Cloud to temporarily stop posting Teams messages to a -certain channel, you can **Mute** that channel. This allows you to easily pause -and resume notifications for a specific channel without losing the configuration -you've put in place. - -1. Navigate to the **Teams** configuration page -2. Toggle the **Active** selector for each Teams channel you want muted. diff --git a/docs/partials/_document-domain-workaround.mdx b/docs/partials/_document-domain-workaround.mdx index b1972f97f4..cd677f5064 100644 --- a/docs/partials/_document-domain-workaround.mdx +++ b/docs/partials/_document-domain-workaround.mdx @@ -1,12 +1,24 @@ :::caution - **Disabling document.domain Injection** + **Re-Enabling document.domain Injection** -In some cases, `document.domain` injection may cause issues. As of Cypress -[v12.4.0](https://on.cypress.io/changelog#12-4-0), disabling `document.domain` -injection is available with the `experimentalSkipDomainInjection` option. Please -read about this -[experiment](/app/references/experiments#Experimental-Skip-Domain-Injection) -for more information +As of Cypress [v14.0.0], [`document.domain`](https://developer.mozilla.org/en-US/docs/Web/API/Document/domain) +will no longer be injected into `text/html` pages by default. -::: +This means you must now use `cy.origin()` when navigating between different _origins_ +in the same test. Previously, `cy.origin()` was only necessary when navigating between +different [superdomains](/app/guides/cross-origin-testing#Parts-of-a-URL) in the same test. + +By setting the `injectDocumentDomain` configuration option to `true`, Cypress will +attempt to inject `document.domain` into `text/html` pages. + +A superdomain is comprised of the trailing two elements of the hostname, delimited by +a `.` (period). Given `https://www.cypress.io`, the superdomain is determined to be +`cypress.io`. When this option is enabled, `cy.origin()` is not necessary when navigating +between `https://www.cypress.io` and `https://docs.cypress.io`, but is necessary +when navigating between `https://www.cypress.io` and `https://www.auth0.com`. + +- This will cause issues with certain sites, especially those that use + [origin-keyed agent cluster](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Origin-Agent-Cluster)s. +- This option is deprecated, and will be removed in Cypress 15. + ::: diff --git a/docs/ui-coverage/get-started/introduction.mdx b/docs/ui-coverage/get-started/introduction.mdx index e275d800fe..a96d65e8ae 100644 --- a/docs/ui-coverage/get-started/introduction.mdx +++ b/docs/ui-coverage/get-started/introduction.mdx @@ -12,7 +12,7 @@ Easily track, monitor, and visualize the test coverage of your UI to prevent reg - -## 2. Record a run - -UI Coverage generates reports using [Test Replay](/cloud/features/test-replay) data and requires Cypress v13+. No additional configuration or instrumentation is required to start using UI Coverage. - -[Record a run](/cloud/get-started/setup) to the Cypress Cloud with Test Replay to start using UI Coverage. - -## 3. View your UI Coverage results - -After recording a test run, you can view your UI Coverage results in the Cypress Cloud via the **UI Coverage** tab. - - - -## 4. Customize your results - -UI Coverage provides flexible configuration options to customize your reports. You can configure: - -- [Attribute filters](/ui-coverage/configuration/attributefilters): specify patterns for attributes and their values that should not be used for identifying and grouping elements -- [Element filters](/ui-coverage/configuration/elementfilters): specify selectors for elements that should be excluded from UI Coverage -- [Element groups](/ui-coverage/configuration/elementgroups): groups elements that are related to one another -- [Elements](/ui-coverage/configuration/elements): specify selectors to uniquely identify elements -- [Significant Attributes](/ui-coverage/configuration/significantattributes): prioritize the identification and grouping of specific attributes -- [View filters](/ui-coverage/configuration/viewfilters): specify URL patterns for URLs that should be excluded from UI Coverage. Excluding a URL also excludes all links to that URL. -- [Views](/ui-coverage/configuration/views): specify URL patterns that represent views - -To add or modify the configuration for your project, navigate to the **App Quality** tab in your project settings. - - diff --git a/docs/ui-coverage/results-api.mdx b/docs/ui-coverage/results-api.mdx index e4469f3a2f..22bd00a455 100644 --- a/docs/ui-coverage/results-api.mdx +++ b/docs/ui-coverage/results-api.mdx @@ -55,10 +55,11 @@ getUICoverageResults({ }).then((results) => { const { runNumber, uiCoverageReportUrl, summary, views } = results - console.log( - `Received ${summary.isPartialReport ? 'partial' : ''} results for run #${runNumber}.` - ) - console.log(`See full report at ${uiCoverageReportUrl}.`) + console + .log( + `Received ${summary.isPartialReport ? 'partial' : ''} results for run #${runNumber}.` + ) + .console.log(`See full report at ${uiCoverageReportUrl}.`) // verify project coverage if (summary.coverage < 80) { @@ -206,10 +207,10 @@ env: jobs: run-cypress: - runs-on: ubuntu-24.04 + runs-on: ubuntu-latest steps: - name: Checkout - uses: actions/checkout@v4 + uses: actions/checkout@v3 - name: install run: npm install - name: Run @@ -252,7 +253,7 @@ run-cypress: pipeline { agent { docker { - image 'cypress/base:22.12.0' + image 'cypress/base:latest' } } @@ -323,7 +324,7 @@ version: 2.1 jobs: linux-test: docker: - - image: cypress/base:22.12.0 + - image: cypress/base:latest working_directory: ~/repo steps: diff --git a/src/components/button/index.tsx b/src/components/button/index.tsx index 3c9f02d5d5..c9bf96ade5 100644 --- a/src/components/button/index.tsx +++ b/src/components/button/index.tsx @@ -22,7 +22,7 @@ interface BtnProps { // Build the Button component with the specified props const Btn: React.FC = ({ size = 32, // The size of the button - variant = 'outline-indigo', // The variant of the button + variant = 'outline-purple', // The variant of the button disabled = false, // Whether the button should be disabled target = '_self', // The target of the button link className, // Custom classes for the button diff --git a/src/css/custom.scss b/src/css/custom.scss index af00a65f71..c3ded3fb9e 100644 --- a/src/css/custom.scss +++ b/src/css/custom.scss @@ -117,6 +117,7 @@ --ifm-color-purple-900: theme(colors.purple.900); --ifm-color-purple-1000: theme(colors.purple.1000); + --ifm-font-family-base: ui-sans-serif, system-ui, -apple-system, Segoe UI, Roboto, Helvetica Neue, Arial; --ifm-heading-font-family: Poppins; @@ -135,12 +136,6 @@ // border color in current area of toc --toc-highlight: var(--ifm-color-primary); --ifm-alert-shadow: none; - - // override the default mapping of footer links to the `secondary` color, - // which is not good for dark backgrounds such as the footer - .footer--dark { - --ifm-footer-link-color: theme(colors.gray.50); - } } html[data-theme] { @@ -432,7 +427,6 @@ div[class^='codeBlockTitle'] { [class^='docItemContainer'] [class^='codeBlockLines'] { max-width: 0; - word-spacing: 0; } /* hides the osano widget */ diff --git a/src/css/markdown.scss b/src/css/markdown.scss index 5324d62941..0620b0044f 100644 --- a/src/css/markdown.scss +++ b/src/css/markdown.scss @@ -70,10 +70,10 @@ div.markdown { color: white; } - &.border-indigo-500:hover { - color: var(--ifm-color-indigo-600); + &.border-purple-500:hover { + color: var(--ifm-color-purple-600); - &.bg-indigo-500 { + &.bg-purple-500 { color: white; } } diff --git a/src/data/plugins.json b/src/data/plugins.json index aed12c19b2..fa1827b960 100644 --- a/src/data/plugins.json +++ b/src/data/plugins.json @@ -849,7 +849,7 @@ { "name": "Applitools", "description": "Fast, easy and reliable visual UI testing with Cypress.", - "link": "https://applitools.com/tutorials/sdks/cypress/quickstart/getting-started", + "link": "https://applitools.com/tutorials/cypress.html", "keyword": ["screenshots", "visual regression", "visual-ai"], "badge": "verified" }, @@ -1014,13 +1014,6 @@ "link": "https://github.com/allure-framework/allure-js/tree/main/packages/allure-cypress", "keywords": ["reporter", "allure", "step", "screenshot"], "badge": "community" - }, - { - "name": "cypress-xray-plugin", - "description": "A plugin for uploading Cypress test results to Xray, including evidence such as screenshots, videos or custom data. Fully compatible with Cucumber.", - "link": "https://github.com/Qytera-Gmbh/cypress-xray-plugin", - "keywords": ["reporter", "xray", "jira", "cucumber", "screenshot", "video"], - "badge": "community" } ] }, diff --git a/static/img/accessibility/guides/a11y-standards.png b/static/img/accessibility/guides/a11y-standards.png deleted file mode 100644 index 41e6dec6dc..0000000000 Binary files a/static/img/accessibility/guides/a11y-standards.png and /dev/null differ diff --git a/static/img/accessibility/guides/github-actions-a11y-block.png b/static/img/accessibility/guides/github-actions-a11y-block.png deleted file mode 100644 index 30e671f2c2..0000000000 Binary files a/static/img/accessibility/guides/github-actions-a11y-block.png and /dev/null differ diff --git a/static/img/accessibility/guides/status-check.png b/static/img/accessibility/guides/status-check.png deleted file mode 100644 index 946bbf5523..0000000000 Binary files a/static/img/accessibility/guides/status-check.png and /dev/null differ