diff --git a/docs/accessibility/configuration/_category_.json b/docs/accessibility/configuration/_category_.json index ff6e7ca250..7f8cb5b6d7 100644 --- a/docs/accessibility/configuration/_category_.json +++ b/docs/accessibility/configuration/_category_.json @@ -1,4 +1,4 @@ { "label": "Configuration", - "position": 30 + "position": 40 } diff --git a/docs/accessibility/core-concepts/accessibility-score.mdx b/docs/accessibility/core-concepts/accessibility-score.mdx index 653ac386e6..94a2af21ef 100644 --- a/docs/accessibility/core-concepts/accessibility-score.mdx +++ b/docs/accessibility/core-concepts/accessibility-score.mdx @@ -9,15 +9,17 @@ sidebar_position: 30 -Cypress produces a top-line percentage score that indicates a general sense of accessibility for a project. This overall score for a run is based on the **average score of** **all application snapshots** that were captured during the run, and is rooted in the ratio of passing and failing accessibility checks performed on each snapshot, weighted according to the severity of the issues. +Cypress provides a top-line percentage score to give a general sense of accessibility for your project. This score, calculated for a run, reflects the **average score** of **all application snapshots** captured during the run. It is based on the ratio of passing versus failing accessibility checks for each snapshot, with results weighted by the severity of the issues. -The severity levels presented in Cypress Cloud match the Axe Core® `impact` level for each rule. This impact classification was developed by the creators of Axe, Deque Systems, to aid in prioritization of accessibility results, by highlighting rules that are likely to reveal the most substantial barriers (such as missing label text) so they can be addressed first. +## Severity levels -It's important to note that the Web Content Accessibility Guidelines (WCAG) standards do not have a concept of relative severity of the Success Criteria. For **compliance** purposes, all failures of the Success Criteria count and must be addressed. For **usability** purposes, however, the impact values provided by Axe Core® provide excellent guidance about which issues to take up first to bring the most benefit to disabled users of your platform as quickly as possible. +Severity levels in Cypress Accessibility align with the Axe Core® impact classification. Developed by Deque Systems, this classification helps prioritize accessibility issues by highlighting those likely to create the most substantial barriers for users (such as missing label text) so they can be addressed first. You can learn more about the Axe Core® impact classification in the [Axe Core® documentation](https://github.com/dequelabs/axe-core/blob/develop/doc/issue_impact.md). -## Algorithm +_Note: WCAG standards do not categorize Success Criteria by severity; all failures require resolution for compliance. However, Axe Core® impact levels are helpful for prioritizing fixes to maximize usability improvements quickly._ -The following weights are applied to the accessibility report data: +## Scoring algorithm + +The accessibility score calculation applies the following weights: | Severity of Issue | Weight | | ----------------- | ------ | @@ -26,34 +28,53 @@ The following weights are applied to the accessibility report data: | Moderate | 3 | | Minor | 1 | -In order to create scores for each snapshot, View, and the entire run, we use the following process: +### Score Calculation Process + +1. **Snapshot Score:** + +- Calculate the total weight of passed checks. +- Calculate the total weight of failed checks. +- Compute the score as: `Score = (Passed Weights) / (Passed Weights + Failed Weights)` + +2. **View Score:** + +- Average the scores of all snapshots within the View. + +3. **Run Score:** -- For each Snapshot - - Sum the weights of each passed check - - Sum the weights of each failed check - - Score equals ratio of passed weight over total weight - - [Sum of weights from passed checks] / [Sum of weights from passed checks + Sum of weights from failed checks] -- For each View - - Score equals the average of all Snapshot scores found in the View -- For the Run - - Score equals the average of all Snapshots in the Run +- Average the scores of all snapshots in the run. ## Interpreting your scores -Whether a score is good or bad for you depends on context. There is no single number that can tell you the accessibility of your application. A 99% score may be good for your goals if the score was 90% before and you are making progress, or it might be bad if you previously had no problems. +Accessibility scores should be viewed in the context of your specific project and your goals. There is no single number that can tell you the accessibility of your application. For example: -Even if 1000 checks pass, if there is 1 check that fails, that may happen to have a huge impact on accessibility. For example, if your login page is not accessible, disabled users may not be able to enter your application at all, even if after the login step all the scores are 100%. So even a single issue requires human judgment as to how much of a barrier it represents. +- A score of 99% may indicate progress if the previous score was lower, but a single failure could represent a significant barrier (e.g., an inaccessible login page). +- Even a 100% score does not guarantee full compliance or user-friendliness but indicates that Axe Core® detected no violations in the tested scope. -The most important thing about the score is that it is deterministic and consistent across runs, so that when changes in the application do bubble up to show an increase or decrease in the score, that is going be meaningful, and not noise. +The key value of the score is its consistency across runs. Changes in the score reliably reflect accessibility improvements or regressions. -In general, the ideal state is a clean 100% score in your Cypress Accessibility report, which you can achieve with a combination of fixing existing issues and ignoring things you know you will not fix. +In general, the ideal state is a clean 100% score in your Cypress Accessibility report, which you can achieve with a combination of fixing existing issues and ignoring things you know you won't fix. ## What a 100% score means -If there are no axe-core violations detected in what you test with Cypress, you can be confident that up to 57% of the issues ([source](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/)) that a manual audit would detect have already been addressed. This does not mean you have a fully compliant, accessible, or user-friendly application, but it does mean your team has considered the experience and needs of disabled users in the application's development process. This often means the team is well placed to understand and fix any issues that are reported by users or revealed in manual testing. +A 100% score with Cypress Accessibility means: -Reaching this "Axe-clean" status is a good sign that basic usability needs are likely met for disabled users well enough that they can experience your application and provide you more general usability feedback as needed. +- No Axe Core® violations were detected in the tested scope. +- Up to 57% of issues typically identified in manual audits have been addressed ([source](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/)). +- While not a guarantee of full compliance, it reflects that the team has actively considered accessibility and is well-positioned to address user-reported issues or findings from manual audits. + +Reaching a 100% score, also known as "Axe-clean," signals that the application likely meets basic usability needs for disabled users, providing a foundation for further usability enhancements. ## Improving Accessibility scores -Many violations are easily addressed by HTML and CSS changes that can be made in your application's code. Click the "Learn More" link for any violation to understand the impact of this violation on users and how to fix it. More details about this are available in the [Inspecting Violation Details section](/accessibility/core-concepts/inspecting-violation-details). +### Address Violations + +Most accessibility violations can be resolved through HTML and CSS adjustments. Each violation in Cypress Accessibility includes a "Learn More" link to understand the impact of the issue on users and get step-by-step guidance to fix it. + +Refer to the [Inspecting Violation Details section](/accessibility/core-concepts/inspecting-violation-details) for more information on understanding and addressing violations. + +### Ignore Non-relevant Issues + +For issues you decide not to fix, update your [configuration](/accessibility/configuration/overview) to ignore them. This keeps the score focused on actionable issues while maintaining visibility into ignored elements for transparency. + +By actively addressing violations and maintaining a clean score, your team can ensure consistent progress towards a more accessible application. diff --git a/docs/accessibility/core-concepts/how-it-works.mdx b/docs/accessibility/core-concepts/how-it-works.mdx index 637640ab96..117edd55eb 100644 --- a/docs/accessibility/core-concepts/how-it-works.mdx +++ b/docs/accessibility/core-concepts/how-it-works.mdx @@ -7,19 +7,17 @@ sidebar_position: 40 # How it works -The stages of the Cypress Accessibility process are as follows: +Cypress Accessibility seamlessly integrates accessibility testing into your development pipeline. Below are the stages of the Cypress Accessibility process: -1. As your run is recorded and specs are uploaded to Cypress Cloud, accessibility checks begin, based on the incoming [Test Replay](/cloud/features/test-replay) data, on the Cypress Cloud servers. -1. When your run completes, the final specs are processed for accessibility checks. All earlier specs have already entered processing and are likely completed by the end of the run. -1. Cypress Cloud organizes all the Views (pages and components) that were rendered during the run, merging and deduplicating across tests so that page-level results and live DOM snapshots can be grouped together. -1. A combined "rule-level" report is created. -1. The completed results are displayed in Cypress Cloud and are flagged in integrations such Slack as GitHub/Gitlab comments, and available CI via the Results API so that you can optionally fail your pipeline if your standards are not met. +1. **Recording and Uploading**: As your test run is recorded, specs are uploaded to Cypress Cloud. Accessibility checks begin based on incoming [Test Replay](/cloud/features/test-replay) data on Cypress Cloud servers. +1. **Processing**: When your run completes, the final specs are processed for accessibility checks. Earlier specs have already entered processing and are likely completed by the end of the run. +1. **Organizing Views**: Cypress Cloud groups Views (pages and components) rendered during the run, merging and deduplicating across tests. This ensures page-level results and live DOM snapshots are organized effectively. +1. **Generating Reports**: A combined "rule-level" report is created, summarizing all detected accessibility violations. +1. **Displaying Results**: Results are displayed in Cypress Cloud, flagged in integrations (e.g., Slack, GitHub/GitLab comments), and accessible via the Results API for optional CI pipeline actions (e.g., failing builds based on unmet standards). ## About Axe Core® -Axe Core® is an open source library for detecting accessibility violations. It is used widely in the industry and have over 1 billion total downloads on npm. -The library may be familiar from other tools, such as Lighthouse, or tools used for code linting, mobile testing and more, and has also been used extensively in Cypress tests through open-source packages (discussed more our [accessibility testing guide](/app/guides/accessibility-testing)). This means that Cypress Accessibility's -configuration can often be easily aligned with your existing accessibility standards across different teams and platforms. +Axe Core® is an open-source library by Deque Systems, widely recognized for detecting accessibility violations. With over a billion downloads on npm, it is used in numerous tools at multiple stages of the software development cycle. If you are already using other Axe Core®-based tools, Cypress Accessibility's configuration can be easily aligned with your existing accessibility standards across different teams and platforms. Axe Core® checks for accessibility violations in your website's user interface, based on common accessibility standards. A violation detected by Axe usually indicates that some users with disabilities would have a problem using or perceiving a feature. An example of a violation would be an image without alternative text provided in the markup, which means the content is not perceivable to visually-impaired users. [Learn more about Axe.](https://github.com/dequelabs/axe-core) @@ -31,16 +29,12 @@ The value of this form of testing in Cypress Accessibility is to give you fast, ## Axe Core® updates -Axe Core® publishes new versions several times a year. Cypress will wait at least 30 days after an Axe Core® release before updating the library used in the Cloud. This gives you a known minimum amount of time to see the [changelog](https://github.com/dequelabs/axe-core/blob/develop/CHANGELOG.md) and make any necessary adjustments to your process in advance of Cypress Cloud bumping the version. Bug fixes or new rules in Axe Core® may cause previously-passing projects to fail. +Axe Core® is regularly updated, with Cypress following a controlled adoption process: -We will strive to update Axe Core® in a timely manner after that 30-day buffer, but set no specific expectation around when that will be. Each release is different, and some do not affect the way Axe Core® is executed in Cypress Cloud at all, so those releases may be skipped. - -The [Cypress Accessibility changelog](/accessibility/changelog) will reflect any Axe Core® updates that are made in Cypress Cloud, starting on December 1, 2025 (30 days after the publication of this policy). - -### Exceptions - -In rare cases - such as if an Axe Core® update contains a critical security patch - we may need to update sooner than the 30-day window. If this happens we will communicate with affected customers. +- Update Policy: Cypress Cloud integrates new Axe Core® versions at least 30 days after their release. This provides teams time to review the Cypress Accessibility [changelog](https://github.com/dequelabs/axe-core/blob/develop/CHANGELOG.md) so you can make any necessary adjustments to your process in advance of Cypress Cloud bumping the version. +- Changelog Tracking: Updates are documented in the Cypress Accessibility [changelog](https://github.com/dequelabs/axe-core/blob/develop/CHANGELOG.md), starting December 1, 2024. +- Critical Exceptions: In cases like critical security patches, updates may occur sooner, with notifications provided to affected customers. ## Powered by Test Replay -Because Cypress Accessibility uses data captured through Cypress Test Replay, it is subject [Test Replay limitations](https://docs.cypress.io/faq/questions/cloud-faq#Is-everything-captured-and-replayed-in-Test-Replay). +Cypress Accessibility leverages data captured through Test Replay, ensuring comprehensive analysis while being subject to its [limitations](https://docs.cypress.io/faq/questions/cloud-faq#Is-everything-captured-and-replayed-in-Test-Replay). diff --git a/docs/accessibility/core-concepts/inspecting-violation-details.mdx b/docs/accessibility/core-concepts/inspecting-violation-details.mdx index 9f73c19a80..33897eec36 100644 --- a/docs/accessibility/core-concepts/inspecting-violation-details.mdx +++ b/docs/accessibility/core-concepts/inspecting-violation-details.mdx @@ -9,35 +9,47 @@ sidebar_position: 20 -After clicking through to specific Rule of View from a [Run-level report](/accessibility/core-concepts/run-level-reports), you will see the Violation Detail view. +After navigating to a specific Rule of View from a [Run-level report](/accessibility/core-concepts/run-level-reports), you'll land on the Violation Detail view. -This page breaks the report down into two sections. +This page is divided into two main sections: **Rules** and **Snapshots**. ### Rules -On the left hand side is the main body of the report that lists all the Rules that match the current filters for this View - or for All Views, if no specific View was selected. +The left-hand side of the detail view lists all Rules that match the current filters for the selected View. If no specific View is selected, the list applies to all Views in the run. #### Rule details -Each Rule can be expanded to see its related Elements. The expanded rule also has extra description text and a "Learn more" link that goes to Deque University to explain more about the rule itself, where it comes from, whose experience is affected by it, how to fix issues related to the rule. +Each Rule can be expanded to reveal related elements and additional information: + +- **Description**: Text explaining the rule, its origin, the affected user experiences, and how to fix the issue. +- **Learn More**: A link to Deque University for in-depth guidance. #### Element details -Each element can be clicked to pin an example of that element in the UI and reveal more details about how to address the accessibility issues. There are two buttons associated with each element: +Clicking on an element in the expanded Rule section provides: -- Copy selector - - This is a quick way to grab the CSS selector that uniquely identifies this element in the page or component -- Print to console - - This will log a reference to the element in the JavaScript console and is the fastest way to debug something in devtools. In Chrome-base browsers for example, you can open your console, right click the logged element, and choose "Reveal in elements panel" to get to the exact right part of the live DOM in two clicks, every time, and debug from there. +- **Pinned Element**: Highlights the element in the UI for easier identification. +- **Copy Selector**: A button to copy the CSS selector uniquely identifying the element. +- **Print to Console**: Logs the element reference to the browser's console for debugging. In Chrome-based browsers, you can right-click the logged element and select "Reveal in elements panel" to navigate directly to the live DOM. +- **Share issue**: A button to copy markdown or plain text snippet to share the issue with your team or collaborators. +- **Solutions**: A list of potential solutions to address the issue. + + ### Snapshots -Note: these are not images or screenshots from the tests. They are fully hydrated HTML and CSS snapshots of your application. This makes it easy to see what each violation represents in context, as well as inspect the entire DOM, make adjustments and see how things appear in the browser's accessibility tree. This makes a big difference in understanding and debugging issues, because you don't have to run the application locally to reproduce the state, and with access to the full DOM you can understand even complex issues and identify solutions. +Snapshots are fully hydrated HTML and CSS representations of your application's state during the test. Unlike screenshots or video, these snapshots allow you to: + +- Inspect the DOM and accessibility tree without running the application locally. +- Understand the context of each violation and debug directly from the snapshot. This area also has the Test Replay button to provide access to any tests where this snapshot appeared, as well as the specific URL of the snapshot displayed at the bottom of the screen. diff --git a/docs/accessibility/core-concepts/run-level-reports.mdx b/docs/accessibility/core-concepts/run-level-reports.mdx index 4de8c1f076..ca804ac4ef 100644 --- a/docs/accessibility/core-concepts/run-level-reports.mdx +++ b/docs/accessibility/core-concepts/run-level-reports.mdx @@ -9,7 +9,7 @@ sidebar_label: Run-level reports -In this document we'll review the three main areas to pay attention to when first reviewing an accessibility report for a Cypress run, as well as explain what terms we use and how we define them. +This guide highlights the three main areas to focus on when reviewing an accessibility report for a Cypress run. It also explains key terms and definitions used in these reports. ## Run summary @@ -19,75 +19,85 @@ In this document we'll review the three main areas to pay attention to when firs alt="A heading reads Cypress Accessibility, above some summary metrics that are describe in detail in this document's text." /> -This section includes some top-line numbers to describe the accessibility information for the run at a glance. These numbers show the following details, and clicking any of them will navigate you to the related report details, with the live HTML and CSS snapshots for every issue available to review. +The Run Summary provides top-line metrics for a quick overview of accessibility in the tested run. Clicking any metric navigates to detailed reports, including live DOM snapshots for each issue. -- The overall combined Score for that run. -- The total number of Elements that were found to fail any accessibility Rule -- The total number of Elements with inconclusive accessibility checks in need of manual review -- The total number Elements that were ignored by your configuration, but had either a failed or inconclusive Rule -- A mini chart showing the Axe Core® severity levels of Rules that failed during the run, and how many failed at each severity level -- critical, serious, moderate, and minor -- The total count of failed Rules +**Key metrics include:** -All of these terms are defined in detail below. +- **Overall Score**: A combined accessibility score for the entire run. +- **Failed Elements**: Total elements failing at least one accessibility rule. +- **Inconclusive Elements**: Elements requiring manual review due to inconclusive accessibility checks. +- **Ignored Elements**: Elements ignored by your configuration but with failed or inconclusive checks. +- **Rule Severity Chart**: A mini chart showing the Axe Core® severity levels (critical, serious, moderate, minor) for failed rules. +- **Total Failed Rules**: The cumulative count of failed rules. + +Each term is defined further below for clarity. ## Views -A View is a collection of snapshots from the application(s) tested in your recorded run. Views represent pages or components in your app. These snapshots are either grouped under their URLs if they came from a full page, or under a component spec path if they were rendered with Cypress Component Testing. View grouping behavior is highly configurable, see the Views config. +A View represents a collection of snapshots from the tested application, grouped by pages (via URLs) or components (via Component Testing paths). Views are configurable and provide structured insights into the application. -### Table data - -#### Snapshots +### Snapshots -The count of snapshots captured for the View. A Snapshot is a live HTML and CSS recreation of your application at a single moment in time. A View will usually have more than one Snapshot, which are collected as tests render variants of the page and modify state by interacting with elements, or as dynamic content is loaded and modified by the application itself. +The number of snapshots captured for a View. A Snapshot represents the live HTML and CSS state of the application at a specific moment during a recorded test. Multiple snapshots are captured as tests interact with elements, modify state, or load dynamic content. -#### Score +### Score -An overall score is calculated based on all the Snapshots for a given View. The score for each snapshot is a ratio of passed checks to failed checks, weighted according to the impact of the issues. This produces a top-line percentage score that indicates a general sense of accessibility for a project. For more details, see the [Accessibility Score section](/accessibility/core-concepts/accessibility-score). +The overall score for a View is a weighted ratio of passed versus failed checks across all snapshots, reflecting the general accessibility level for that segment of the application. This produces a top-line percentage score that indicates a general sense of accessibility for a project. Learn more in the [Accessibility Score section](/accessibility/core-concepts/accessibility-score).. -#### Elements +### Elements -The count of elements in a view that failed a Rule, had an inconclusive check for a Rule, or were explicitly ignored by configuration. +The count of elements in a View categorized as: -- **Failed** elements had at least one accessibility Rule fail during the run. These are considered in your accessibility Score and must be fixed. -- **Inconclusive** elements had at least one check marked "incomplete" by Axe Core® during the run. These represent checks that couldn't be completed for technical reasons, or items that need manual review to confirm if there is an accessibility issue. These do not affect your score. -- **Ignored** elements are those that had either failing or incomplete checks, but were ignored with [CSS selectors in your configuration](/accessibility/configuration/elementfilters). They do not affect your score. Cypress still processes these elements and makes the results available, so that you always know results were was ignored, since these elements are still present in your UI and users will encounter them. +- **Failed:** Elements violating at least one rule. These affect your accessibility score. +- **Inconclusive:** Elements had at least one check marked "incomplete" by Axe Core® during the run. These represent checks that couldn't be completed for technical reasons, or items that need manual review to confirm if there is an accessibility issue. These do not affect your accessibility score. +- **Ignored:** Elements with failed or incomplete checks that were excluded via your [configuration](/accessibility/configuration/elementfilters) but still processed. They do not affect your accessibility score. -#### Failed rules +### Failed rules -Failed rules are broken down for each View with a mini chart displaying Axe Core® severity levels, as well as a total count for failed rules. Both of these help you identify at a glance which pages or components have high numbers serious issues. +Failed rules are displayed with severity charts (critical, serious, moderate, minor) and total counts, enabling quick identification of areas with the most significant issues. -#### Severity +### Severity -In Cypress Accessibility, severity levels are mapped to Axe Core's `impact` level for each rule. It's important to note that these levels were created by Deque Systems, the creators of Axe Core, and not a part of the Web Content Accessibility Guidelines themselves. For more details on this, see our [accessibility score documentation](https://github.com/dequelabs/axe-core/blob/develop/doc/issue_impact.md). +Severity levels correspond to Axe Core® impact levels for each rule, providing context on the criticality of detected issues. It's important to note that these levels were created by Deque Systems, the creators of Axe Core, and not a part of the Web Content Accessibility Guidelines themselves. Learn more in the [Axe Core® documentation](https://github.com/dequelabs/axe-core/blob/develop/doc/issue_impact.md). ## Rules -This section lists the outcome of every Axe Core® rule that was enabled at the time of this run. +This section lists all Axe Core® rules enabled during the run, detailing their outcomes. -### Table data +Attributes for each rule include: -Each Rule in the table has the following attributes: - -- An icon representing the overall status for the rule, which can be any one of these: - - **Passed** -- every time this rule applied to an element, the element either passed that rule's checks or was intentionally ignored - - **Failed** -- at least one failing element was detected in the run - - **Inconclusive** -- no elements failed this rule, but at least one element was labelled "incomplete" by Axe Core® and requires manual review - - **Inapplicable** -- this rule was enabled during the run, but there were no elements detected that required the rule to run - - **Ignored by config** -- this rule was ignored at the config level for the project and did not run -- The description for the rule -- The Axe Core® severity level for the rule -- The counts of failed, inconclusive, and ignored elements for each rule +- **Status Icon**: An icon representing the overall status for the rule, which can be any one of these: + - **Passed**: All elements passed checks or were ignored. + - **Failed**: At least one element failed. + - **Inconclusive**: At least one element was labelled "incomplete" by Axe Core® and requires manual review, but no elements were failed. + - **Inapplicable**: No applicable elements detected for this rule. + - **Ignored by config**: The rule was excluded at the project level. +- **Description**: A summary of the rule. +- **Severity**: The Axe Core® severity level for the rule. +- **Counts**: The counts of failed, inconclusive, and ignored elements for the rule. ## Filtering -The Views and Rules for your accessibility reports can be filtered but WCAG conformance level, severity, and the status of the related content. This allows you to get a full understanding of everything that Cypress saw and evaluated during the run, as well as the effects of any configuration that was applied, but also drill into see something like "only components where critical rules are failing". +You can filter Views and Rules in accessibility reports by: + +- **Rule severity**: Critical, serious, moderate, or minor. +- **Rules**: WCAG 2.1 A, WCAG 2.1AA, or Best practices. +- **Element status**: Failed, inconclusive, or other (other includes ignored elements and rules, 100% passing views and rules, and inapplicable rules). +- **Name**: Search for specific URLs or paths to filter the views or specific names to filter rules. + +This allows a full understanding of what Cypress evaluated during the run, while enabling targeted analysis of critical issues. + + diff --git a/docs/accessibility/get-started/introduction.mdx b/docs/accessibility/get-started/introduction.mdx index e4e1f1ae81..ed05901825 100644 --- a/docs/accessibility/get-started/introduction.mdx +++ b/docs/accessibility/get-started/introduction.mdx @@ -44,34 +44,104 @@ From there, you can integrate with CI to set your own standards for handling the [Get started with Cypress Accessibility ➜](/accessibility/get-started/setup) -## How it Works - -Cypress Accessibility provides organized, visual, and actionable accessibility reports, based completely on the tests you record to Cypress Cloud, and powered by Axe Core® by Deque Systems. This is the documentation page for that product. To learn about the general topic of accessibility testing using the Cypress App, see [our guide on this topic](/app/guides/accessibility-testing). - -Cypress Accessibility generates a sortable, filterable report, to see scores and violation counts by page/component, or a combined rule-based report for the entire run. It also includes a drill-in view with live DOM snapshots highlighting each issue detected. - -:::caution - -Cypress Accessibility generates reports using [Cypress Test Replay](/cloud/features/test-replay) data and requires Cypress v13+. +## Top Guides + + -::: - -Cypress Cloud generates an accessibility report for each unique state reached during your tests on a given Cypress run. This includes pages visited in end-to-end tests, as well as individual components rendered in component tests, and any states or pages that were encountered as a result of interactions performed during test execution. - -All of these page or component states are detected using the same protocol that powers [Test Replay](/cloud/features/test-replay), meaning no additional code or setup is needed anywhere in your tests. - -These individual accessibility "snapshots" are grouped into page- or component-level reports (called "Views"), and also form the bases of a unified report covering the entire run. This gives you multiple ways to examine your report - by focusing on the most important parts of your application, or by +## How it Works -Reports are generated using the industry-standard open source [Axe Core®](https://github.com/dequelabs/axe-core) library created by Deque Systems, with a default configuration focused on the [Web Content Accessibility Guidelines (WCAG) 2.1](https://www.w3.org/WAI/standards-guidelines/wcag/). +Cypress Accessibility delivers detailed, actionable accessibility reports directly from your tests, powered by the industry-leading Axe Core® library by Deque Systems. These reports require no extra setup and are automatically generated for every unique state reached during your Cypress tests, whether in end-to-end or component testing. -## Highlights +- **Effortless Setup**: Cypress Accessibility utilizes the same capture protocol as [Test Replay](/cloud/features/test-replay) (requires Cypress v13+), so no additional code or configuration is needed. +- **Performance-Friendly**: Server-side execution ensures no impact on test performance or existing code. +- **Comprehensive Reporting**: During test execution, unique states—defined by distinct DOM structures—are recorded for end-to-end views and components. +- **DOM Snapshots**: Each detected violation is accompanied by a full-page, inspectable DOM snapshot, highlighting the exact location and context of the violation. +- **Rule Compliance**: Violations are mapped directly to WCAG 2.1 criteria, making it easier to prioritize and remediate issues. +- **Report Generation**: [Detailed reports](/accessibility/core-concepts/run-level-reports) are generated, offering sortable and filterable views by page, component, or the entire test run. Rule-based summaries for an aggregated view of accessibility outcomes for the entire run are also available. +- **Selective Element Handling**: Elements marked as "ignored" in tests are excluded from scoring but remain visible for manual review, ensuring flexibility via [configuration](/accessibility/configuration/overview). +- **Configurable CI Integration**: The [Results API](/accessibility/results-api) allows you to programmatically control your CI pipeline's behavior based on accessibility outcomes, enabling enforcement of accessibility policies. -- All steps of all user flows tested in Cypress are captured automatically -- Server-side execution means no disruption to existing test code or practices, and no performance impact from accessibility checks -- Reports are generated at the page or component level, as well as a combined report showing the outcome of all accessibility rules checked in the run -- Debuggable full-page DOM snapshots with highlights are provided for every violation -- Most iframe and Shadow DOM uses cases are supported out-of-the-box -- The [Results API](/accessibility/results-api) allows you to own how your CI pipeline reacts to the accessibility results of the run to implement a policy -- "Ignored" elements are isolated and don't affect your score, but can still be reviewed +Learn more about [how it works](/accessibility/core-concepts/how-it-works). _Note: DEQUE, DEQUELABS, AXE®, and AXE-CORE® are trademarks of Deque Systems, Inc. in the US and other countries. These terms are used throughout our documentation to refer to the accessibility testing library used by Cypress Accessibility._ diff --git a/docs/accessibility/guides/_category_.json b/docs/accessibility/guides/_category_.json index 371eacc4b7..c9532523a7 100644 --- a/docs/accessibility/guides/_category_.json +++ b/docs/accessibility/guides/_category_.json @@ -1,4 +1,4 @@ { "label": "Guides", - "position": 70 + "position": 30 } diff --git a/docs/accessibility/guides/accessibility-automation.mdx b/docs/accessibility/guides/accessibility-automation.mdx index c1b6565503..83cf4d75fe 100644 --- a/docs/accessibility/guides/accessibility-automation.mdx +++ b/docs/accessibility/guides/accessibility-automation.mdx @@ -1,46 +1,40 @@ --- -title: Accessibility automation principles +sidebar_label: Accessibility automation principles +title: 'Accessibility automation principles | Cypress Accessibility Documentation' +description: " +Apply these core principles to maximize the impact of Cypress Accessibility’s automation: complement automation with human judgment for deeper insights, manage false positives effectively, and ensure robust assistive technology support." sidebar_position: 80 --- -This guide describes some general ideas that are relevant to Cypress Accessibility, but are also somewhat shared among accessibility automation tools of any kind. +# Accessibility automation principles -## What it means to fix all Axe Core® violations {#what-it-means-to-fix-all-axe-core-violations} +This guide outlines foundational concepts for using Cypress Accessibility, which also apply to accessibility automation tools in general. -Cypress Accessibility runs Axe Core® checks on every significant state and variation of the applications you test with Cypress. This kind of automation can catch up to [57% of issues that would appear in a manual audit](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/), but it does not replace the human judgment that is applied during an audit. Auditors test a wide variety of scenarios and technologies in a way that would be impossible to fully automate. +## Levels of Accessibility -It helps to think of accessibility at three levels. The first is “basic usability”. Paying attention to issues reported by Cypress and Axe Core® should help you to this point. This is the threshold at which a disabled user is likely to be able to make substantial progress through the application, but may still hit major barriers or usability issues. If your contact forms and issue reporting systems are accessible, you will be able to learn about these problems from actual users. +Cypress Accessibility runs Axe Core® checks on every significant state and variation of the applications you test with Cypress. This kind of automation can catch up to [57% of issues that would appear in a manual audit](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/), but it does not replace the human judgment that is applied during an audit. Auditors test a wide variety of scenarios and technologies in a way that would be impossible to fully automate. -The second level of accessibility is full conformance with all Web Content Accessibility Guidelines (WCAG) success criteria. As mentioned above, more than half of issues reported in manual audits can be reported and caught early with Axe Core ®automation. Conformance with the WCAG is often the standard set by legislation and regulation related to web accessibility. If your goal is regulatory compliance, this is the bar. No generic automation tool can “prove” compliance with this standard, it **requires** human assessment. +Accessibility can be thought of as three progressive levels: -The third level, and the true goal of many accessibility programs, is a good user experience. This needs to be validated by actual disabled users who are daily users of assistive technology. It’s entirely possible to have a website that passes the first two levels, and still offers a poor user experience. +- **Basic Usability**: This is the threshold at which a disabled user is likely to be able to make substantial progress through the application, but may still hit major barriers or usability issues. If your contact forms and issue reporting systems are accessible, you will be able to learn about these problems from actual users. Paying attention to issues reported by Cypress and Axe Core® should help you to this point. +- **WCAG Conformance**: Automated tools like Axe Core® catch up to [57% of accessibility issues that would appear in a manual audit](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/), but human assessment is essential for validating conformance. Conformance with the WCAG is often the standard set by legislation and regulation related to web accessibility. If your goal is regulatory compliance, this is the bar. No generic automation tool can "prove" compliance with this standard, it **requires** human assessment. +- **Good User Experience**: This needs to be validated by actual disabled users who are daily users of assistive technology. It's entirely possible to have a website that passes the first two levels, and still offers a poor user experience. -Accessibility is about communicating the nature and structure of the application, and the available actions and information, in a way that allows disabled users to independently understand the interface and complete all tasks. The foundational [POUR principles](https://www.24a11y.com/2019/pour-the-foundation-first/) of the WCAG \- that software must be Perceivable, Operable, Understandable and Robust \- are bigger than any specific set of test plans or static checks. While Axe Core® tell us about many genuine accessibility barriers, it is important to leverage that information into making good decisions about accessibility and providing a truly equal experience. +Accessibility is about communicating the nature and structure of the application, and the available actions and information, in a way that allows disabled users to independently understand the interface and complete all tasks. The [POUR principles](https://www.24a11y.com/2019/pour-the-foundation-first/) (Perceivable, Operable, Understandable, Robust) form the foundation for effective accessibility beyond specific tests or checks. While Axe Core® tells us about many genuine accessibility barriers, it's important to leverage that information into making good decisions about accessibility and providing a truly equal experience. ## False positives -False positives are always a risk in any automated scans. Both Cypress Accessibility and Axe Core® are designed in a way that minimizes them, but it’s always possible that there will be some violations in the report that you recognize as invalid and would rather exclude, or are genuinely incorrect. - -While false positives are rare, if in doubt, use the “provide feedback” button to alert us to something you see as a false positive and we can work on diagnosing and solving the issue from our side. - -### Genuine false positives - -There are two main possible sources of genuine false positive results: - -1. The check was run on an **invalid** **state** \- for example a partially hydrated DOM \- that no user would ever see. These are scenarios that Cypress can account for in the product if they are reported. -2. The state tested was correct but Axe Core® reported a violation that it shouldn’t have, as a result of a bug or missing check. Since Axe Core® is an open-source product, if and when we encounter bugs, we can open issues in the project and even make pull requests if needed. - -### Loading states - -There’s another scenario that can present as a potential false positive: legitimate user-facing loading states. These are situations where some data is pending in your application. +Cypress Accessibility and Axe Core® attempt to minimize false positives, but they can still occur. Some example of false positives include: -It might be tempting to say “this state shouldn’t be captured, it’s not fully loaded” \- but in reality, these types of “pending” states are valid, and often contain visual information like loading spinners, UI skeletons, and progress bars which should be accessible themselves, so the user knows what data is loading. It’s also possible that the data doesn’t come in at all, or is very slow, and a user needs to figure out what is happening and what they can do, so the application needs to remain in a fully accessible, usable condition even if data is pending. +- **Invalid States**: False positives may arise from testing a partially hydrated DOM or other states users would never encounter. You can report invalid states by using the 'Provide Feedback' button in the Cypress UI and we'll continuously improve the product to account for these scenarios. +- **Incorrect Violations**: Axe Core® may report violations due to bugs or limitations in its rules. Since Axe Core® is an open-source product, if and when we encounter bugs, we can open issues in the project and even make pull requests if needed. -In general it’s a good idea in front-end development to “make impossible states impossible”, and in doing so you can follow the accessibility principle of Robustness (the last of the 4 POUR principles mentioned above), while also making it much less likely that an invalid state is possible to reach in Cypress in the first place. This can also have a beneficial effect on the overall stability of your tests. +## Commonly mistaken as false positives -### Assistive technology support +Some issues that are not false positives, but may seem like them, include: -Certain accessibility techniques are not fully supported by all assistive technologies like screen readers or voice control software. Axe Core® accounts for this. The software and operating systems tested against are listed in their [Accessibility Supported document](https://github.com/dequelabs/axe-core/blob/develop/doc/accessibility-supported.md). For example: it can seem like a false positive from Axe Core® if a failing technique used in your code appears to work correctly in a certain screen reader, browser and operating system combination that you test with. But if it does not have sufficient accessibility support in commonly used technology configurations, Axe Core® will intentionally fail that technique, allowing you to switch to something with better support. +- **Loading States**: Legitimate user-facing loading states can present as potential false positives. They inform users about pending data and provide fallback functionality if data is slow to load. It's a good idea in front-end development to "make impossible states impossible", and in doing so you can follow the accessibility principle of Robustness. +- **Assistive Technology Support**: Certain accessibility techniques are not fully supported by all assistive technologies like screen readers or voice control software. Axe Core® accounts for this. The software and operating systems tested against are listed in their [Accessibility Supported document](https://github.com/dequelabs/axe-core/blob/develop/doc/accessibility-supported.md). For example: it can seem like a false positive from Axe Core® if a failing technique used in your code appears to work correctly in a certain screen reader, browser and operating system combination that you test with. But if it does not have sufficient accessibility support in commonly used technology configurations, Axe Core® will intentionally fail that technique, allowing you to switch to something with better support. ## Summary diff --git a/docs/accessibility/guides/block-pull-requests.mdx b/docs/accessibility/guides/block-pull-requests.mdx new file mode 100644 index 0000000000..ec608e23f4 --- /dev/null +++ b/docs/accessibility/guides/block-pull-requests.mdx @@ -0,0 +1,63 @@ +--- +sidebar_label: Block pull requests and set policies +title: 'Block pull requests and set policies | Cypress Accessibility Documentation' +description: "Set policies and block pull requests automatically with Cypress Accessibility's Results API, enabling custom CI workflows to enforce accessibility standards and prevent regressions." +sidebar_position: 40 +--- + +# Block pull requests and set policies + +Cypress Accessibility reports are generated server-side in Cypress Cloud, based on test artifacts uploaded during execution. This ensures there is no performance impact on your Cypress test runs. + +## Using the Results API + +The [Cypress Accessibility Results API](/accessibility/results-api) allows you to access accessibility results post-test run, enabling workflows like blocking pull requests or triggering alerts based on specific accessibility criteria. This involves adding a dedicated accessibility verification step to your CI pipeline. With a Cypress helper function, you can automatically fetch the report for the relevant test run within the CI build context. Below is an example of how to block a pull request in GitHub Actions if accessibility violations occur: + + + +## Implementing a status check + +The Results API offers full flexibility to analyze results and take tailored actions. It can also integrate with status checks on pull requests. Below is an example of a passing accessibility verification step: + + + +## Defining policies in the verification step + +The [Results API Documentation](/accessibility/results-api) provides detailed guidance on the API's capabilities. Here's a simplified example demonstrating how to enforce a "no new failing accessibility rules" policy: + +```js +const { getAccessibilityResults } = require('@cypress/extract-cloud-results') + +// List of known failing rules +const rulesWithExistingViolations = [ + 'button-name', + // Add more rules as needed +] + +// Fetch accessibility results +getAccessibilityResults().then((results) => { + // Identify new rule violations + const newRuleViolations = results.rules.filter((rule) => { + return !rulesWithExistingViolations.includes(rule.name) + }) + + if (newRuleViolations.length > 0) { + // Trigger appropriate action if new violations are detected + throw new Error( + `${newRuleViolations.length} new rule violations detected. These must be resolved.` + ) + } +}) +``` + +By examining the results and customizing your response, you gain maximum control over how to handle accessibility violations. Leverage CI environment context, such as tags, to fine-tune responses to specific accessibility outcomes. diff --git a/docs/accessibility/guides/blocking-pull-requests.mdx b/docs/accessibility/guides/blocking-pull-requests.mdx deleted file mode 100644 index 2b4fd92b51..0000000000 --- a/docs/accessibility/guides/blocking-pull-requests.mdx +++ /dev/null @@ -1,60 +0,0 @@ ---- -title: Blocking pull requests and setting policies -sidebar_position: 40 ---- - -Cypress Accessibility reports are generated on the server in Cypress Cloud, based on artifacts uploaded by the test runs. Because of this, there is no impact to your actual Cypress test execution, in terms of performance or anything else - including failing the Cypress run due to accessibility violations. - -## Using the Results API - -The [Cypress Accessibility Results API](/accessibility/results-api) provides access to accessibility results after the run is completed, so that you can implement patterns like blocking a pull request or sending out alerts based on specific accessibility criteria. This is done by adding a separate accessibility verification step to your CI pipeline, and using a helper function provided by Cypress to automatically request the report for the appropriate Cypress run based on the build context. The example below shows how this can look in GitHub Actions when there is an accessibility failure based on your own chosen policy: - - - -## Implementing a status check - -The API gives you full flexibility to examine the results and react to it in whatever way you need to. It also provides a signal that can be used in status checks on Pull Requests. Here is an example where the accessibility verification step is passing: - - - -## Implementing a policy in the verification step - -The [Results API Documentation](/accessibility/results-api) describes the capabilities of the API in more detail, but here is a simplified example of how to use the API to implement the "no new failing accessibility rules" policy shown above. - -```js -const { getAccessibilityResults } = require('@cypress/extract-cloud-results') - -// define your known failing rules -const rulesWithExistingViolations = [ - 'button-name', - ... -] - -// call the API -getAccessibilityResults() - .then((results) => { - // compare the accessibility results with your acceptable standard - const newRuleViolations = results.rules.filter((rule) => { - return !rulesWithExistingViolations.includes(rule.name) - }) - - if (newRuleViolations.length > 0) { - // based on the standard, trigger any side effect you like, such as failing the step - throw new Error( - `${newRuleViolations.length} rule regressions were introduced and must be fixed.` - ) - } -}) -``` - -This process of examining the results and then deciding the appropriate response gives you the most flexibility in terms of how exactly you want to force a stopping point or specific reaction based on the contents. You have access to the results as well as any context needed from the CI environment, such as tags, to implement the correct response to any specific result values. diff --git a/docs/accessibility/guides/improve-accessibility.mdx b/docs/accessibility/guides/improve-accessibility.mdx new file mode 100644 index 0000000000..7e870b74f4 --- /dev/null +++ b/docs/accessibility/guides/improve-accessibility.mdx @@ -0,0 +1,74 @@ +--- +sidebar_label: Improve accessibility +title: 'Improve Accessibility | Cypress Accessibility Documentation' +description: 'Discover how to break down accessibility reports, prioritize fixes, and make impactful progress toward accessible software with Cypress Accessibility.' +sidebar_position: 20 +--- + +# Improve accessibility + +This guide explains how to break down accessibility reports and create a focused plan for remediation using Cypress Accessibility. + + + +## Starting from Scratch + +If you're beginning with a legacy application that has many accessibility issues, Cypress Accessibility provides actionable insights to help you prioritize fixes. Use the [Accessibility tab for a run](/accessibility/core-concepts/run-level-reports) to identify violations across pages and components. Although the volume of issues might seem overwhelming, focusing on key factors can guide your approach: + +- **Goals**: Define your accessibility objectives. +- **Timeline**: Establish a realistic timeline for improvements. +- **Resources**: Assess the resources available for remediation. +- **Future Code Changes**: Focus on areas of the codebase likely to evolve. +- **Risk Assessment**: Consider the risks and costs of remaining inaccessible. + +Effective remediation often requires collaboration across departments, emphasizing the importance of an organization-wide commitment to accessibility, so understanding your organization's stance on accessibility and who needs to be in the conversation is also helpful. + +If your organization or team is newer to accessibility, you're likely to find plenty of issues reported by Axe Core® in Cypress Accessibility. Many of the issues in your first report may relate to concepts that are new, such as [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) or [invalid nesting of interactive controls](https://dequeuniversity.com/rules/axe/4.2/nested-interactive). Since it will be impossible to fix all issues overnight, we recommend finding some low-hanging fruit as a starting point. This can help your team build confidence by getting some wins on the board, and teach them the overall idea of how accessibility problems get fixed. + +## Good low-hanging fruit + +Some accessibility issues are easier to address but have a significant impact. Consider issues related to [WCAG SC 4.2.1 \- Name, role, value](https://www.w3.org/WAI/WCAG21/Understanding/name-role-value.html), which ensures interactive controls are appropriately labeled. Violations in this category often: + +- Appear frequently in applications. +- Are flagged as Critical or Serious by Axe Core®. +- Directly impact usability for screen reader users. + +### Example: "Buttons Must Have Discernible Text" + +A common issue in modern interfaces is unlabeled icons used as buttons. The [button-name rule](https://dequeuniversity.com/rules/axe/4.7/button-name), which ensures buttons have descriptive text, is an ideal starting point because: + +- Adding a label is a simple code update. +- Proper labels improve usability for all users. +- Fixes typically don't require cross-team involvement. +- Straightforward to validate with a screen reader. + +Existing code failing the button-name rule can usually be remediated pretty quickly, as long as it is clear what the correct text label is for a given control. And when it comes to new code, this text label content is also easy to add to discussions in the design and requirements stages of the development cycle, shifting the accessibility conversation left. + +Since every violation in Cypress is linked to a Deque University page with details about the nature of the problem and how to fix it, all the needed context is available, even to engineers who may be new to the topic of accessibility. + +Even if your first rule is not "Buttons must have discernible text", we suggest finding one with a similarly small amount of coordination or understanding needed to fix, so the team can experience success as quickly as possible. + +## Setting achievable goals + +Define a clear objective for each effort, such as fixing all violations of a specific rule within a defined scope. For example: + +- **View Focused**: Address all rule violations on a single page or component. +- **Rule Focused**: Resolve a single rule violation across multiple pages or the entire application. + +Achieving these milestones provides a clear finish line for you and your team. It also provides time to assess the impact of the changes. Often, fixing one issue can cascade changes across your application, especially in componentized architectures: it's fine for these first goals to be small at first. In fact, it's preferable. + +## Iterating towards success {#iterating-towards-success} + +Incremental progress is key to long-term accessibility improvements. By focusing on manageable goals, your team can: + +- Gain expertise in specific accessibility rules. +- Build workflows for preventing future regressions. +- Gradually raise the accessibility standard across your projects. + +With support from Deque University's "Learn More" links in the reports, even teams new to accessibility can learn the related context and confidently address violations. + +For guidance on preventing regressions and enforcing accessibility policies, explore our [Blocking Pull Requests Section](/accessibility/guides/block-pull-requests). diff --git a/docs/accessibility/guides/improving-accessibility.mdx b/docs/accessibility/guides/improving-accessibility.mdx deleted file mode 100644 index ecbf2b27c3..0000000000 --- a/docs/accessibility/guides/improving-accessibility.mdx +++ /dev/null @@ -1,60 +0,0 @@ ---- -title: Improving accessibility -sidebar_position: 20 ---- - -In this section we'll discuss how to break down the data in your accessibility report and make a plan for remediation. - - - -## Starting from Scratch - -When you first start to use Cypress Accessibility, you may be testing a legacy application with many existing accessibility violations. The [Accessibility tab for a run](/accessibility/core-concepts/run-level-reports) can show multiple violations detected on dozens or hundreds of pages in your application. The number of failing elements and page or component snapshots available might be overwhelming, making it difficult to know where to start. - -In this situation, the first step is to understand some of the key variables and how they are related within your organization. The kinds of things to consider are: - -- your goals for the accessibility of the application -- your timeline for making it accessible -- the available resources for remediation -- what code is likely to change in the near future, where accessibility can be included in designs and requirements -- the consequences, risks, and costs of remaining inaccessible - -These factors will help you find the right strategy for engaging with the results displayed in your accessibility reports. While incremental accessibility efforts by designers and engineers are never wasted, the best outcomes come from an overall organizational commitment to ship accessible software, which requires investment from all departments, so understanding your organization's stance on accessibility and who needs to be in the conversation is also helpful. - -If your organization or team is newer to accessibility, you are likely to find plenty of issues reported by Axe Core® in Cypress Cloud. Many of the issues in your first report may relate to concepts that are new, such as [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) or [invalid nesting of interactive controls](https://dequeuniversity.com/rules/axe/4.2/nested-interactive). Since it will be impossible to fix all issues overnight, we recommend finding some low-hanging fruit as a starting point. This can help your team build confidence by getting some wins on the board, and teach them the overall idea of how accessibility problems get fixed. - -### Good low-hanging fruit - -Some of the easiest, and most valuable issues to tackle first come from [WCAG SC 4.2.1 \- Name, role, value](https://www.w3.org/WAI/WCAG21/Understanding/name-role-value.html). This is the part of the Web Content Accessibility Guidelines that’s concerned with making sure interactive controls have appropriate labels for all users. In general, your app is likely to have some violations in this category, and Axe-Core® tends to flag these as Critical or Serious, because the lack of a label usually makes it difficult or impossible for some users to know what something does in a user interface. - -Within the “Name, role, value” success criterion, the [`button-name`](https://dequeuniversity.com/rules/axe/4.7/button-name) rule in Axe Core® (“Buttons must have discernible text”) is a great place to start. Icons being used as buttons are common in modern user interfaces and they are often missing descriptive text content for screen reader users, which directly impacts their ability to understand the purpose and effect of the button and independently complete tasks in your application. - -This rule, and others like it, are good starting points because the rule is easy to understand and easy to fix. Since the solution is purely a code update to add the missing label text to buttons, this category of rule requires little or no coordination across departments or teams to address. These types of issues are also straightforward to demonstrate and validate even for beginners with a screen reader. - -Existing code failing the button-name rule can usually be remediated pretty quickly, as long as it is clear what the correct text label is for a given control. And when it comes to new code, this text label content is also easy to add to discussions in the design and requirements stages of the development cycle, shifting the accessibility conversation left. - -Since every violation in Cypress is linked to a Deque University page with details about the nature of the problem and how to fix it, all the needed context is available, even to engineers who may be new to the topic of accessiblity in general, or to a specific violation. - -Even if your first rule is not “Buttons must have discernible text”, we suggest finding one with a similarly small amount of coordination or understanding needed to fix, so the team can experience success as quickly as possible. - -### Finding low-hanging fruit with Cypress - -Cypress Accessibility generates a separate report for each page and component tested in a run. It’s also possible to see all accessibility rules that failed across the entire run. The rules-first approach is useful when finding low-hanging fruit. The rules with the smallest element count are the closest to being fully fixed, and removed from the report. If, at the “All Views” level, there’s not a good rule with a low element count, the next step is to focus on just a single View in the report. You might not be able to fix all buttons across the application, for example, but maybe you can pick a single page and fix just the issues on that specific page. - -### Setting the goal - -Your goal is to get your target rule’s failing elements down to zero for your chosen scope. That scope can be a specific page, a set of pages, or the entire application. Seeing that rule disappear from a report provides a clear finish line for you and your team. It also provides time to assess the impact of the changes. - -Often, due to the componentized nature of modern web apps, fixing an issue in one place has knock-on benefits around the app. Even if you focused on a narrow scope, your changes may still have had a wider impact, so remember: it’s fine for these first goals to be small at first. In fact, it’s preferable. - -### Iterating towards success {#iterating-towards-success} - -By setting a reasonable goal to understanding and fix specific rules scoped to defined areas of the application being tested, your team can make incremental progress on accessibility without becoming overwhelmed. With support from the Deque University “Learn More” links embedded into the reports, your team can specialize in one rule at a time, learn the related context, and then go deeper on more complicated or debatable topics. - -Given enough capacity, as dictated by the urgency of accessibility remediation combined with other business constraints, you will eventually solve all issues that are reported with Cypress Accessibility, and be well on your way to a more usable, accessible experience. - -If you are interested in using Cypress to detect and prevent regressions in the rules that you already have passing, you can learn how to use the Results API to do this in our [Blocking Pull Requests Section](/accessibility/guides/blocking-pull-requests). diff --git a/docs/accessibility/guides/introduction.mdx b/docs/accessibility/guides/introduction.mdx index 0ed4f67cbd..1cb85bdddb 100644 --- a/docs/accessibility/guides/introduction.mdx +++ b/docs/accessibility/guides/introduction.mdx @@ -12,26 +12,34 @@ sidebar_position: 10 - Common issues and questions related to all kinds of accessibility automation - How to use Cypress Accessibility to monitor a production website - Switching between "improvement" and "maintenance" mindsets - ::: -## Making the most of Cypress Accessibility +::: -In these guides, we'll describe how to leverage Cypress Accessibility to improve and maintain the accessibility of the websites and applications you test with Cypress. We'll also discuss how to work with your Cypress point-of-contact to set the product up to provide the most value to you based on the goals you have for each specific app you are testing. +## Maximize the Impact of Cypress Accessibility + +These guides will help you leverage Cypress Accessibility to improve and maintain the accessibility of the websites and applications you test with Cypress. Learn how to configure the tool for your specific goals and collaborate with your Cypress representative to maximize its value based on the goals you have for each application you are testing. Every project will be in a different state when you start to use Cypress Accessibility. You might find you have thousands of issues on hundreds of different pages and components, or you might have just a few issues detected in an already-accessible experience. -For projects that have a high volume of issues, we'll describe how to use Cypress Accessibility to drive the process of remediation and improvement, get some measurable wins on the board for your team, and support the gradual introduction of accessibility standards. +**For projects with many accessibility issues, we'll show you how to:** + +- Drive remediation and measurable improvement. +- Establish accessibility standards incrementally. +- Celebrate quick wins to build momentum. + +**For well-maintained projects, we'll focus on:** -For projects with few or no issues currently, we’ll lay out how Cypress supports a resilient, efficient, and timely approach to maintain your standards and avoid surprises over the long term. +- Efficiently maintaining high standards. +- Preventing regressions with resilient workflows. -The end goal of this process is not to simply pass all your automated accessibility checks, but to deliver a truly usable, accessible, and equal experience to all your users. So we'll also discuss limitations related to static automatic checks and how to write explicit test coverage to prevent regressions and increase confidence. +Passing automated checks is not the end goal—creating a truly usable, accessible, and equal experience to all your users is. These guides will also discuss limitations related to static automatic checks and how to write explicit test coverage to prevent regressions and increase confidence. -## Contents +## Next Steps -1. [Improving Accessibility](/accessibility/guides/improving-accessibility) -1. [Maintaining Accessibility](/accessibility/guides/maintaining-accessibility) -1. [Blocking pull requests and setting policies](/accessibility/guides/blocking-pull-requests) -1. [Getting feedback during local development](/accessibility/guides/local-development) +1. [Improve Accessibility](/accessibility/guides/improve-accessibility) +1. [Maintain Accessibility](/accessibility/guides/maintain-accessibility) +1. [Block pull requests and set policies](/accessibility/guides/block-pull-requests) +1. [Feedback during local development](/accessibility/guides/local-development) 1. [Production monitoring](/accessibility/guides/production-monitoring) -1. [Maximizing coverage](/accessibility/guides/maximizing-coverage) +1. [Maximize coverage](/accessibility/guides/maximize-coverage) 1. [Accessibility automation principles](/accessibility/guides/accessibility-automation) diff --git a/docs/accessibility/guides/local-development.mdx b/docs/accessibility/guides/local-development.mdx index ee0190c1c5..03f1f889bd 100644 --- a/docs/accessibility/guides/local-development.mdx +++ b/docs/accessibility/guides/local-development.mdx @@ -1,24 +1,48 @@ --- -title: Feedback during local development +sidebar_label: Feedback during local development +title: 'Feedback during local development | Cypress Accessibility Documentation' +description: 'Get precise, fast, and scoped accessibility feedback during local development by recording Cypress tests directly to Cypress Cloud—bypassing CI pipelines and ensuring accurate insights with minimal effort.' sidebar_position: 50 --- # Feedback during local development -While there are some workarounds possible with browser extensions or Cypress plugins, the most reliable solution for local development is to record to Cypress Cloud from your local development machine. This allows you to run a small set of tests directly related to your changes and fully skip your build pipeline and CI processes. It can be run with a one line command: +For reliable accessibility feedback during local development, the best approach is recording your tests directly to Cypress Cloud. By running tests related to your changes locally, you can bypass build pipelines and CI processes entirely. This can be achieved with a single command: -``` + + + +```shell npx cypress run --key --record --spec "cypress/e2e/my-spec.cy.js" ``` -(You can skip the `--key` option by setting `CYPRESS_RECORD_KEY` as an environment variable. Learn more about recording in our [docs about recorded runs](/cloud/features/recorded-runs)). + + + +```shell +yarn cypress run --key --record --spec "cypress/e2e/my-spec.cy.js" +``` + + + + +```shell +pnpm cypress run --key --record --spec "cypress/e2e/my-spec.cy.js" +``` + + + + +(You can skip the `--key` option by setting `CYPRESS_RECORD_KEY` as an environment variable. Learn more about recording in our [Recorded Runs documentation](/cloud/features/recorded-runs)). + +## Key benefits -This has 3 key benefits: +- **Accuracy:** See how Cypress processes your new code changes, taking into account your project's configuration, and using the Cloud's version of Axe Core®. There will be no violation reports for rules, pages, or elements your team has chosen to ignore, for example. +- **Speed:** With the application already built locally, tests can run immediately without requiring your build pipeline. +- **Scope:** Execute only the tests you need to focus on your changes, ensuring faster results and excluding unrelated issues. -- Accuracy \- you will see exactly what Cypress does with your new code changes, taking into account your project's configuration, and using the Cloud's version of Axe Core®. There will be no violations reports for rules, pages, or elements your team has chosen to ignore, for example. -- Speed \- since the application is already built locally, while you are working on it, it is ready to test immediately without your build pipeline -- Scope \- you can run only the tests you need, which can make results even faster, and helps you exclude issues unrelated to the area you are working on +Recording tests locally offers early feedback, helping you quickly detect if resolving one accessibility issue introduces another. This targeted testing workflow can significantly enhance development efficiency and quality. -One other reason to get early feedback this way is to quickly know if, in fixing one accessibility problem, you’ve revealed or introduced another. Recording small runs locally is a big quality-of-life improvement when you are specifically looking for this kind of feedback. +## Organizing Local Runs -If you expect your team to do a lot of runs this way and that these partial results would clutter your Cypress project, you can set up a dedicated Cloud project for “locally generated” runs to keep things organized. +If your team frequently records local runs and you're concerned about cluttering your Cypress Cloud project, consider setting up a dedicated Cloud project for "local runs". This keeps your main project organized while supporting local testing workflows. diff --git a/docs/accessibility/guides/maintain-accessibility.mdx b/docs/accessibility/guides/maintain-accessibility.mdx new file mode 100644 index 0000000000..0b18887624 --- /dev/null +++ b/docs/accessibility/guides/maintain-accessibility.mdx @@ -0,0 +1,68 @@ +--- +sidebar_label: Maintain Accessibility +title: 'Maintain Accessibility | Cypress Accessibility Documentation' +description: 'Learn how to maintain long-term accessibility with Cypress, using continuous feedback and monitoring to prevent regressions and keep your application accessible.' +sidebar_position: 30 +--- + +# Maintain accessibility + +Accessibility isn't a one-time project—it's a continuous process. This guide explains how to transition from addressing known issues to maintaining long-term accessibility in your projects. + +## Switching from "improving" to "maintaining" + +When a specific accessibility rule reaches zero violations for a page, component, or application, you enter _maintenance mode_ for that rule. From this point a few areas are imprortant to focus on: + +- **Pre-Merge Insights**: Any new violations detected on a feature branch are clear indicators that changes on that branch introduced the issue. This could stem from updates in application code that impact accessibility or expanded test coverage revealing new issues. +- **Post-Merge Monitoring**: If violations resurface on the main branch, tracing their introduction to specific pull requests helps pinpoint code or test changes causing the problem and facilitate follow-ups and training for developers. + +Your ultimate goal is to achieve a fully passing state across all rules. Cypress provides you a lot of flexibility to manage large amounts of accessibility violations and make incremental progress towards well-defined targets. Through a combination of fixing accessibility problems, and configuring Cypress to focus on a tight scope of rules, standards, and application areas that matter most to you, you might be closer than you think to a "clean" main branch, from which you can expand your standards over time. + +## Maintaining accessibility standards + +A clean report marks a pivotal change in your workflow. Instead of working through backlogs, the focus shifts to preventing accessibility issues from merging into the main branch. + +When those violations are all fixed, what exactly is Cypress Accessibility "doing" in your pipeline? And how should you measure success? + +### What Cypress Accessibility is doing when a project is in maintenance mode + +In maintenance mode, Cypress Accessibility provides two critical functions: + +- **Pre-Merge Detection**: Early feedback ensures that developers address issues before they reach production. +- **Post-Merge Monitoring**: Alerts highlight regressions, preventing accessibility debt from accumulating unnoticed. + +Accessibility debt tends to accumulate silently over time, leading to large, problematic backlogs that are difficult to work through, and land on your desk all at once after an audit. As you will have learned by going through the remediation process to get to a clean report in the first place: it's not always easy to fix an already-inaccessible experience. Sometimes you have to start a whole feature or component over, because you've built multiple structures on top of an inaccessible foundation. + +Early feedback about accessibility on feature branches will allow developers to review small, precise reports, and get all the context they need to either fix issues immediately, or raise problematic patterns for wider discussion. This is especially helpful when a new team member joins, who isn't as familiar with accessibility practices, or when a well-meaning full-stack developer - somebody who can "get by" in the front-end - unintentionally makes a minor change with major accessibility implications. + +Detecting problems before code merges allows the training, awareness-building, discussion, and remediation to all happen in the same cycle as regular code review, when everybody has all the context they need, not weeks or months later when it will be difficult to inject a ticket into the backlog to untangle what might have become a "load-bearing" accessibility problem. + +### How to measure success in maintenance mode + +The issues surfaced in Cypress Accessibility should follow certain patterns, allowing you to get a good "pulse" on your team's effectiveness with accessibility and time spent dealing with issues. Key metrics for evaluating success include: + +- **Trend Analysis**: Track violations on both the main branch and feature branches. A stable or downward trend indicates improved team awareness and fewer recurring issues. +- **Shifting Left**: Developers begin considering accessibility earlier, during ticket creation or design handoffs, reducing errors before they occur. At that point, mistakes are avoided, instead of corrected. + +While perfection isn't the goal, mature teams will notice fewer violations in reports over time, enabling them to focus on delivering value without compromising quality. + +Using Cypress Accessibility, you can treat accessibility failures as quality issues. Just like functional bugs and UI problems, they are important and need to be addressed. This is why optimizing and minimizing the effort related to accessibility is so valuable in a literal sense - your team is going to spend time on accessibility anyway, every cycle saved on dealing with accessibility is freed up for feature work, quality improvements, and other important activities. + +### Preparing for change + +Even with a strong maintenance strategy, disruptions can arise: + +- **Teams Change**: New hires or role shifts can impact an accessibility program. +- **Priorities Evolve**: Shifting focus within the organization might deprioritize accessibility. + +One of the biggest pain points that has been described to us by customers is unexpectedly having a large backlog of high-priority accessibility issues surfaced in an annual audit, even when they had fixed everything from the previous audit. + +Cypress Accessibility serves as an early warning system. Spikes in violations signal when action is needed, whether through training, process adjustments, or renewed focus on accessibility. + +## Summary + +Maintaining accessibility is an ongoing effort requiring organization-wide commitment. The automation layer of accessibility tests offered by Cypress Accessibility maximizes the amount of information you can have at your fingertips to improve your accessibility status and maintain an accessible application over the long term, in a software development environment where both expected and unexpected changes create constant challenges for quality. + +Cypress Accessibility provides a consolidated report with page- and component-level breakdowns and interactive, visual DOM snapshots to understand exactly what every violation relates to in your application. With filtering and configuration you can create a manageable series of goals to solve every accessibility issue that Axe Core® can detect. Once your accessibility report is fully passing for the rules and pages that make sense for your business and project, Cypress Accessibility supports continuous feedback on new code changes before they merge, and gives you an early warning system if things are getting off track. + +The clarity and efficiency from using Cypress Accessibility will optimize your team's time spent managing and fixing accessibility issues that are detectable by automation, helping them focus more on building great things that create value for your organization, without sacrificing this critical aspect of quality software development and building up debt over time to do so. diff --git a/docs/accessibility/guides/maintaining-accessibility.mdx b/docs/accessibility/guides/maintaining-accessibility.mdx deleted file mode 100644 index 2f63756a53..0000000000 --- a/docs/accessibility/guides/maintaining-accessibility.mdx +++ /dev/null @@ -1,52 +0,0 @@ ---- -title: Maintaining accessibility -sidebar_position: 30 ---- - -This section discusses the different mindset and success metrics you may have when a project moves from remediating known issues, to long term accessibility maintenance. - -## Switching from “improving” to “maintaining” - -As soon as a given accessibility rule reaches zero violations for a given page, component, or application, you enter “maintenance mode” for that rule. If the content being tested is stable, any time a failure of that rule shows back up in the reports for that area on a feature branch, it will be clear that changes on that feature branch are the reason for the failure. These changes could be changes in the application code that made it less accessible, **or** changes in the tests to cover new parts of the application. - -If a previously-passing rule starts failing on your main branch after not being fixed in a pull request, you can trace back to the re-introduction of the failure and understand what code or test change is responsible. This allows for follow-up and training with developers who introduce issues and gives you the needed information to know about and fix the problem itself. - -Your overall goal should be to reach a fully-passing state on every rule. Cypress provides you a lot of flexibility to manage large amounts of accessibility violations and make incremental progress towards well-defined targets. Through a combination of fixing accessibility problems, and configuring Cypress to focus on a tight scope of rules, standards, and application areas that matter most to you, you might be closer than you think to a “clean” main branch, from which you can expand your standards over time. - -## Maintaining accessibility - -Once you reach the state of a report that contains no violations, your organization’s usage of Cypress Accessibility will change dramatically, and start to focus mainly on detecting and preventing accessibility problems before they merge. This can be a big adjustment. Chipping away at a large backlog of known issues, and fixing things rule-by-rule, provides a clear measure of progress to your team and a sense of momentum. It’s likely you will have been able to measure success and set goals in relation to the accessibility violations present on your main branch up to now. - -When those violations are all fixed, what exactly is Cypress Accessibility “doing” in your pipeline? And how should you measure success? - -### What Cypress Accessibility is doing when a project is in maintenance mode {#what-cypress-accessibility-is-doing-when-a-project-is-in-maintenance-mode} - -The short answer is: providing **pre-merge insights**, and **post-merge monitoring**, that supports you in maintaining accessibility. Accessibility debt tends to accumulate silently over time, leading to large, problematic backlogs that are difficult to work through, and land on your desk all at once after an audit. As you will have learned by going through the remediation process to get to a clean report in the first place: it’s not always easy to fix an already-inaccessible experience. Sometimes you have to start a whole feature or component over, because you’ve built multiple structures on top of an inaccessible foundation. - -Early feedback about accessibility on feature branches will allow developers to review small, precise reports, and get all the context they need to either fix issues immediately, or raise problematic patterns for wider discussion. This is especially helpful when a new team member joins, who isn’t as familiar with accessibility practices, or when a well-meaning full-stack developer – somebody who can “get by” in the front-end – unintentionally makes a minor change with major accessibility implications. - -Detecting problems before code merges allows the training, awareness-building, discussion, and remediation to all happen in the same cycle as regular code review, when everybody has all the context they need, not weeks or months later when it will be difficult to inject a ticket into the backlog to untangle what might have become a “load-bearing” accessibility problem. - -### How to measure success in maintenance mode {#how-to-measure-success-in-maintenance-mode} - -The issues surfaced in Cypress Accessibility should follow certain patterns, allowing you to get a good “pulse” on your team’s effectiveness with accessibility and time spent dealing with issues. The best-case scenario is that, with a stable team of front-end developers working on the product, the total number of violations on the main branch, as well as the number of violations identified and corrected on feature branches, should trend downwards over time. This driven by a buildup of education as a result of the continuous feedback. The presence of the feedback should _change developer behavior_ and truly help shift accessibility left, into questions that get asked and answered even before development, when a ticket is created or a design is handed off. At that point, mistakes are avoided, instead of corrected. - -We don’t expect most teams to reach a state where they always get everything right in the first code commit: there will always be some issues surfacing in Cypress Cloud – but as teams mature, it’s reasonable to expect the reports to contain less and less violations. This state is ideal \- your development teams should be focused on adding value to your company by making changes to the product. - -For most of you reading this, accessibility problems are quality issues, just like bugs and UI issues, and they are important and need to be addressed. This is why optimizing and minimizing the effort related to accessibility is so valuable in a literal sense \- your team is going to spend time on accessibility anyway, every cycle saved on dealing with accessibility is freed up for feature work, quality improvements, and other important activities. - -### Ideal conditions never last forever - -The main thing that would disrupt the downward trend of pre-merge violations is plain old reality. Teams change, priorities change, people move around an organization, they get hired, they leave, they go on vacation. All of this can be disruptive to an accessibility program – sometimes it’s a specific person or small group that is keeping standards up and providing guidance to their peers. When that person moves out of their team, a previously stable accessibility situation can start to regress. - -Cypress Accessibility will help you see this kind of disruption in real time and help you take action as needed. One of the biggest pain points that has been described to us by customers is unexpectedly having a large backlog of high-priority accessibility issues surfaced in an annual audit, even when they had fixed everything from the previous audit. This happens for a few reasons: sometimes the fixes introduce new problems, sometimes net-new work is added that includes new accessibility violations, or design changes lead to front-end modifications with unexpected accessibility implications. - -With Cypress Accessibility, you have an early warning system for these situations. They will show up as an increase in violations reported during code reviews, and there will be no chance for this accessibility debt to accumulate silently. Your team has an opportunity to “self-correct” when this happens, or if accessibility is disregarded and violations do get merged, you will be able to track that on your main branch. - -## Summary - -Accessibility is an ongoing process that affects the whole organization, not a single project of fixing a known list of issues and “completing” accessibility. The automation layer of accessibility tests offered by Cypress Accessibility maximizes the amount of information you can have at your fingertips to improve your accessibility status and maintain an accessible application over the long term, in a software development environment where both expected and unexpected changes create constant challenges for quality. - -Cypress Accessibility provides a consolidated report with page- and component-level breakdowns and interactive, visual DOM snapshots to understand exactly what every violation relates to in your application. With filtering and configuration you can create a manageable series of goals to solve every accessibility issue that Axe Core® can detect. Once your accessibility report is fully passing for the rules and pages that make sense for your business and project, Cypress Accessibility supports continuous feedback on new code changes before they merge, and gives you an early warning system if things are getting off track. - -The clarity and efficiency from using Cypress Accessibility will optimize your team’s time spent managing and fixing accessibility issues that are detectable by automation, helping them focus more on building great things that create value for your organization, without sacrificing this critical aspect of quality software development and building up debt over time to do so. diff --git a/docs/accessibility/guides/maximize-coverage.mdx b/docs/accessibility/guides/maximize-coverage.mdx new file mode 100644 index 0000000000..b63884cd57 --- /dev/null +++ b/docs/accessibility/guides/maximize-coverage.mdx @@ -0,0 +1,46 @@ +--- +sidebar_label: Maximize coverage +title: 'Maximize coverage | Cypress Accessibility Documentation' +description: "Expand accessibility testing by combining Cypress Accessibility's automated checks with custom assertions tailored to your application." +sidebar_position: 70 +--- + +# Maximize coverage + +As you become more familiar with accessibility violations and their underlying principles, you'll notice the connection between generic, automated checks (e.g., Axe Core®) and the specific user-facing behaviors of your application. Often, the insights gained from Cypress Accessibility's automated checks can guide you to create custom assertions tailored to your application's unique requirements. + +## Examples + +### Button Accessibility Test + +For example, Cypress Accessibility might flag a button with a missing accessible name. While adding any text might satisfy Axe Core's rule, the actual text should align with your application's context. Once you address the issue in your codebase, you can enhance your Cypress tests by asserting the specific text, ensuring that future developers understand its importance as part of the application's specification. + +```js +describe('Button Accessibility Test', () => { + it('should have the correct accessible name for the button', () => { + // Visit the page containing the button + cy.visit('/your-page-url') + + // Select the button by its identifier + cy.get('[data-test-id="submit-button"]') + // Ensure the button has an accessible name + .should('have.attr', 'aria-label', 'Submit Form') + // Optionally, assert the button contains the visible text + .and('contain', 'Submit') + }) +}) +``` + +This principle can extend to larger and more complex scenarios. + +## Accessibility "hotspots" + +Each violation detected by Cypress Accessibility serves as a signal that accessibility might not have been adequately considered in a specific area of your application. Since Cypress Accessibility provides full DOM visibility for every violation, you can efficiently perform manual checks on related patterns in the UI. This approach often reveals accessibility "hotspots", where usability issues may exist that automated tools cannot detect. + +Conducting manual audits in these areas can uncover deeper usability challenges. These findings can then be cataloged, addressed, and regression-tested using Cypress automation. + +## Combining testing approaches + +A hybrid approach that combines Cypress Accessibility's always-on automated checks with custom assertions can significantly improve long-term accessibility quality. This approach ensures accessibility regressions are caught and fixed, regardless of whether they were initially detected automatically or through manual review. + +Automated tools like Axe Core® are estimated to detect approximately 57% of common accessibility issues. By incorporating manual insights and user feedback into your Cypress test suite, you can achieve far greater coverage. Explicit assertions on top of automated error detection empower your pipeline to deliver consistent, actionable feedback about accessibility regressions. diff --git a/docs/accessibility/guides/maximizing-coverage.mdx b/docs/accessibility/guides/maximizing-coverage.mdx deleted file mode 100644 index cb41b402f5..0000000000 --- a/docs/accessibility/guides/maximizing-coverage.mdx +++ /dev/null @@ -1,18 +0,0 @@ ---- -title: Maximizing coverage -sidebar_position: 70 ---- - -As you gain more familiarity with the nature of accessibility violations and the underlying ideas behind them, you will see the connection between the generic, static checks offered by Axe Core®, and the specific user-facing behavior of your own applications. In many cases, you will be able to add new assertions to your tests, based on what you learned from Cypress Accessibility, but customized to your specific situation. - -An example of this is when Cypress Accessibility reports a button with a missing accessible name. The actual text that should go with the button depends upon your own application, and is unknown to Axe-Core, so a label check will pass for almost _any_ text that is placed in the button. After you’ve added the required text to the codebase, you can write an assertion about that specific text in your Cypress tests, to make clear to future developers that it is part of the spec of the application. That’s a small example, but it applies in many larger situations too. - -## Accessibility “hotspots” - -Another way to leverage the violations detected by Cypress Accessibility is to treat each one as a sign that accessibility in that area of the application may not have been considered when it was built. Because the whole DOM of the page or component is available for every violation, it’s faster to do a manual check of the patterns being used in the page around an Axe Core® violation. This can reveal accessibility “hotspots” containing usability issues that aren’t otherwise detectable by automation, but would appear in a manual audit. These can be cataloged and corrected, then regression tested with traditional Cypress automation. - -## Combining testing approaches - -Combining the always-on generic checks provided by Cypress Accessibility with regular Cypress assertions about specific details of your application can help accessibility quality stay high once things are fixed, regardless of whether problems were originally detected manually or automatically. This can take your accessibility confidence far beyond the 57% of issues detected, often cited when thinking about Axe Core® as a standalone library. - -In the testing context, you are already specifying user-facing aspects of your application, and that means almost any findings from manual regression checks or user feedback, can be explicitly covered by your standard testing approach. So the overall potential of your automation pipeline to provide good feedback about accessibility regressions is very high, if you leverage explicit assertions on top of the static check "error detection" approach to find mistakes. diff --git a/docs/accessibility/guides/production-monitoring.mdx b/docs/accessibility/guides/production-monitoring.mdx index 786002f7b4..ec807cba4b 100644 --- a/docs/accessibility/guides/production-monitoring.mdx +++ b/docs/accessibility/guides/production-monitoring.mdx @@ -1,17 +1,27 @@ --- -title: Production monitoring +sidebar_label: Production monitoring +title: 'Production monitoring | Cypress Accessibility Documentation' +description: 'Monitor accessibility issues in production by running scheduled Cypress tests against live environments, capturing dynamic content changes, and generating automated reports for a comprehensive accessibility overview.' sidebar_position: 60 --- -Many organizations use Cypress automation in CI, and for local development, but also run scheduled tests against production or staging environments. This pattern works especially well for situations where a Content Management System is involved and content authors can add or modify things that affect the user interface, or anything else where the content being tested is not fully controlled by the testing environment. Since it can be impossible to write explicit assertions for ever-changing content like this, or for A/B tests of different UI variations implemented outside your codebase, production smoke tests tend to be more high-level health checks. +# Production monitoring -Cypress Accessibility makes this kind of testing even more useful, as it will be able to run just fine against that dynamic set of content. Just by visiting a production URL in your Cypress tests, and taking one or two light actions in the UI, that page will be captured in your accessibility report, and alert you about content changes outside of your development lifecycle that introduce accessibility issues. +Cypress is commonly used in CI and local development workflows. It can also be leveraged to monitor production or staging environments through scheduled tests. This approach is particularly useful for environments involving dynamic content, such as those managed by Content Management Systems (CMS), where content changes can impact accessibility. -One common approach to this kind of monitoring, if the project does not already have Cypress tests, is to use something like a site map, or just an array of target URLs, to quickly spin up a lot of coverage without writing new tests for every page, and then run that Cypress test on a schedule to get daily updated accessibility reports. Here is an example of doing this by requesting the `sitemap.xml` file from a website and scrolling to the footer on each page. +Dynamic or externally controlled content, such as A/B tests or user-generated content, often makes it challenging to write explicit assertions. For these cases, production smoke tests can serve as high-level health checks, ensuring the accessibility of dynamic UI variations without requiring full test coverage. + +## Using Cypress for Production Monitoring + +Cypress Accessibility enables you to test dynamic content seamlessly. By visiting a production URL within your Cypress tests and performing minimal UI interactions, you can capture the page's accessibility state in reports. This allows teams to detect accessibility issues introduced outside the regular development lifecycle. + +## Example: Automated Sitemap-Based Testing + +If your project lacks existing Cypress tests, a common approach is to generate test coverage from a sitemap or an array of target URLs. These URLs can be used to perform light interactions and capture accessibility reports. Below is an example of how to automate this process by using a sitemap and scrolling to the footer on each page: ```js -describe('accessibility scan', () => { - it('checks accessibility with the sitemap.xml', () => { +describe('Accessibility Scan', () => { + it('Checks accessibility with the sitemap.xml', () => { cy.request('https:///sitemap.xml').then((response) => { const xmlString = response.body const parser = new DOMParser() @@ -20,10 +30,8 @@ describe('accessibility scan', () => { (loc) => loc.textContent ) Cypress._.each(URLs, (URL) => { - it(`visits ${URL}`, () => { - cy.visit(URL) - cy.contains('').scrollIntoView() - }) + cy.visit(URL) + cy.contains('').scrollIntoView() }) }) }) diff --git a/src/components/docs-image/index.tsx b/src/components/docs-image/index.tsx index c4deade233..acc2983855 100644 --- a/src/components/docs-image/index.tsx +++ b/src/components/docs-image/index.tsx @@ -7,15 +7,18 @@ export default function DocsImage({ caption, src, title, - width + width, + noBorder, + style, }: DocsImageProps) { return ( -
+
{alt {caption && (

diff --git a/src/components/docs-image/style.module.css b/src/components/docs-image/style.module.css index 420ee68c3b..079cc46de8 100644 --- a/src/components/docs-image/style.module.css +++ b/src/components/docs-image/style.module.css @@ -1,9 +1,11 @@ .docsImage { margin-bottom: 1rem; + box-shadow: 0 2px 10px 1px rgba(0, 0, 0, 0.26); } -.docsImage img { - box-shadow: 0 2px 10px 1px rgba(0, 0, 0, 0.26) !important; +.noBorder { + margin-bottom: 1rem; + box-shadow: none; } .caption { diff --git a/src/components/docs-image/types.d.ts b/src/components/docs-image/types.d.ts index 646262bb0e..4026a5c1eb 100644 --- a/src/components/docs-image/types.d.ts +++ b/src/components/docs-image/types.d.ts @@ -3,7 +3,8 @@ export interface DocsImageProps { caption: string src: string title: string - width: number; + width: number noBorder: boolean style: string + className: string } diff --git a/src/components/icon/index.tsx b/src/components/icon/index.tsx index 17dee5a0fa..1834e3f8ea 100644 --- a/src/components/icon/index.tsx +++ b/src/components/icon/index.tsx @@ -1,8 +1,16 @@ import React from "react"; import { FontAwesomeIcon } from "@fortawesome/react-fontawesome"; -// TODO: refactor this component to be more efficient -export default function Icon({ name, color, inline, url, callout, contentType }) { +interface IconProps { + name: string; + color?: string; + inline?: boolean; + url?: string; + callout?: string; + contentType?: string; +} + +export default function Icon({ name, color, inline, url, callout, contentType }: IconProps) { const iconName = name === "github" ? ["fab", "github"] : name; const space = inline ? '.25rem' : '0.5rem' @@ -21,7 +29,7 @@ export default function Icon({ name, color, inline, url, callout, contentType }) return ( <> - + { url ? { content } : null} ) diff --git a/src/css/custom.scss b/src/css/custom.scss index af00a65f71..81bbf7ddf6 100644 --- a/src/css/custom.scss +++ b/src/css/custom.scss @@ -374,7 +374,7 @@ div[class^='codeBlockTitle'] { .tabs-container .tabs { background-color: #1f2937; border-radius: 0.4rem 0.4rem 0 0; - box-shadow: inset 0px -6px 0px -4px var(--ifm-code-background-color-light); + box-shadow: inset 0 -6px 0 -4px var(--ifm-code-background-color-light); } @media (min-width: 997px) and (max-width: 1180px) { @@ -505,3 +505,48 @@ h6 { #__docusaurus_skipToContent_fallback { scroll-margin-top: var(--ifm-navbar-height); } + +ul.guidesList { + display: grid; + grid-template-columns: repeat(auto-fit, minmax(400px, 1fr)); + gap: 1.5rem; + margin: 0 !important; + padding: 0 !important; +} + +body ul.guidesList li { + box-sizing: border-box; + list-style: none; + margin: 0 !important; + + a { + display: block; + height: 100%; + padding: 20px 30px; + border-bottom: none !important; + + svg { + height: 26px; + width: 26px; + } + } + + h3 { + padding: 6px 0 4px; + font-size: 1.2rem; + color: var(--ifm-color-indigo-500); + background-size: 0 2px; + transition: all 0.3s ease-in-out; + @apply bg-gradient-to-r bg-bottom bg-no-repeat; + } + + &:hover h3 { + background-size: 100% 2px; + @apply from-indigo-300 to-indigo-300; + } + + p { + margin-bottom: 0.5em; + color: var(--ifm-color-gray-800); + } +} \ No newline at end of file diff --git a/src/theme/MDXComponents.js b/src/theme/MDXComponents.js index 4b9499717e..ec28107fa1 100644 --- a/src/theme/MDXComponents.js +++ b/src/theme/MDXComponents.js @@ -69,12 +69,21 @@ import { faPlus, faQuestionCircle, faSearch, + faLaptopCode, faStar, faSyncAlt, faTerminal, faTimes, faTree, faVideo, + faMagnifyingGlassChart, + faComputer, + faBugSlash, + faChartLine, + faClock, + faBookOpenReader, + faUserShield, + faCircleHalfStroke, } from '@fortawesome/free-solid-svg-icons' library.add( @@ -111,12 +120,21 @@ library.add( faPlus, faQuestionCircle, faSearch, + faLaptopCode, faStar, faSyncAlt, faTerminal, faTimes, faTree, - faVideo + faVideo, + faMagnifyingGlassChart, + faComputer, + faBugSlash, + faChartLine, + faClock, + faBookOpenReader, + faUserShield, + faCircleHalfStroke, ) export default { diff --git a/static/img/accessibility/core-concepts/cypress-accessibility-element-details-on-click.png b/static/img/accessibility/core-concepts/cypress-accessibility-element-details-on-click.png new file mode 100644 index 0000000000..6ea7554e1d Binary files /dev/null and b/static/img/accessibility/core-concepts/cypress-accessibility-element-details-on-click.png differ diff --git a/static/img/accessibility/core-concepts/cypress-accessibility-filter-views-and-rules.png b/static/img/accessibility/core-concepts/cypress-accessibility-filter-views-and-rules.png new file mode 100644 index 0000000000..8a086825c3 Binary files /dev/null and b/static/img/accessibility/core-concepts/cypress-accessibility-filter-views-and-rules.png differ