diff --git a/docs/accessibility/changelog.mdx b/docs/accessibility/changelog.mdx index 3c56015ac6..a61eafbb3c 100644 --- a/docs/accessibility/changelog.mdx +++ b/docs/accessibility/changelog.mdx @@ -7,6 +7,10 @@ sidebar_position: 200 # Changelog +## Week of 11/11/2024 + +- A "Share Issue" button has been added to the element details section for each reported rule violation. This allows for easily copying the key details and related links for a particular issue into your clipboard to add to a message or to a ticket in your issue tracking system. + ## Week of 10/28/2024 - Our [Axe Core® versioning policy](/accessibility/core-concepts/how-it-works#Axe-Core-updates) has been published, providing a 30-day buffer between any Axe Core® releases and the adoption of those releases in Cypress Cloud. diff --git a/docs/accessibility/guides/_category_.json b/docs/accessibility/guides/_category_.json new file mode 100644 index 0000000000..371eacc4b7 --- /dev/null +++ b/docs/accessibility/guides/_category_.json @@ -0,0 +1,4 @@ +{ + "label": "Guides", + "position": 70 +} diff --git a/docs/accessibility/guides/accessibility-automation.mdx b/docs/accessibility/guides/accessibility-automation.mdx new file mode 100644 index 0000000000..c1b6565503 --- /dev/null +++ b/docs/accessibility/guides/accessibility-automation.mdx @@ -0,0 +1,47 @@ +--- +title: Accessibility automation principles +sidebar_position: 80 +--- + +This guide describes some general ideas that are relevant to Cypress Accessibility, but are also somewhat shared among accessibility automation tools of any kind. + +## What it means to fix all Axe Core® violations {#what-it-means-to-fix-all-axe-core-violations} + +Cypress Accessibility runs Axe Core® checks on every significant state and variation of the applications you test with Cypress. This kind of automation can catch up to [57% of issues that would appear in a manual audit](https://www.deque.com/blog/automated-testing-study-identifies-57-percent-of-digital-accessibility-issues/), but it does not replace the human judgment that is applied during an audit. Auditors test a wide variety of scenarios and technologies in a way that would be impossible to fully automate. + +It helps to think of accessibility at three levels. The first is “basic usability”. Paying attention to issues reported by Cypress and Axe Core® should help you to this point. This is the threshold at which a disabled user is likely to be able to make substantial progress through the application, but may still hit major barriers or usability issues. If your contact forms and issue reporting systems are accessible, you will be able to learn about these problems from actual users. + +The second level of accessibility is full conformance with all Web Content Accessibility Guidelines (WCAG) success criteria. As mentioned above, more than half of issues reported in manual audits can be reported and caught early with Axe Core ®automation. Conformance with the WCAG is often the standard set by legislation and regulation related to web accessibility. If your goal is regulatory compliance, this is the bar. No generic automation tool can “prove” compliance with this standard, it **requires** human assessment. + +The third level, and the true goal of many accessibility programs, is a good user experience. This needs to be validated by actual disabled users who are daily users of assistive technology. It’s entirely possible to have a website that passes the first two levels, and still offers a poor user experience. + +Accessibility is about communicating the nature and structure of the application, and the available actions and information, in a way that allows disabled users to independently understand the interface and complete all tasks. The foundational [POUR principles](https://www.24a11y.com/2019/pour-the-foundation-first/) of the WCAG \- that software must be Perceivable, Operable, Understandable and Robust \- are bigger than any specific set of test plans or static checks. While Axe Core® tell us about many genuine accessibility barriers, it is important to leverage that information into making good decisions about accessibility and providing a truly equal experience. + +## False positives + +False positives are always a risk in any automated scans. Both Cypress Accessibility and Axe Core® are designed in a way that minimizes them, but it’s always possible that there will be some violations in the report that you recognize as invalid and would rather exclude, or are genuinely incorrect. + +While false positives are rare, if in doubt, use the “provide feedback” button to alert us to something you see as a false positive and we can work on diagnosing and solving the issue from our side. + +### Genuine false positives + +There are two main possible sources of genuine false positive results: + +1. The check was run on an **invalid** **state** \- for example a partially hydrated DOM \- that no user would ever see. These are scenarios that Cypress can account for in the product if they are reported. +2. The state tested was correct but Axe Core® reported a violation that it shouldn’t have, as a result of a bug or missing check. Since Axe Core® is an open-source product, if and when we encounter bugs, we can open issues in the project and even make pull requests if needed. + +### Loading states + +There’s another scenario that can present as a potential false positive: legitimate user-facing loading states. These are situations where some data is pending in your application. + +It might be tempting to say “this state shouldn’t be captured, it’s not fully loaded” \- but in reality, these types of “pending” states are valid, and often contain visual information like loading spinners, UI skeletons, and progress bars which should be accessible themselves, so the user knows what data is loading. It’s also possible that the data doesn’t come in at all, or is very slow, and a user needs to figure out what is happening and what they can do, so the application needs to remain in a fully accessible, usable condition even if data is pending. + +In general it’s a good idea in front-end development to “make impossible states impossible”, and in doing so you can follow the accessibility principle of Robustness (the last of the 4 POUR principles mentioned above), while also making it much less likely that an invalid state is possible to reach in Cypress in the first place. This can also have a beneficial effect on the overall stability of your tests. + +### Assistive technology support + +Certain accessibility techniques are not fully supported by all assistive technologies like screen readers or voice control software. Axe Core® accounts for this. The software and operating systems tested against are listed in their [Accessibility Supported document](https://github.com/dequelabs/axe-core/blob/develop/doc/accessibility-supported.md). For example: it can seem like a false positive from Axe Core® if a failing technique used in your code appears to work correctly in a certain screen reader, browser and operating system combination that you test with. But if it does not have sufficient accessibility support in commonly used technology configurations, Axe Core® will intentionally fail that technique, allowing you to switch to something with better support. + +## Summary + +Accessibility automation is a crucial part of an accessibility strategy in any fast-moving, modern codebase. When multiple code changes are merged on a daily or weekly basis, there is no possibility to run manual checks on every state and variation of the application prior to merge. Understanding some of the details of an "always on" set of accessibility checks in your test pipeline, where a lot of states are automatically detected, will help you develop a feel for how Cypress Accessibility behaves and what kinds of things it is best at finding, as well as what might be left over for other forms of testing. diff --git a/docs/accessibility/guides/accessibility-automation/_category_.json b/docs/accessibility/guides/accessibility-automation/_category_.json new file mode 100644 index 0000000000..cbe6ea57c2 --- /dev/null +++ b/docs/accessibility/guides/accessibility-automation/_category_.json @@ -0,0 +1,5 @@ +{ + "label": "Accessibility automation", + "collapsible": true, + "collapsed": false +} diff --git a/docs/accessibility/guides/blocking-pull-requests.mdx b/docs/accessibility/guides/blocking-pull-requests.mdx new file mode 100644 index 0000000000..2b4fd92b51 --- /dev/null +++ b/docs/accessibility/guides/blocking-pull-requests.mdx @@ -0,0 +1,60 @@ +--- +title: Blocking pull requests and setting policies +sidebar_position: 40 +--- + +Cypress Accessibility reports are generated on the server in Cypress Cloud, based on artifacts uploaded by the test runs. Because of this, there is no impact to your actual Cypress test execution, in terms of performance or anything else - including failing the Cypress run due to accessibility violations. + +## Using the Results API + +The [Cypress Accessibility Results API](/accessibility/results-api) provides access to accessibility results after the run is completed, so that you can implement patterns like blocking a pull request or sending out alerts based on specific accessibility criteria. This is done by adding a separate accessibility verification step to your CI pipeline, and using a helper function provided by Cypress to automatically request the report for the appropriate Cypress run based on the build context. The example below shows how this can look in GitHub Actions when there is an accessibility failure based on your own chosen policy: + + + +## Implementing a status check + +The API gives you full flexibility to examine the results and react to it in whatever way you need to. It also provides a signal that can be used in status checks on Pull Requests. Here is an example where the accessibility verification step is passing: + + + +## Implementing a policy in the verification step + +The [Results API Documentation](/accessibility/results-api) describes the capabilities of the API in more detail, but here is a simplified example of how to use the API to implement the "no new failing accessibility rules" policy shown above. + +```js +const { getAccessibilityResults } = require('@cypress/extract-cloud-results') + +// define your known failing rules +const rulesWithExistingViolations = [ + 'button-name', + ... +] + +// call the API +getAccessibilityResults() + .then((results) => { + // compare the accessibility results with your acceptable standard + const newRuleViolations = results.rules.filter((rule) => { + return !rulesWithExistingViolations.includes(rule.name) + }) + + if (newRuleViolations.length > 0) { + // based on the standard, trigger any side effect you like, such as failing the step + throw new Error( + `${newRuleViolations.length} rule regressions were introduced and must be fixed.` + ) + } +}) +``` + +This process of examining the results and then deciding the appropriate response gives you the most flexibility in terms of how exactly you want to force a stopping point or specific reaction based on the contents. You have access to the results as well as any context needed from the CI environment, such as tags, to implement the correct response to any specific result values. diff --git a/docs/accessibility/guides/improving-accessibility.mdx b/docs/accessibility/guides/improving-accessibility.mdx new file mode 100644 index 0000000000..ecbf2b27c3 --- /dev/null +++ b/docs/accessibility/guides/improving-accessibility.mdx @@ -0,0 +1,60 @@ +--- +title: Improving accessibility +sidebar_position: 20 +--- + +In this section we'll discuss how to break down the data in your accessibility report and make a plan for remediation. + + + +## Starting from Scratch + +When you first start to use Cypress Accessibility, you may be testing a legacy application with many existing accessibility violations. The [Accessibility tab for a run](/accessibility/core-concepts/run-level-reports) can show multiple violations detected on dozens or hundreds of pages in your application. The number of failing elements and page or component snapshots available might be overwhelming, making it difficult to know where to start. + +In this situation, the first step is to understand some of the key variables and how they are related within your organization. The kinds of things to consider are: + +- your goals for the accessibility of the application +- your timeline for making it accessible +- the available resources for remediation +- what code is likely to change in the near future, where accessibility can be included in designs and requirements +- the consequences, risks, and costs of remaining inaccessible + +These factors will help you find the right strategy for engaging with the results displayed in your accessibility reports. While incremental accessibility efforts by designers and engineers are never wasted, the best outcomes come from an overall organizational commitment to ship accessible software, which requires investment from all departments, so understanding your organization's stance on accessibility and who needs to be in the conversation is also helpful. + +If your organization or team is newer to accessibility, you are likely to find plenty of issues reported by Axe Core® in Cypress Cloud. Many of the issues in your first report may relate to concepts that are new, such as [ARIA attributes](https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA) or [invalid nesting of interactive controls](https://dequeuniversity.com/rules/axe/4.2/nested-interactive). Since it will be impossible to fix all issues overnight, we recommend finding some low-hanging fruit as a starting point. This can help your team build confidence by getting some wins on the board, and teach them the overall idea of how accessibility problems get fixed. + +### Good low-hanging fruit + +Some of the easiest, and most valuable issues to tackle first come from [WCAG SC 4.2.1 \- Name, role, value](https://www.w3.org/WAI/WCAG21/Understanding/name-role-value.html). This is the part of the Web Content Accessibility Guidelines that’s concerned with making sure interactive controls have appropriate labels for all users. In general, your app is likely to have some violations in this category, and Axe-Core® tends to flag these as Critical or Serious, because the lack of a label usually makes it difficult or impossible for some users to know what something does in a user interface. + +Within the “Name, role, value” success criterion, the [`button-name`](https://dequeuniversity.com/rules/axe/4.7/button-name) rule in Axe Core® (“Buttons must have discernible text”) is a great place to start. Icons being used as buttons are common in modern user interfaces and they are often missing descriptive text content for screen reader users, which directly impacts their ability to understand the purpose and effect of the button and independently complete tasks in your application. + +This rule, and others like it, are good starting points because the rule is easy to understand and easy to fix. Since the solution is purely a code update to add the missing label text to buttons, this category of rule requires little or no coordination across departments or teams to address. These types of issues are also straightforward to demonstrate and validate even for beginners with a screen reader. + +Existing code failing the button-name rule can usually be remediated pretty quickly, as long as it is clear what the correct text label is for a given control. And when it comes to new code, this text label content is also easy to add to discussions in the design and requirements stages of the development cycle, shifting the accessibility conversation left. + +Since every violation in Cypress is linked to a Deque University page with details about the nature of the problem and how to fix it, all the needed context is available, even to engineers who may be new to the topic of accessiblity in general, or to a specific violation. + +Even if your first rule is not “Buttons must have discernible text”, we suggest finding one with a similarly small amount of coordination or understanding needed to fix, so the team can experience success as quickly as possible. + +### Finding low-hanging fruit with Cypress + +Cypress Accessibility generates a separate report for each page and component tested in a run. It’s also possible to see all accessibility rules that failed across the entire run. The rules-first approach is useful when finding low-hanging fruit. The rules with the smallest element count are the closest to being fully fixed, and removed from the report. If, at the “All Views” level, there’s not a good rule with a low element count, the next step is to focus on just a single View in the report. You might not be able to fix all buttons across the application, for example, but maybe you can pick a single page and fix just the issues on that specific page. + +### Setting the goal + +Your goal is to get your target rule’s failing elements down to zero for your chosen scope. That scope can be a specific page, a set of pages, or the entire application. Seeing that rule disappear from a report provides a clear finish line for you and your team. It also provides time to assess the impact of the changes. + +Often, due to the componentized nature of modern web apps, fixing an issue in one place has knock-on benefits around the app. Even if you focused on a narrow scope, your changes may still have had a wider impact, so remember: it’s fine for these first goals to be small at first. In fact, it’s preferable. + +### Iterating towards success {#iterating-towards-success} + +By setting a reasonable goal to understanding and fix specific rules scoped to defined areas of the application being tested, your team can make incremental progress on accessibility without becoming overwhelmed. With support from the Deque University “Learn More” links embedded into the reports, your team can specialize in one rule at a time, learn the related context, and then go deeper on more complicated or debatable topics. + +Given enough capacity, as dictated by the urgency of accessibility remediation combined with other business constraints, you will eventually solve all issues that are reported with Cypress Accessibility, and be well on your way to a more usable, accessible experience. + +If you are interested in using Cypress to detect and prevent regressions in the rules that you already have passing, you can learn how to use the Results API to do this in our [Blocking Pull Requests Section](/accessibility/guides/blocking-pull-requests). diff --git a/docs/accessibility/guides/introduction.mdx b/docs/accessibility/guides/introduction.mdx new file mode 100644 index 0000000000..0ed4f67cbd --- /dev/null +++ b/docs/accessibility/guides/introduction.mdx @@ -0,0 +1,37 @@ +--- +title: Introduction +sidebar_position: 10 +--- + +:::info + +##### What you'll learn + +- How to set goals and make progress using Cypress Accessibility +- How to generate reports during local development cycles +- Common issues and questions related to all kinds of accessibility automation +- How to use Cypress Accessibility to monitor a production website +- Switching between "improvement" and "maintenance" mindsets + ::: + +## Making the most of Cypress Accessibility + +In these guides, we'll describe how to leverage Cypress Accessibility to improve and maintain the accessibility of the websites and applications you test with Cypress. We'll also discuss how to work with your Cypress point-of-contact to set the product up to provide the most value to you based on the goals you have for each specific app you are testing. + +Every project will be in a different state when you start to use Cypress Accessibility. You might find you have thousands of issues on hundreds of different pages and components, or you might have just a few issues detected in an already-accessible experience. + +For projects that have a high volume of issues, we'll describe how to use Cypress Accessibility to drive the process of remediation and improvement, get some measurable wins on the board for your team, and support the gradual introduction of accessibility standards. + +For projects with few or no issues currently, we’ll lay out how Cypress supports a resilient, efficient, and timely approach to maintain your standards and avoid surprises over the long term. + +The end goal of this process is not to simply pass all your automated accessibility checks, but to deliver a truly usable, accessible, and equal experience to all your users. So we'll also discuss limitations related to static automatic checks and how to write explicit test coverage to prevent regressions and increase confidence. + +## Contents + +1. [Improving Accessibility](/accessibility/guides/improving-accessibility) +1. [Maintaining Accessibility](/accessibility/guides/maintaining-accessibility) +1. [Blocking pull requests and setting policies](/accessibility/guides/blocking-pull-requests) +1. [Getting feedback during local development](/accessibility/guides/local-development) +1. [Production monitoring](/accessibility/guides/production-monitoring) +1. [Maximizing coverage](/accessibility/guides/maximizing-coverage) +1. [Accessibility automation principles](/accessibility/guides/accessibility-automation) diff --git a/docs/accessibility/guides/local-development.mdx b/docs/accessibility/guides/local-development.mdx new file mode 100644 index 0000000000..ee0190c1c5 --- /dev/null +++ b/docs/accessibility/guides/local-development.mdx @@ -0,0 +1,24 @@ +--- +title: Feedback during local development +sidebar_position: 50 +--- + +# Feedback during local development + +While there are some workarounds possible with browser extensions or Cypress plugins, the most reliable solution for local development is to record to Cypress Cloud from your local development machine. This allows you to run a small set of tests directly related to your changes and fully skip your build pipeline and CI processes. It can be run with a one line command: + +``` +npx cypress run --key --record --spec "cypress/e2e/my-spec.cy.js" +``` + +(You can skip the `--key` option by setting `CYPRESS_RECORD_KEY` as an environment variable. Learn more about recording in our [docs about recorded runs](/cloud/features/recorded-runs)). + +This has 3 key benefits: + +- Accuracy \- you will see exactly what Cypress does with your new code changes, taking into account your project's configuration, and using the Cloud's version of Axe Core®. There will be no violations reports for rules, pages, or elements your team has chosen to ignore, for example. +- Speed \- since the application is already built locally, while you are working on it, it is ready to test immediately without your build pipeline +- Scope \- you can run only the tests you need, which can make results even faster, and helps you exclude issues unrelated to the area you are working on + +One other reason to get early feedback this way is to quickly know if, in fixing one accessibility problem, you’ve revealed or introduced another. Recording small runs locally is a big quality-of-life improvement when you are specifically looking for this kind of feedback. + +If you expect your team to do a lot of runs this way and that these partial results would clutter your Cypress project, you can set up a dedicated Cloud project for “locally generated” runs to keep things organized. diff --git a/docs/accessibility/guides/maintaining-accessibility.mdx b/docs/accessibility/guides/maintaining-accessibility.mdx new file mode 100644 index 0000000000..2f63756a53 --- /dev/null +++ b/docs/accessibility/guides/maintaining-accessibility.mdx @@ -0,0 +1,52 @@ +--- +title: Maintaining accessibility +sidebar_position: 30 +--- + +This section discusses the different mindset and success metrics you may have when a project moves from remediating known issues, to long term accessibility maintenance. + +## Switching from “improving” to “maintaining” + +As soon as a given accessibility rule reaches zero violations for a given page, component, or application, you enter “maintenance mode” for that rule. If the content being tested is stable, any time a failure of that rule shows back up in the reports for that area on a feature branch, it will be clear that changes on that feature branch are the reason for the failure. These changes could be changes in the application code that made it less accessible, **or** changes in the tests to cover new parts of the application. + +If a previously-passing rule starts failing on your main branch after not being fixed in a pull request, you can trace back to the re-introduction of the failure and understand what code or test change is responsible. This allows for follow-up and training with developers who introduce issues and gives you the needed information to know about and fix the problem itself. + +Your overall goal should be to reach a fully-passing state on every rule. Cypress provides you a lot of flexibility to manage large amounts of accessibility violations and make incremental progress towards well-defined targets. Through a combination of fixing accessibility problems, and configuring Cypress to focus on a tight scope of rules, standards, and application areas that matter most to you, you might be closer than you think to a “clean” main branch, from which you can expand your standards over time. + +## Maintaining accessibility + +Once you reach the state of a report that contains no violations, your organization’s usage of Cypress Accessibility will change dramatically, and start to focus mainly on detecting and preventing accessibility problems before they merge. This can be a big adjustment. Chipping away at a large backlog of known issues, and fixing things rule-by-rule, provides a clear measure of progress to your team and a sense of momentum. It’s likely you will have been able to measure success and set goals in relation to the accessibility violations present on your main branch up to now. + +When those violations are all fixed, what exactly is Cypress Accessibility “doing” in your pipeline? And how should you measure success? + +### What Cypress Accessibility is doing when a project is in maintenance mode {#what-cypress-accessibility-is-doing-when-a-project-is-in-maintenance-mode} + +The short answer is: providing **pre-merge insights**, and **post-merge monitoring**, that supports you in maintaining accessibility. Accessibility debt tends to accumulate silently over time, leading to large, problematic backlogs that are difficult to work through, and land on your desk all at once after an audit. As you will have learned by going through the remediation process to get to a clean report in the first place: it’s not always easy to fix an already-inaccessible experience. Sometimes you have to start a whole feature or component over, because you’ve built multiple structures on top of an inaccessible foundation. + +Early feedback about accessibility on feature branches will allow developers to review small, precise reports, and get all the context they need to either fix issues immediately, or raise problematic patterns for wider discussion. This is especially helpful when a new team member joins, who isn’t as familiar with accessibility practices, or when a well-meaning full-stack developer – somebody who can “get by” in the front-end – unintentionally makes a minor change with major accessibility implications. + +Detecting problems before code merges allows the training, awareness-building, discussion, and remediation to all happen in the same cycle as regular code review, when everybody has all the context they need, not weeks or months later when it will be difficult to inject a ticket into the backlog to untangle what might have become a “load-bearing” accessibility problem. + +### How to measure success in maintenance mode {#how-to-measure-success-in-maintenance-mode} + +The issues surfaced in Cypress Accessibility should follow certain patterns, allowing you to get a good “pulse” on your team’s effectiveness with accessibility and time spent dealing with issues. The best-case scenario is that, with a stable team of front-end developers working on the product, the total number of violations on the main branch, as well as the number of violations identified and corrected on feature branches, should trend downwards over time. This driven by a buildup of education as a result of the continuous feedback. The presence of the feedback should _change developer behavior_ and truly help shift accessibility left, into questions that get asked and answered even before development, when a ticket is created or a design is handed off. At that point, mistakes are avoided, instead of corrected. + +We don’t expect most teams to reach a state where they always get everything right in the first code commit: there will always be some issues surfacing in Cypress Cloud – but as teams mature, it’s reasonable to expect the reports to contain less and less violations. This state is ideal \- your development teams should be focused on adding value to your company by making changes to the product. + +For most of you reading this, accessibility problems are quality issues, just like bugs and UI issues, and they are important and need to be addressed. This is why optimizing and minimizing the effort related to accessibility is so valuable in a literal sense \- your team is going to spend time on accessibility anyway, every cycle saved on dealing with accessibility is freed up for feature work, quality improvements, and other important activities. + +### Ideal conditions never last forever + +The main thing that would disrupt the downward trend of pre-merge violations is plain old reality. Teams change, priorities change, people move around an organization, they get hired, they leave, they go on vacation. All of this can be disruptive to an accessibility program – sometimes it’s a specific person or small group that is keeping standards up and providing guidance to their peers. When that person moves out of their team, a previously stable accessibility situation can start to regress. + +Cypress Accessibility will help you see this kind of disruption in real time and help you take action as needed. One of the biggest pain points that has been described to us by customers is unexpectedly having a large backlog of high-priority accessibility issues surfaced in an annual audit, even when they had fixed everything from the previous audit. This happens for a few reasons: sometimes the fixes introduce new problems, sometimes net-new work is added that includes new accessibility violations, or design changes lead to front-end modifications with unexpected accessibility implications. + +With Cypress Accessibility, you have an early warning system for these situations. They will show up as an increase in violations reported during code reviews, and there will be no chance for this accessibility debt to accumulate silently. Your team has an opportunity to “self-correct” when this happens, or if accessibility is disregarded and violations do get merged, you will be able to track that on your main branch. + +## Summary + +Accessibility is an ongoing process that affects the whole organization, not a single project of fixing a known list of issues and “completing” accessibility. The automation layer of accessibility tests offered by Cypress Accessibility maximizes the amount of information you can have at your fingertips to improve your accessibility status and maintain an accessible application over the long term, in a software development environment where both expected and unexpected changes create constant challenges for quality. + +Cypress Accessibility provides a consolidated report with page- and component-level breakdowns and interactive, visual DOM snapshots to understand exactly what every violation relates to in your application. With filtering and configuration you can create a manageable series of goals to solve every accessibility issue that Axe Core® can detect. Once your accessibility report is fully passing for the rules and pages that make sense for your business and project, Cypress Accessibility supports continuous feedback on new code changes before they merge, and gives you an early warning system if things are getting off track. + +The clarity and efficiency from using Cypress Accessibility will optimize your team’s time spent managing and fixing accessibility issues that are detectable by automation, helping them focus more on building great things that create value for your organization, without sacrificing this critical aspect of quality software development and building up debt over time to do so. diff --git a/docs/accessibility/guides/maximizing-coverage.mdx b/docs/accessibility/guides/maximizing-coverage.mdx new file mode 100644 index 0000000000..cb41b402f5 --- /dev/null +++ b/docs/accessibility/guides/maximizing-coverage.mdx @@ -0,0 +1,18 @@ +--- +title: Maximizing coverage +sidebar_position: 70 +--- + +As you gain more familiarity with the nature of accessibility violations and the underlying ideas behind them, you will see the connection between the generic, static checks offered by Axe Core®, and the specific user-facing behavior of your own applications. In many cases, you will be able to add new assertions to your tests, based on what you learned from Cypress Accessibility, but customized to your specific situation. + +An example of this is when Cypress Accessibility reports a button with a missing accessible name. The actual text that should go with the button depends upon your own application, and is unknown to Axe-Core, so a label check will pass for almost _any_ text that is placed in the button. After you’ve added the required text to the codebase, you can write an assertion about that specific text in your Cypress tests, to make clear to future developers that it is part of the spec of the application. That’s a small example, but it applies in many larger situations too. + +## Accessibility “hotspots” + +Another way to leverage the violations detected by Cypress Accessibility is to treat each one as a sign that accessibility in that area of the application may not have been considered when it was built. Because the whole DOM of the page or component is available for every violation, it’s faster to do a manual check of the patterns being used in the page around an Axe Core® violation. This can reveal accessibility “hotspots” containing usability issues that aren’t otherwise detectable by automation, but would appear in a manual audit. These can be cataloged and corrected, then regression tested with traditional Cypress automation. + +## Combining testing approaches + +Combining the always-on generic checks provided by Cypress Accessibility with regular Cypress assertions about specific details of your application can help accessibility quality stay high once things are fixed, regardless of whether problems were originally detected manually or automatically. This can take your accessibility confidence far beyond the 57% of issues detected, often cited when thinking about Axe Core® as a standalone library. + +In the testing context, you are already specifying user-facing aspects of your application, and that means almost any findings from manual regression checks or user feedback, can be explicitly covered by your standard testing approach. So the overall potential of your automation pipeline to provide good feedback about accessibility regressions is very high, if you leverage explicit assertions on top of the static check "error detection" approach to find mistakes. diff --git a/docs/accessibility/guides/production-monitoring.mdx b/docs/accessibility/guides/production-monitoring.mdx new file mode 100644 index 0000000000..786002f7b4 --- /dev/null +++ b/docs/accessibility/guides/production-monitoring.mdx @@ -0,0 +1,33 @@ +--- +title: Production monitoring +sidebar_position: 60 +--- + +Many organizations use Cypress automation in CI, and for local development, but also run scheduled tests against production or staging environments. This pattern works especially well for situations where a Content Management System is involved and content authors can add or modify things that affect the user interface, or anything else where the content being tested is not fully controlled by the testing environment. Since it can be impossible to write explicit assertions for ever-changing content like this, or for A/B tests of different UI variations implemented outside your codebase, production smoke tests tend to be more high-level health checks. + +Cypress Accessibility makes this kind of testing even more useful, as it will be able to run just fine against that dynamic set of content. Just by visiting a production URL in your Cypress tests, and taking one or two light actions in the UI, that page will be captured in your accessibility report, and alert you about content changes outside of your development lifecycle that introduce accessibility issues. + +One common approach to this kind of monitoring, if the project does not already have Cypress tests, is to use something like a site map, or just an array of target URLs, to quickly spin up a lot of coverage without writing new tests for every page, and then run that Cypress test on a schedule to get daily updated accessibility reports. Here is an example of doing this by requesting the `sitemap.xml` file from a website and scrolling to the footer on each page. + +```js +describe('accessibility scan', () => { + it('checks accessibility with the sitemap.xml', () => { + cy.request('https:///sitemap.xml').then((response) => { + const xmlString = response.body + const parser = new DOMParser() + const xmlDoc = parser.parseFromString(xmlString, 'application/xml') + const URLs = Array.from(xmlDoc.querySelectorAll('loc')).map( + (loc) => loc.textContent + ) + Cypress._.each(URLs, (URL) => { + it(`visits ${URL}`, () => { + cy.visit(URL) + cy.contains('').scrollIntoView() + }) + }) + }) + }) +}) +``` + +The outcome of this is a first-page-load accessibility report for every URL in the site. Any Cypress UI tests for specific workflows will increase the coverage area automatically to include the states and variations reached during the workflows. diff --git a/src/css/custom.scss b/src/css/custom.scss index 1f5a35d697..af00a65f71 100644 --- a/src/css/custom.scss +++ b/src/css/custom.scss @@ -135,6 +135,12 @@ // border color in current area of toc --toc-highlight: var(--ifm-color-primary); --ifm-alert-shadow: none; + + // override the default mapping of footer links to the `secondary` color, + // which is not good for dark backgrounds such as the footer + .footer--dark { + --ifm-footer-link-color: theme(colors.gray.50); + } } html[data-theme] { diff --git a/static/img/accessibility/guides/a11y-standards.png b/static/img/accessibility/guides/a11y-standards.png new file mode 100644 index 0000000000..41e6dec6dc Binary files /dev/null and b/static/img/accessibility/guides/a11y-standards.png differ diff --git a/static/img/accessibility/guides/github-actions-a11y-block.png b/static/img/accessibility/guides/github-actions-a11y-block.png new file mode 100644 index 0000000000..30e671f2c2 Binary files /dev/null and b/static/img/accessibility/guides/github-actions-a11y-block.png differ diff --git a/static/img/accessibility/guides/status-check.png b/static/img/accessibility/guides/status-check.png new file mode 100644 index 0000000000..946bbf5523 Binary files /dev/null and b/static/img/accessibility/guides/status-check.png differ