Impact of requirements and assertions on product accessibility

[rachaelbradley]

From @ChrisLoiselle:

From Yesterday's AG meeting: Loiselle on Alastair's point on Detlev's point:

1. Regarding prerequisites and baseline, prerequisites and baseline may have tangentially related assertions. If they are weighted equally, differently or at all, that would be part of the assertation or VPAT/ACR and/or accessibility statement?
2. Who audits the assertions in your requirements, are they one in the same? Are we meant to do that differently?
3. Are firms and companies now going to validate their product / services via an VPAT/ACR and then their assertions?
4. Are assertions part of the accessibility statement provided within a given VPAT/ACR? Are they to live somewhere else and be related to but not part of a VPAT/ACR?
5. If you've done your base requirements or whatever we end up calling that, do we have to also assert that we've done that or is that automatically met as an assertion based on the fact you are reporting and stating you have passed/met the base requirements?

This discussion is related to #112 as well and my comment within that PR, regarding prerequisites, which I believe are now part of baseline per the above.

The defined term of prerequisite is either a thing that is required as a prior condition for something else to happen or exist. or required as a prior condition.

How does this differ from must, should, could A, AA, and AAA, for example, when comparing the Baseline, Enhanced and PreReq?

1. Are PreReqs a phase 0? Seems like they are now part of Baseline.
2. Level 1 failing to meet baseline would fail to meet PreReqs? Seems like both are part of baseline.
3. In conformance conversations 2024 (slide deck) - baseline is defined as A core set of required outcomes that must be met before scoring occurs. Are you stating that you are at Level 1 (baseline) and fail baseline, but haven't met any prior level (0) that may include some other baselines and prerequisites? Am I mis-reading this?
4. Baselines are required before scoring can occur, but prerequisites are required before baselines are required? I know this is more out of terminology and gets in to conformance, which is a separate discussion/issue/PR, so if you have a link to that, I'll read through those if we have an open issue tracking this.

Capturing this here in case it prompts thoughts on the subject.

[rachaelbradley]

From @GreggVan

My thoughts

From Yesterday's AG meeting: Loiselle on Alastair's point on Detlev's point:

Regarding prerequisites and baseline, prerequisites and baseline may have tangentially related assertions. If they are weighted equally, differently or at all, that would be part of the assertation or VPAT/ACR and/or accessibility statement?

yes assertions will vary widely -- from specific (we do the following when creating alt text) to global (we do user testing)

They are all assertions -- and if we try to do anything other than treat them equally -- we will spend the next 8 years arguing weights for each I am afraid.

Who audits the assertions in your requirements, are they one in the same? Are we meant to do that differently?

You don't audit assertions. The only test of an assertion is that it was made. And if you have it in front of you to test -- it must have been made.

Are firms and companies now going to validate their product / services via an VPAT/ACR and then their assertions?

You don't validate an assertion. You just make it. (or read it). By their very nature, assertions are not auditable.

Are assertions part of the accessibility statement provided within a given VPAT/ACR? Are they to live somewhere else and be related to but not part of a VPAT/ACR?

The VPAT and ACR seem like good places to list assertions. Esp if VPAT is public. But that is outside of our role as RULER maker.

We simply define what accessible is (or minimum accessibility). We don't require reporting 0r ACRs or VPATs etc.

If you've done your base requirements or whatever we end up calling that, do we have to also assert that we've done that or is that automatically met as an assertion based on the fact you are reporting and stating you have passed/met the base requirements?

Think RULER -- not rule. If you say that x mm is strong enough for a beam used in a ceiling with a span of 12 feet -- that is a RULER. Whether you do that or report it or write it on the Deed -- is a rule or regulation. We just say what is accessible (or min accessibility). our job ends there.

This discussion is related to #112 as well and my comment within that PR, regarding prerequisites, which I believe are now part of baseline per the above.

The defined term of prerequisite is either a thing that is required as a prior condition for something else to happen or exist. or required as a prior condition.

I find the "precondition' language to be confusing. if require it should just be called required.

How does this differ from must, should, could A, AA, and AAA, for example, when comparing the Baseline, Enhanced and PreReq?

A AA AAA are not must should could

All three are required for that level of conformance. Since only SHALLS contribute to conformance - all are shalls - if you like. But AAA has not been required outside of LEVEL AAA conformance.

Are PreReqs a phase 0? Seems like they are now part of Baseline.
Level 1 failing to meet baseline would fail to meet PreReqs? Seems like both are part of baseline.

I believe these stack. you can't do a higher level without first doing the lower level

In conformance conversations 2024 (slide deck) - baseline is defined as A core set of required outcomes that must be met before scoring occurs. Are you stating that you are at Level 1 (baseline) and fail baseline, but haven't met any prior level (0) that may include some other baselines and prerequisites? Am I mis-reading this?

See above. you can't pass 1 without first passing 0 - in your example

Baselines are required before scoring can occur, but prerequisites are required before baselines are required? I know this is more out of terminology and gets in to conformance, which is a separate discussion/issue/PR, so if you have a link to that, I'll read through those if we have an open issue tracking this.

I agree. if you arent going to stop at prereqs (and I hope not) then they are just requirements for the next level. no reason to complicate it by making them their own level

Capturing this here in case it prompts thoughts on the subject.

[ChrisLoiselle]

Hi, if a company asserts they did something, but a customer can prove they didn't (filing a bug on a workflow or product), then I'd say what they reviewed and found was a result of testing. For example, Company A - I assert that all informative images have alt text. Customer Y - Image 2, on page B doesn't. Dear Company A, your assertion is false per my testing of x, y, z.

[ChrisLoiselle]

To me, the terminology definition(s) impacts conformance and what would conform per what terminology is used. The discussion name should be changed to align more with conformance and terminology, it is not so much about third party, but what is testable, where it is located in the structure we are building and what weight it provides.

In what has been proposed, it seems that assertions can but also may not relate to requirements but are more like cousins to a parent child relationship of the current structure being brought forward in the larger discussions in AG.

Thus, If I'm following the logic of assertions - I could ask a company about their hypothetical assertions, which reads as a cousin to but not a brother or sister to requirements. That assertion from Company Y could say "Yes, we have a design guide, we note proper use of alt text on page 100-102". That statement could well be true if they do mention it on page 100-102.

If they don't actually follow it or have it listed on those pages, that would be an issue, but is that an accessibility issue / bug or a problem with their design guide?

Does the end user / customer reach out to support on the design guide but not the product? Are you stating documentation (assertions) are one in the same as product?

What if the company / product says they follow the design guide, but have bugs listed about alt text in their baseline/requirements / VPAT/ACR output available to a customer ? Does a bug within their baseline/requirements testing showcase this?

Did they miss it with code inspection coverage / testing? If they don't mention it on a VPAT/ACR per baselines tested for, but they are asserting that they do have a design guide and they do follow it but a customer finds out in product that they don't really follow it, how is that ranked within the overall scheme of baseline, requirements, assertions?

What if the design guide is written incorrectly ? Yes they have a design guide, but in turn they are following wrong "recommendations" and "style guide processes" to mark up an image with alt text appropriately? Is the assertion and baseline test now broken and are they (the company/product) , required to update both?