Using runOnly with tags for wcag21 results in passing tests with 0 violations

[tzbarkan]

If I use runOnly with tags for wcag2a and wcag2aa, I find violations as expected: cy.checkA11y(subject, { runOnly: { type: "tag", values: ["wcag2a", "wcag2aa"] } })

But if I use the tags wcag21a and wcag21aa as documented in axe-core 4.4.1, no violations are found: cy.checkA11y(subject, { runOnly: { type: "tag", values: ["wcag21a", "wcag21aa"] } })

Expectation: using wcag21a and wcag21aa should return a comparable number of violations. Most likely more.

Cypress 9.5.3 Cypress Axe 0.14.0 axe-core 4.4.1 with cypress-watch-and-reload 1.7.4

on Windows 10

[AndreaPontrandolfo]

If I use runOnly with tags for wcag2a and wcag2aa, I find violations as expected: cy.checkA11y(subject, { runOnly: { type: "tag", values: ["wcag2a", "wcag2aa"] } })

But if I use the tags wcag21a and wcag21aa as documented in axe-core 4.4.1, no violations are found: cy.checkA11y(subject, { runOnly: { type: "tag", values: ["wcag21a", "wcag21aa"] } })

Expectation: using wcag21a and wcag21aa should return a comparable number of violations. Most likely more.

Cypress 9.5.3 Cypress Axe 0.14.0 axe-core 4.4.1 with cypress-watch-and-reload 1.7.4

on Windows 10

I'm having a very similar issue. wcag2a level is the only one that works (it gives me a bunch of severity warnings). All the other levels that i tested, just passed without raising warnings.

Cypress 9.7.0 Cypress Axe 0.14.0 axe-core ^4.4.2 Windows 10

[bjornlauwerijs]

I am having the same issue. I expect a higher level to have more issues, but I get none on all tags with wcag21a, wcag21aa, wcag21aaa. I also got some strange results with running wcag2a compared to wcag2aa and wcag2aaa.

Screenshot below explains it better (test is always running on the same page but with different tags).

[nurulquamar]

I am also facing the same issue. wcag21a and wcag21aa are not finding any violations. But when I use wcag2a and wcag2aa I am able to. Also, these violations clearly mention that they are part of wcag21a as well.

axe-core: 4.4.3 cypress-axe: 1.0.0 cypress": 9.5.2 Mac OS 12.6

[nurulquamar]

@tzbarkan I opened an issue with axe-core and it was closed with a comment. See the below link if it helps: https://github.com/dequelabs/axe-core/issues/3760

[vladttt1123]

can someone tell me what is the benefit of using : values: ['wcag2a', 'wcag2aa'], If as per the documentation that I found:

The WCAG criteria are cumulative—to conform to WCAG Level AA, you must first conform to A. To conform to WCAG Level AAA accessibility criteria, you just first conform to A and AA.

So using only wcag2aa is sufficient, or am I missing something?