Accessibility checker contrast check inconsistent with Google's Lighthouse

Developer Tools
#1

hey there!

I just had a first play with Firefox’s accessibility tools and it seems very powerful, thanks for making it available.

I want to report a compat issue. Unfortunately, this concerns an internal dev link that I cannot share, but the situation is simple to explain: the text in question is 14px in size. Arguably this is relatively small, yet it’s UI text, navigation links and such on various colored backgrounds.

All of these texts/links have a contrast ratio somewhere between 3-4.

Firefox’s accessibility checker reports these as failures, mentioning 4.5 is required. Google Lighthouse’s accessibility test accepts them as valid.

Both tools claim to test for the same WCAG success criteria, yet have completely different outcomes. This is confusing to developers.

I don’t know which is right or wrong, but would be good to clarify these differences.

1 Like
#2

14px text is not considered “large text” by WCAG 2, so the target contrast ratio should be 4.5:1. If the contrast is indeed in the 3–4 range (below 4.5), then Firefox is right and Lighthouse is wrong.

WCAG 2 defines large text as either:

  • regular text that is 18pt larger
  • bold text that is 14pt or larger

Maybe some of the text that Lighthouse does not flag is bold?
Lighthouse would still be wrong here, because WCAG makes it clear that it uses typographic points, which do not translate 1:1 to CSS pixels:

What’s ‘large text’ in WCAG 2.0 parlance?

For many mainstream body text fonts, 14 and 18 point is roughly equivalent to 18.75px and 24px or 1.2 and 1.5 em or to 120% or 150% of the default size for body text (assuming that the body font is 100%)

1 Like
#3

Since the page is private, could you create a small test case with a couple paragraphs of text using similar styles and colors, maybe on Codepen or somewhere else, and which gives similar results in Firefox and Lighthouse?

1 Like
#4

Thanks. I’ve seen the definition for large text before, Google also refers to it. Strange that it still somehow passes in my example.

I fully believe your answer to be correct, I’m just wondering why I get different outputs. Even if you consider LH wrong, know that the built-in Hint tool of Firefox also passes the contrast check. So not even in the same browser its consistent.

1 Like
#5

Would be good to have screenshots at least, and ideally a reduced test case.

1 Like
#6

I can actually just give you the live link if you’re able to look at it in the short term:

It’s a work in progress site. Yet it’s on a VM from my desktop so it’s often down. Also, I’m actively working on it, so things will change.

Let me know if this works for you. I’ll remove the link when you had a look.

1 Like
#7

Okay, I think I see what’s going on.
You have white text on green gradients in a few places: in the header, and in the “Discover Biodiversity” footer, and maybe other subtle gradients in other places.

The Accessibility panel in Firefox DevTools is great at finding out contrast ranges for text on background images and gradients, and works by screenshotting the text node with the text made invisible, and comparing the text color to each pixel in that screenshot. This can lead to some false positives if you have a couple pixels near to some letters that are statistical outliers (much darker/lighter than the rest of the background). But usually it’s a fair representation of the actual contrast of this text on top of that background-image or gradient background.

So the Accessibility inspector will show a range like 3.57—4.12 (the whole range fails the 4.5 requirement) or 4.13—5.10 (part of the range fails the 4.5 requirement).

If you’ve found instances where the Accessibility inspector shows one result, and the Inspector’s color picker shows a different result, do tell, because they should be identical. I haven’t found one yet, but there are situations where you can have different results because you actually are getting a different display:

  1. If you use the accessibility inspector’s “Picker” mode and hover a text node in the header, you will probably get your “hover” style which adds a lighter radial gradient below the hovered element. So your contrast range will be lower (in the 2-3 range).
  2. If you select this text node in the Inspector, and use the color picker when editing a color property in the Rules tab for instance, you don’t have a “hover” style displayed so your contrast range will be higher (in the 3-4 range).

Finally, why is Lighthouse passing here? Well, Lighthouse’s contrast check is more primitive and only checks CSS values, not the actual rendering. It works for text on top of solid colors (background-color), but usually not for text on top of background images, gradients, or text with a transparent background that is absolutely positioned on top of a different element (that is not its ancestor, e.g. a sibling). Not 100% sure what algorithm Lighthouse uses, I think they may be using the aXe engine (same results as the aXe extension, which is good and useful by the way).

Basically, Lighthouse/aXe will not detect some contrast issues that should not pass.
Firefox’s Accessibility inspector as its limits too (e.g. when you use some specific CSS filters or text with strokes or “gradient text” effects), but detects more issues.

1 Like
#8

Thanks so much for the quick checking and the extensive reply, it clears up a lot of things.

I was wondering about the gradients earlier, played around with values last night and concluded that Firefox does sample them the best it can. Yet with your help I now know much more about the complexities of it and the differences between tools.

It also means that the interpretation of what is normal/large text does not seem to differ between tools, it’s mostly a difference in how reliable contrast is detected.

A personal thought is that either 18pt/3 or 14pt/4.5 truly is a very harsh requirement for UI text (not body copy or header text), but it is what it is. Life pro tip: don’t ever use green as a brand color :slight_smile:

Thanks again for your help, will remove that link now.