It would be nice if we could configure the contrast checker to "ignore" elements that aren't text or icons. This would exclude things like borders from our counts because they don't need to meet the same contrast threshold as interactive elements.

The checks should only be on text right now. We unfortunately cannot test for icons with this tool.

For the following lists of wikis, perform manual checks of the night mode checker with 1000 articles:

What wikis are we wanting to test on - there is no list.

Wikis for which the difference between dark and light mode is less than 20%

This probably should be qualified in terms of percentage increase/decrease. Does this mean we see at least a 20% decrease in accessibility issues when switching from light to dark theme?

In T367310#9886701, @Jdlrobson wrote:

For the following lists of wikis, perform manual checks of the night mode checker with 1000 articles:

What wikis are we wanting to test on - there is no list.

The list would depend on the day we test (since we might see some improvements between now and next week when we start testing). Right now the requirement is "Wikis for which dark mode has fewer accessibility concerns" based on https://night-mode-checker.wmcloud.org/ - I'll add a link to the checker to make this clearer

Wikis for which the difference between dark and light mode is less than 20%

This probably should be qualified in terms of percentage increase/decrease. Does this mean we see at least a 20% decrease in accessibility issues when switching from light to dark theme?

A couple of options:

• Wikis for which the difference in the number of errors between dark and light mode is less than 20% of the total number of errors
• Wikis for which dark mode has up to 20% more errors than light mode according to https://night-mode-checker.wmcloud.org/ (I think this definition is a little bit clearer - I'll add it)