The wikis banwiki, bjnwiki, frwiki, kowiki,pawiki, tumwiki were descoped from our deploy as we get inconsistent results in desktop and mobile.
Understand why the color contrast checker gives different results for mobile and desktop
Description
Description
Related Objects
Related Objects
- Mentioned In
- T370099: Roll out dark mode to all projects (non-Wikipedia sites, logged-in users)
- Mentioned Here
- T370099: Roll out dark mode to all projects (non-Wikipedia sites, logged-in users)
T369667: Date contrast color in dark mode of talk pages
T367092: Inconsistent views for contents within <dl> tag between desktop and mobile view
T369656: Re-evaluate tier 1 and 2 wikis
Event Timeline
Comment Actions
Thanks @KSarabia-WMF for looking at this. Feel free to move this to sign off and to assign to me once you've had a chance to update T369656 with any insights based on this analysis (e.g. use mobile instead of desktop for testing).
Comment Actions
Moving to signoff @Jdrewniak will summarize spike outcomes and we will proceed with the updated list in T367092: Inconsistent views for contents within <dl> tag between desktop and mobile view
Comment Actions
During this investigation, the following improvements were made:
- Added screenshot capabilities to the contrast checker to to visually debug issues
- Limit the axe core library to only run the color contrast test
- Diagnosing the performance issues when running the contrast checker locally
This work was somewhat tangential to the question at hand, and took up the majority of time for this investigation, but it enabled us to see what might be the cause of these bugs.
Misfires
Running the contrast checker locally on
https://ami.m.wikipedia.org/wiki/Sa%E2%80%99ayayaw_pising_no_tyin-naw flags the following errors:
 |  |
| light mode | dark mode |
Although the contrast checker is running once in light mode and a second time in dark mode (as visible in the screenshots) it still calculates the same colors values in both modes:
Fix any of the following: Element has insufficient color contrast of 1.61 (foreground color: #ffffff, background color: #c8ccd1, font size: 9.8pt (13px), font weight: bold). Expected contrast ratio of 4.5:1
I don't know what the cause of this error could be, but it could require further investigation.
Small font-size
As @KSarabia-WMF discovered, the WCAG rating take into account font-size when evaluating color contrast. This could effect elements that have small text.
The official axe core contrast checker can be used online to manually evaluate this issue: https://dequeuniversity.com/rules/axe/4.1/color-contrast
UI related bugs vs. template bugs
Many UI related bugs have gotten caught in the contrast checker. Notably, the dark-mode date color issue T369667 has been responsible for a lot of the errors flagged by this tool.
Summary
I'd recommend further investigation into why the tool looks like it's calculating the light-mode colors in dark-mode (sometimes). As to whether mobile vs desktop is more reliable, it seems most of the errors are legitimate on both platforms, just different than those we were expecting. Since mobile has a larger default font-size though, it might be better to use that instead of desktop for errors that are related specifically to color.
Comment Actions
Thanks @Jdrewniak seems clear to me!
Given we are wrapping up this project I'm reluctant for us to do further investigation but I'm noting this in T370099 for when we do our next big roll out.
Looks like T369656 has been updated per the recommendation.
Comment Actions
💡One more idea! (that's pretty simple to implement).
Since this tool will be focused on helping editors fix template issues going forward, we can also limit the scan to just the #mw-content-text or .mw-parser-output element so that it doesn't catch other UI related issues. (It would still be good to know what those are though).