Provide detailed reports for wikis currently passing contrast checks
Closed, ResolvedPublic3 Estimated Story Points
Actions

Assigned To

Authored By

	ovasileva
	Jun 12 2024, 2:20 PM

Description

Background

Prior to wider dark mode deployment for desktop, we would like to deepen the accessibility checks we perform for wikis we

User story

As a designer, I want to understand the accessibility of dark mode across wikis so that I can gauge preparation for deployment

Requirements

Perform manual checks of the night mode checker with top 1000 articles: Wikis for which dark mode has fewer accessibility concerns than light mode according to https://night-mode-checker.wmcloud.org/

Array.from(document.querySelectorAll('.ontrack')).map((s)=>s.querySelector('a').getAttribute('href').split('-')[0])

Perform manual checks of the night mode checker with top 1000 articles: Wikis for which dark mode has up to 20% more errors than light mode according to https://night-mode-checker.wmcloud.org/
Publish results for discussion

BDD

For QA engineer to fill out

Test Steps

For QA engineer to fill out

Design

n/a

Acceptance criteria

see requirements

Communication criteria - does this need an announcement or discussion?

n/a

Rollback plan

n/a
This task was created by Version 1.0.0 of the Web team task template using phabulous

Related Objects
Search...

Status	Subtype	Assigned	Task
Resolved		ovasileva	T341631 [EPIC] Q1 Main hypothesis: Typographical and palette customizations
Resolved		ovasileva	T341632 [EPIC] Palette customizations and dark mode
Resolved	Feature	ovasileva	T26070 [GOAL] Provide a dark / night mode skin or theme
Resolved		ovasileva	T367151 [Config] Deploy dark mode to all users in tier 1 wikis on the Minerva skin
Resolved		ovasileva	T367150 Deploy dark mode to logged-out users in tier 1 and 2 wikis on the Vector2022 and Minerva skin
Resolved		ovasileva	T367310 Provide detailed reports for wikis currently passing contrast checks

Event Timeline

ovasileva created this task.Jun 12 2024, 2:20 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 12 2024, 2:20 PM

ovasileva triaged this task as High priority.Jun 12 2024, 2:20 PM

JScherer-WMF awarded a token.Jun 12 2024, 4:39 PM

It would be nice if we could configure the contrast checker to "ignore" elements that aren't text or icons. This would exclude things like borders from our counts because they don't need to meet the same contrast threshold as interactive elements.

The checks should only be on text right now. We unfortunately cannot test for icons with this tool.

For the following lists of wikis, perform manual checks of the night mode checker with 1000 articles:

What wikis are we wanting to test on - there is no list.

Wikis for which the difference between dark and light mode is less than 20%

This probably should be qualified in terms of percentage increase/decrease. Does this mean we see at least a 20% decrease in accessibility issues when switching from light to dark theme?

Jdlrobson assigned this task to ovasileva.Jun 12 2024, 8:54 PM

Jdlrobson moved this task from Backlog to Dark mode on the FY2023-24-WE 2.1 Typography and palette customizations board.Jun 13 2024, 12:58 AM

In T367310#9886701, @Jdlrobson wrote:

For the following lists of wikis, perform manual checks of the night mode checker with 1000 articles:

What wikis are we wanting to test on - there is no list.

The list would depend on the day we test (since we might see some improvements between now and next week when we start testing). Right now the requirement is "Wikis for which dark mode has fewer accessibility concerns" based on https://night-mode-checker.wmcloud.org/ - I'll add a link to the checker to make this clearer

Wikis for which the difference between dark and light mode is less than 20%

This probably should be qualified in terms of percentage increase/decrease. Does this mean we see at least a 20% decrease in accessibility issues when switching from light to dark theme?

A couple of options:

Wikis for which the difference in the number of errors between dark and light mode is less than 20% of the total number of errors
Wikis for which dark mode has up to 20% more errors than light mode according to https://night-mode-checker.wmcloud.org/ (I think this definition is a little bit clearer - I'll add it)

ovasileva updated the task description. (Show Details)Jun 13 2024, 1:04 PM

ovasileva updated the task description. (Show Details)Jun 13 2024, 5:24 PM

ovasileva updated the task description. (Show Details)

Jdlrobson updated the task description. (Show Details)Jun 13 2024, 5:26 PM

Jdlrobson set the point value for this task to 3.Jun 13 2024, 5:32 PM

Jdlrobson moved this task from Needs analysis to Ready for estimation on the Web-Team-Backlog-Archived (FY2023-24 Q4 Sprint 6) board.

Jdlrobson raised the priority of this task from High to Needs Triage.Jun 13 2024, 5:41 PM

Jdlrobson triaged this task as High priority.

Jdlrobson moved this task from Ready for estimation to Ready for Development on the Web-Team-Backlog-Archived (FY2023-24 Q4 Sprint 6) board.

ovasileva added parent tasks: T367150: Deploy dark mode to logged-out users in tier 1 and 2 wikis on the Vector2022 and Minerva skin, T367151: [Config] Deploy dark mode to all users in tier 1 wikis on the Minerva skin .Jun 17 2024, 12:39 PM

KSarabia-WMF claimed this task.Jun 17 2024, 6:49 PM

KSarabia-WMF moved this task from Ready for Development to Design Review on the Web-Team-Backlog-Archived (FY2023-24 Q4 Sprint 6) board.

KSarabia-WMF moved this task from Design Review to Doing on the Web-Team-Backlog-Archived (FY2023-24 Q4 Sprint 6) board.

Jdlrobson mentioned this in T367151: [Config] Deploy dark mode to all users in tier 1 wikis on the Minerva skin .Jun 18 2024, 5:15 PM

A blocker for the upcoming Minerva deployment

Jdlrobson lowered the priority of this task from High to Medium.Jun 20 2024, 4:55 PM

We do not currently have a way to automate this. Since manual checks take a significant amount of time, we decided to proceed with 500 articles instead of 1000.

Discussed in standup. We will be making the deploy decisions on which deployment tier (logged-out and logged-in deploy or logged-in only deploy) wikis will get on dark mode based on the results for desktop only. We will compare these to the automated results from Minerva for the top 50 articles, but will make the decision based on desktop so wikis have dark mode consistently across wiki on both desktop and mobile

Work in progress doc: https://docs.google.com/document/d/1VCbIZTUhwk2-3lJIDbMVkCWF4GRDjEFoNArj-Kr_eFM/edit#heading=h.5u964xm5qjmg

Jdlrobson mentioned this in T367150: Deploy dark mode to logged-out users in tier 1 and 2 wikis on the Vector2022 and Minerva skin.Jun 26 2024, 11:44 PM

Jdrewniak reassigned this task from KSarabia-WMF to ovasileva.Jun 27 2024, 5:04 PM

Jdrewniak moved this task from Doing to Ready for Signoff on the Web-Team-Backlog-Archived (FY2023-24 Q4 Sprint 6) board.

Jdrewniak added a subscriber: KSarabia-WMF.

Thank you @KSarabia-WMF - I've updated the deployment list on T367151: [Config] Deploy dark mode to all users in tier 1 wikis on the Minerva skin . We're ready to close this one.

Jdlrobson mentioned this in T369656: Re-evaluate tier 1 and 2 wikis.Jul 9 2024, 6:24 PM