Maniphest T305549

Collect analytics for theme issues across different wikis which uses specific templates. Add performance tests.
Closed, ResolvedPublic
Actions

Assigned To

vadim-kovalenko

Authored By

	vadim-kovalenko
	Apr 6 2022, 1:15 PM

Tags

Referenced Files

	F35158036: pagesWithDiffIssues.csv
	May 20 2022, 2:39 PM

Subscribers

View All 10 Subscribers

Description

Create a service for analyzing theme issues using a headless browser (puppeteer). Deploy it to the Toolforge, and prepare basic statistics. There are at least 4 templates that cause most theme issues:

Calculate the contrast ratio for these templates in articles in en and fr wikis with the default/dark/black/sepia themes.
Statistics to collect: parse 500 articles per request for each template multiplying on 4 theme options.
Add performance tests to calculate the min/max/average/median time of the theme issue processing.

UPD:
Instead of the puppeteer use pa11y since it has an in-build API for user accessibility calculation.
Repository - https://github.com/VadimKovalenkoSNF/wmf-theme-issues-analytics

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Open		None	T286643 [EPIC] Systematic solution on preventing rendering issues on user styles between dark/light themes
		Resolved		vadim-kovalenko	T305549 Collect analytics for theme issues across different wikis which uses specific templates. Add performance tests.
		Resolved		vadim-kovalenko	T307783 Generate dataset of pages to check for colour contrast issues
		Resolved		vadim-kovalenko	T307784 Identify which template is used for each test run

Event Timeline

vadim-kovalenko created this task.Apr 6 2022, 1:15 PM

vadim-kovalenko changed the task status from Open to In Progress.

vadim-kovalenko claimed this task.

• JMinor moved this task from Needs Triage to Tracking on the Wikipedia-iOS-App-Backlog board.Apr 11 2022, 7:00 PM

• MattCleinman moved this task from Needs Triage to Tracking on the Wikipedia-Android-App-Backlog board.Apr 21 2022, 4:20 PM

Jgiannelos moved this task from Needs triage to Kanban on the Product-Infrastructure-Team-Backlog-Deprecated board.Apr 29 2022, 12:14 PM

Jgiannelos edited projects, added Product-Infrastructure-Team-Backlog-Deprecated (Kanban); removed Product-Infrastructure-Team-Backlog-Deprecated.

Related repo with analytics functionality - https://github.com/VadimKovalenkoSNF/wmf-theme-issues-analytics
cc: @Jgiannelos

vadim-kovalenko updated the task description. (Show Details)Apr 29 2022, 2:50 PM

Jgiannelos moved this task from To Do to Doing on the Product-Infrastructure-Team-Backlog-Deprecated (Kanban) board.May 5 2022, 4:42 PM

Jgiannelos closed subtask T307783: Generate dataset of pages to check for colour contrast issues as Resolved.May 19 2022, 11:14 AM

Here is the report for the readability issues after testing all template testcase pages for en.wikipedia.org:

From all the tested templates ~20% had readability issues
This means around 700 templates are rendered with dark theme issues
Here is an extensive report of all the tests with readability issues

pagesWithDiffIssues.csv258 KBDownload

Jgiannelos closed this task as Resolved.May 20 2022, 2:44 PM

Jgiannelos closed subtask T307784: Identify which template is used for each test run as Resolved.May 25 2022, 12:52 PM