Explore pointing Pixel to the standalone Codex sandbox
Closed, ResolvedPublic5 Estimated Story Points
Actions

Assigned To

Authored By

	• Catrope
	Sep 21 2023, 6:32 PM

Description

In addition to the existing Special:VueTest-based visual regression test that tests Codex in an MW environment, we would like to also be able to run visual regression tests on the standalone Codex sandbox.

I can think of two different ways to do this, but maybe there are more:

Have Pixel download the main branch of Codex, build its sandbox, and point Pixel to sandbox/index.html (as well as the other subpages of the sandbox)
Have the npm run doc command build the sandbox and put it in the output directory under /sandbox, similar to what npm run branch-deploy does. That would make the sandbox available at https://doc.wikimedia.org/codex/main/sandbox , and we could then point Pixel to that URL

Details

	Subject	Repo	Branch	Lines +/-
	Update Codex from v1.3.2 to v1.3.3	mediawiki/core	master	+1 K -164
	build: Respect CODEX_DOC_ROOT when building the sandbox	design/codex	main	+2 -1

Customize query in gerrit

Related Objects

Mentioned In: T357185: Set up Pixel testing for the Codex sandbox
Mentioned Here: T357185: Set up Pixel testing for the Codex sandbox

Event Timeline

• Catrope created this task.Sep 21 2023, 6:32 PM

Restricted Application added a project: Design-System-Team. · View Herald TranscriptSep 21 2023, 6:32 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

CCiufo-WMF moved this task from Inbox to Needs Refinement on the Design-System-Team board.Sep 25 2023, 3:41 PM

CCiufo-WMF moved this task from Needs Refinement to Backlog on the Design-System-Team board.Sep 29 2023, 3:19 PM

AnneT merged a task: T291525: Consider automated visual regression testing in the new Vue component library.Oct 26 2023, 8:08 PM

AnneT added subscribers: AnneT, Michael, • EUdoh-WMF and 3 others.

bmartinezcalvo set the point value for this task to 5.Jan 22 2024, 6:29 PM

bmartinezcalvo edited projects, added Design-System-Team (DST-Sprint-15 (2024-01-22 to 2024-02-02)); removed Design-System-Team.

The Pixel repo is here: https://github.com/wikimedia/pixel . The configuration for Codex also lives in that repo: https://github.com/wikimedia/pixel/blob/f6bff7f83e4bce782fda37a5bc17921195ffa78d/configCodex.js

CCiufo-WMF triaged this task as Medium priority.Jan 22 2024, 6:41 PM

Change 994351 had a related patch set uploaded (by Catrope; author: Catrope):

[design/codex@main] build: Respect CODEX_DOC_ROOT when building the sandbox

https://gerrit.wikimedia.org/r/994351

gerritbot added a project: Patch-For-Review.Jan 30 2024, 11:35 PM

I got something sort of working. It needs more work, but here's what I have for now: https://github.com/catrope/pixel/commit/fc79f446ad734e49be0d4245cf1a825aeae85bf6

Instructions for how to use this proof of concept:

Install Pixel locally
Check out https://github.com/catrope/pixel/commit/fc79f446ad734e49be0d4245cf1a825aeae85bf6 in your Pixel repo
./pixel.js clean
./pixel.js reference -g codex -c 997544 (until https://gerrit.wikimedia.org/r/c/997544 is merged, this change is necessary to make things work; after that is merged, omit the -c 997544 part)
./pixel.js test -g codex -c NNNNNN (number of the change you want to test; for just testing the Pixel setup you can use https://gerrit.wikimedia.org/r/c/994352 . Until 997544 is merged, the tested change has to be rebased on top of 997544)
View /path/to/pixel/report/codex/index.html in your browser

Change 997544 had a related patch set uploaded (by Catrope; author: Catrope):

[design/codex@main] build: Automatically add all demos/*.html files to the sandbox build

https://gerrit.wikimedia.org/r/997544

Change 994351 merged by jenkins-bot:

[design/codex@main] build: Respect CODEX_DOC_ROOT when building the sandbox

https://gerrit.wikimedia.org/r/994351

Remaining issues with this proof of concept:

-b latest-release doesn't work, because it attempts to check out the wmf/1.42.0-wmf.17 branch in the Codex repo, which doesn't exist. We want latest-release to mean the latest Codex release instead.
As a consequence, runAll doesn't work, which is why this needs to be fixed before this PR can be merged.

Areas for improvement:

Special treatment for the Codex repo is currently hard-coded with if ( repoId === 'design/codex' ), which is not ideal
We still have to hard-code the list of components in configCodex.js; it would be nicer (and more resilient) if we didn't have to do this, and we instead got the list of components from Codex itself somehow

This task was filed as an exploration, and that exploration is now done. I've filed T357185: Set up Pixel testing for the Codex sandbox for productionizing the locally built sandbox approach.

Change 1005171 had a related patch set uploaded (by VolkerE; author: VolkerE):

[mediawiki/core@master] Update Codex from v1.3.2 to v1.3.3

https://gerrit.wikimedia.org/r/1005171

Change 1005171 merged by jenkins-bot: