Page MenuHomePhabricator

mw-parsoid improvements
Open, In Progress, MediumPublic

Description

After completing T386246: Migrate parsoidtest functionality to kubernetes (mw-parsoid), we can continue with mw-parsoid improvements:

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
C. Scott Ananianmediawiki/vendorwmf/1.47.0-wmf.8+414 -121
C. Scott Ananianmediawiki/vendormaster+414 -121
Jgiannelosmediawiki/services/parsoidmaster+23 -76
Jgiannelosmediawiki/services/parsoidmaster+41 -37
C. Scott Ananianmediawiki/vendorwmf/1.47.0-wmf.6+861 -183
C. Scott Ananianmediawiki/vendormaster+861 -183
Jgiannelosmediawiki/services/parsoidmaster+5 -0
Arlolramediawiki/vendorwmf/1.47.0-wmf.4+61 -34
Arlolramediawiki/vendormaster+61 -34
Jgiannelosmediawiki/services/parsoidmaster+15 -2
C. Scott Ananianmediawiki/vendorwmf/1.47.0-wmf.2+344 -157
Subramanya Sastrymediawiki/vendormaster+344 -157
Jgiannelosmediawiki/services/parsoidmaster+43 -0
C. Scott Ananianmediawiki/vendorwmf/1.47.0-wmf.1+14 K -14 K
C. Scott Ananianmediawiki/vendormaster+14 K -14 K
Jgiannelosmediawiki/services/parsoidmaster+83 -24
Effie Mouzelioperations/deployment-chartsmaster+1 -0
Effie Mouzelioperations/deployment-chartsmaster+3 -2
Show related patches Customize query in gerrit

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Mentioned in SAL (#wikimedia-operations) [2026-05-07T21:15:34Z] <cscott@deploy1003> Started scap sync-world: Backport for [[gerrit:1284828|Upgrading webonyx/graphql-php (v15.31.5 => v15.32.3)]], [[gerrit:1284834|composer.json: Update webonyx/graphql-php to ^15.32.3]], [[gerrit:1284832|Bump wikimedia/parsoid to 0.24.0-a2 (T319058 T368724 T373384 T420336 T423241 T423701 T424446 T424773 T425008 T425056 T425107 T425731)]], [[gerrit:1284837|Bump wikimedia/parsoid to 0.24.0-a2 (T425731)]

Mentioned in SAL (#wikimedia-operations) [2026-05-07T21:27:35Z] <cscott@deploy1003> Finished scap sync-world: Backport for [[gerrit:1284828|Upgrading webonyx/graphql-php (v15.31.5 => v15.32.3)]], [[gerrit:1284834|composer.json: Update webonyx/graphql-php to ^15.32.3]], [[gerrit:1284832|Bump wikimedia/parsoid to 0.24.0-a2 (T319058 T368724 T373384 T420336 T423241 T423701 T424446 T424773 T425008 T425056 T425107 T425731)]], [[gerrit:1284837|Bump wikimedia/parsoid to 0.24.0-a2 (T425731)

Here are some docs about the current status of the migration:
https://www.mediawiki.org/wiki/Parsoid/Round-trip_testing#Migration_steps

the tl;dr is:

  • Our tooling (rt-testing orchestration) scripts work with a bunch of CLI args overriden
  • A testreduce instance in wikitextexp cloudvps project is working against mw-parsoid
  • Devs can get shell access to the running container instance via a shell script
  • Devs can fetch files from the running container instance via a shell script

TODO:

  • Keep both envs running at the same time for a few weeks to see if the results are the same
  • Formalize where the scripts [1][2] live so they can be properly maintained given that they are part of our regular chores (cc @jijiki)

Overall I think we are very close to having parity on what we need from the new setup to be able to decommission the old nodes.

[1] https://phabricator.wikimedia.org/P84273
[2] https://phabricator.wikimedia.org/P92440

For the last part, from a practical point of view its as easy as adding in RTTestSettings:

$wgCachePrefix="mw-parsoid"
$wgParserCacheExpireTime = 43200;

I am curious if this might have other consequences though.

Adding Data-Persistence in case folks have some input on that

cc @Ladsgroup

Change #1282990 merged by jenkins-bot:

[mediawiki/services/parsoid@master] rt-testing: Add an optional step to deploy mw-parsoid latest on k8s

https://gerrit.wikimedia.org/r/1282990

Change #1285896 had a related patch set uploaded (by Subramanya Sastry; author: Subramanya Sastry):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a3

https://gerrit.wikimedia.org/r/1285896

For the last part, from a practical point of view its as easy as adding in RTTestSettings:

$wgCachePrefix="mw-parsoid"
$wgParserCacheExpireTime = 43200;

I am curious if this might have other consequences though.

Adding Data-Persistence in case folks have some input on that

cc @Ladsgroup

I assume it's for the test setup? it should be fine if it's for the test infra.

Change #1285896 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a3

https://gerrit.wikimedia.org/r/1285896

Change #1286484 had a related patch set uploaded (by C. Scott Ananian; author: Subramanya Sastry):

[mediawiki/vendor@wmf/1.47.0-wmf.2] Bump wikimedia/parsoid to 0.24.0-a3

https://gerrit.wikimedia.org/r/1286484

Change #1286484 merged by jenkins-bot:

[mediawiki/vendor@wmf/1.47.0-wmf.2] Bump wikimedia/parsoid to 0.24.0-a3

https://gerrit.wikimedia.org/r/1286484

Mentioned in SAL (#wikimedia-operations) [2026-05-12T21:05:01Z] <cscott@deploy1003> Started scap sync-world: Backport for [[gerrit:1286484|Bump wikimedia/parsoid to 0.24.0-a3 (T409751 T420336 T425981)]], [[gerrit:1286485|Bump wikimedia/parsoid to 0.24.0-a3 (T425981)]], [[gerrit:1286488|Disable unit tests that fail with new vendor release]], [[gerrit:1286489|Skip ContentHolderTest that fails with new vendor release]]

Mentioned in SAL (#wikimedia-operations) [2026-05-12T21:06:56Z] <cscott@deploy1003> cscott: Backport for [[gerrit:1286484|Bump wikimedia/parsoid to 0.24.0-a3 (T409751 T420336 T425981)]], [[gerrit:1286485|Bump wikimedia/parsoid to 0.24.0-a3 (T425981)]], [[gerrit:1286488|Disable unit tests that fail with new vendor release]], [[gerrit:1286489|Skip ContentHolderTest that fails with new vendor release]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Change

Mentioned in SAL (#wikimedia-operations) [2026-05-12T21:19:53Z] <cscott@deploy1003> Finished scap sync-world: Backport for [[gerrit:1286484|Bump wikimedia/parsoid to 0.24.0-a3 (T409751 T420336 T425981)]], [[gerrit:1286485|Bump wikimedia/parsoid to 0.24.0-a3 (T425981)]], [[gerrit:1286488|Disable unit tests that fail with new vendor release]], [[gerrit:1286489|Skip ContentHolderTest that fails with new vendor release]] (duration: 14m 51s)

Change #1293788 had a related patch set uploaded (by Jgiannelos; author: Jgiannelos):

[mediawiki/services/parsoid@master] regression-testing: Fix default values for CLI args

https://gerrit.wikimedia.org/r/1293788

Change #1293788 merged by jenkins-bot:

[mediawiki/services/parsoid@master] regression-testing: Fix default values for CLI args

https://gerrit.wikimedia.org/r/1293788

Change #1295051 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a6

https://gerrit.wikimedia.org/r/1295051

Change #1295051 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a6

https://gerrit.wikimedia.org/r/1295051

Change #1295066 had a related patch set uploaded (by Arlolra; author: Arlolra):

[mediawiki/vendor@wmf/1.47.0-wmf.4] Bump wikimedia/parsoid to 0.24.0-a6

https://gerrit.wikimedia.org/r/1295066

Change #1295066 merged by jenkins-bot:

[mediawiki/vendor@wmf/1.47.0-wmf.4] Bump wikimedia/parsoid to 0.24.0-a6

https://gerrit.wikimedia.org/r/1295066

Mentioned in SAL (#wikimedia-operations) [2026-05-28T20:41:14Z] <arlolra@deploy1003> Started scap sync-world: Backport for [[gerrit:1295066|Bump wikimedia/parsoid to 0.24.0-a6 (T420336 T427098 T427354 T427082)]], [[gerrit:1295067|Bump wikimedia/parsoid to 0.24.0-a6 (T427082)]]

Mentioned in SAL (#wikimedia-operations) [2026-05-28T20:48:48Z] <arlolra@deploy1003> Finished scap sync-world: Backport for [[gerrit:1295066|Bump wikimedia/parsoid to 0.24.0-a6 (T420336 T427098 T427354 T427082)]], [[gerrit:1295067|Bump wikimedia/parsoid to 0.24.0-a6 (T427082)]] (duration: 07m 34s)

Change #1295471 had a related patch set uploaded (by Jgiannelos; author: Jgiannelos):

[mediawiki/services/parsoid@master] rt-testing: Use dedicated cache for testing instance

https://gerrit.wikimedia.org/r/1295471

Change #1295471 merged by jenkins-bot:

[mediawiki/services/parsoid@master] rt-testing: Use dedicated cache for testing instance

https://gerrit.wikimedia.org/r/1295471

Change #1299571 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a8

https://gerrit.wikimedia.org/r/1299571

Change #1299571 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a8

https://gerrit.wikimedia.org/r/1299571

Change #1299588 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@wmf/1.47.0-wmf.6] Bump wikimedia/parsoid to 0.24.0-a8

https://gerrit.wikimedia.org/r/1299588

Change #1299588 merged by jenkins-bot:

[mediawiki/vendor@wmf/1.47.0-wmf.6] Bump wikimedia/parsoid to 0.24.0-a8

https://gerrit.wikimedia.org/r/1299588

Mentioned in SAL (#wikimedia-operations) [2026-06-09T20:41:15Z] <cscott@deploy1003> cscott: Backport for [[gerrit:1299588|Bump wikimedia/parsoid to 0.24.0-a8 (T378906 T420336 T424427 T427664 T427972 T428452 T428270)]], [[gerrit:1299589|Bump wikimedia/parsoid to 0.24.0-a8 (T428270)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Mentioned in SAL (#wikimedia-operations) [2026-06-09T20:50:28Z] <cscott@deploy1003> Finished scap sync-world: Backport for [[gerrit:1299588|Bump wikimedia/parsoid to 0.24.0-a8 (T378906 T420336 T424427 T427664 T427972 T428452 T428270)]], [[gerrit:1299589|Bump wikimedia/parsoid to 0.24.0-a8 (T428270)]] (duration: 11m 13s)

I also verified the RegressionTesting script we have to double check that between 2 rt-test runs we can actually identify regressions and it looks like its working fine.

Change #1303408 had a related patch set uploaded (by Jgiannelos; author: Jgiannelos):

[mediawiki/services/parsoid@master] rt-testing: Use args using cloudvps as default

https://gerrit.wikimedia.org/r/1303408

Change #1305045 had a related patch set uploaded (by Jgiannelos; author: Jgiannelos):

[mediawiki/services/parsoid@master] rt-testing: Target only k8s/cloudvps based setup

https://gerrit.wikimedia.org/r/1305045

Change #1303408 abandoned by Arlolra:

[mediawiki/services/parsoid@master] rt-testing: Use args using cloudvps as default

Reason:

In favour of I0a7146fb2ba5f29644f2287069e140f05ffa3cad

https://gerrit.wikimedia.org/r/1303408

Change #1305045 merged by jenkins-bot:

[mediawiki/services/parsoid@master] rt-testing: Target only k8s/cloudvps based setup

https://gerrit.wikimedia.org/r/1305045

Change #1305690 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a12

https://gerrit.wikimedia.org/r/1305690

Change #1305690 merged by jenkins-bot:

[mediawiki/vendor@master] Bump wikimedia/parsoid to 0.24.0-a12

https://gerrit.wikimedia.org/r/1305690

Change #1305719 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[mediawiki/vendor@wmf/1.47.0-wmf.8] Bump wikimedia/parsoid to 0.24.0-a12

https://gerrit.wikimedia.org/r/1305719

Change #1305719 merged by jenkins-bot:

[mediawiki/vendor@wmf/1.47.0-wmf.8] Bump wikimedia/parsoid to 0.24.0-a12

https://gerrit.wikimedia.org/r/1305719

Mentioned in SAL (#wikimedia-operations) [2026-06-25T16:36:37Z] <cscott@deploy1003> Started scap sync-world: Backport for [[gerrit:1305719|Bump wikimedia/parsoid to 0.24.0-a12 (T353697 T384490 T387374 T387520 T387521 T391624 T393295 T420336 T429624 T429688 T429822)]], [[gerrit:1305720|Bump wikimedia/parsoid to 0.24.0-a12 (T429822)]]

Mentioned in SAL (#wikimedia-operations) [2026-06-25T16:46:32Z] <cscott@deploy1003> Finished scap sync-world: Backport for [[gerrit:1305719|Bump wikimedia/parsoid to 0.24.0-a12 (T353697 T384490 T387374 T387520 T387521 T391624 T393295 T420336 T429624 T429688 T429822)]], [[gerrit:1305720|Bump wikimedia/parsoid to 0.24.0-a12 (T429822)]] (duration: 09m 54s)