The parsoid servers (reachable via localhost:6002 on any appserver, or any server running the service mesh proxy) can reply to requests for any url that mediawiki would respond to on an api or appserver. So your code could just call the api URL connecting to port 6002 on localhost (inside mediawiki-config we might need to add a configuration for that).

Thanks Joe.

Some additional bits of info:

• While we are waiting for the parser migration extension to go through security review for redeploy, we have meanwhile added an option in the Parsoid extension to support a "?useparsoid=1" query param.
• The visual diff tests will run on a WMCS server and the test runners there will issue requests to tests pages with and without "?useparsoid=1" query param and compare that output.
• Since these requests will be going to the appservers, we need some mechanism for those requests to be fulfilled

I am agnostic as to the specific solution you provide us with -- for now at least, all we care is that we get Parsoid-rendered HTML for a page when "?useparsoid=1" is passed in. :)

I can see the following two paths to solve this:

• Add a switch to send any request containing useparsoid=1 to the parsoid cluster from the traffic layer
• Enable parsoid on all servers

I am inclined to go with the latter, given eventually parsoid will be the One True Parser (or so we hope) so it will eventually be enabled everywhere.

To do so, we need to just remove any conditional for enabling parsoid and use the same memory limit everywhere - which probably means we also need to beef up the memory for our mediawiki pods a bit.

Change 927115 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/mediawiki-config@master] Use the parsoid memory limit everywhere

https://gerrit.wikimedia.org/r/927115

Change 927116 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/mediawiki-config@master] Load and enable parsoid everywhere

https://gerrit.wikimedia.org/r/927116

Change 927115 abandoned by Giuseppe Lavagetto:

[operations/mediawiki-config@master] Use the parsoid memory limit everywhere

Reason:

https://gerrit.wikimedia.org/r/927115

Change 927670 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/mediawiki-config@master] Raise memory limit to match parsoid

https://gerrit.wikimedia.org/r/927670

Change 927670 merged by jenkins-bot:

[operations/mediawiki-config@master] Raise memory limit to match parsoid

https://gerrit.wikimedia.org/r/927670

Mentioned in SAL (#wikimedia-operations) [2023-06-06T13:33:31Z] <oblivian@deploy1002> Started scap: Backport for [[gerrit:927670|Raise memory limit to match parsoid (T334980)]]

Mentioned in SAL (#wikimedia-operations) [2023-06-06T13:35:06Z] <oblivian@deploy1002> oblivian: Backport for [[gerrit:927670|Raise memory limit to match parsoid (T334980)]] synced to the testservers: mwdebug2001.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1002.eqiad.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-06-06T13:41:24Z] <oblivian@deploy1002> Finished scap: Backport for [[gerrit:927670|Raise memory limit to match parsoid (T334980)]] (duration: 07m 53s)

Change 927116 merged by jenkins-bot:

[operations/mediawiki-config@master] Load and enable parsoid everywhere

https://gerrit.wikimedia.org/r/927116

Mentioned in SAL (#wikimedia-operations) [2023-06-06T14:10:24Z] <oblivian@deploy1002> Started scap: Backport for [[gerrit:927116|Load and enable parsoid everywhere (T334980)]]

Mentioned in SAL (#wikimedia-operations) [2023-06-06T14:12:00Z] <oblivian@deploy1002> oblivian: Backport for [[gerrit:927116|Load and enable parsoid everywhere (T334980)]] synced to the testservers: mwdebug1002.eqiad.wmnet, mwdebug2002.codfw.wmnet, mwdebug1001.eqiad.wmnet, mwdebug2001.codfw.wmnet

Mentioned in SAL (#wikimedia-operations) [2023-06-06T14:25:25Z] <oblivian@deploy1002> Finished scap: Backport for [[gerrit:927116|Load and enable parsoid everywhere (T334980)]] (duration: 15m 00s)

The serviceops part of the work is concluded, parsoid is enabled everywhere

Change 965608 had a related patch set uploaded (by C. Scott Ananian; author: C. Scott Ananian):

[operations/mediawiki-config@master] Enable Parsoid interal REST API only on Parsoid cluster

https://gerrit.wikimedia.org/r/965608

Change 965608 merged by jenkins-bot:

[operations/mediawiki-config@master] Disable Parsoid internal REST API everywhere except on Parsoid cluster

https://gerrit.wikimedia.org/r/965608

Mentioned in SAL (#wikimedia-operations) [2023-10-24T20:53:33Z] <jdrewniak@deploy2002> Started scap: Backport for [[gerrit:965608|Disable Parsoid internal REST API everywhere except on Parsoid cluster (T334980)]]

Mentioned in SAL (#wikimedia-operations) [2023-10-24T20:54:53Z] <jdrewniak@deploy2002> jdrewniak and cscott: Backport for [[gerrit:965608|Disable Parsoid internal REST API everywhere except on Parsoid cluster (T334980)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Mentioned in SAL (#wikimedia-operations) [2023-10-24T21:06:12Z] <jdrewniak@deploy2002> Finished scap: Backport for [[gerrit:965608|Disable Parsoid internal REST API everywhere except on Parsoid cluster (T334980)]] (duration: 12m 39s)