Thank you for this list Joe. It's very useful and helpful.

@daniel and I spoke about this today in our sync call. I'll be investigating which services are using parsoid via RESTBase. Once we have that list, we can then begin migrate the services to access Parsoid via core endpoints directly.

After a conversation with Daniel yesterday, and a few findings, there are some points to consider:

I've spotted a patch about variants here: https://gerrit.wikimedia.org/r/c/mediawiki/services/chromium-render/+/897192, so reopening this issue for some more investigation.

This is covered in the patch that covers T329821: HTTP response headers compatibility for "Proton + RESTBase" vs "Proton directly"

Proton just calls MediaWiki and delegates redirect mechanisms to it so in this case, redirects such as:

Thank you @pmiazga for joining me in the deployment. And thanks to @Jgiannelos for the guidance and help, and getting patches merged. 🎉

Deployed to codfw and other (staging, eqiad) environments. Resolving this now.

@cscott, I've been thinking about this and so far, when it comes to proton, things happening in the MW world are all MW based. All proton does is calls MediaWiki directly when it comes hitting a page to render it's PDF.

I had a detailed conversation with Piotr yesterday about these 3 and it turns out we can ignore them. Below is our reasoning:

New image of proton: https://integration.wikimedia.org/ci/blue/organizations/jenkins/trigger-chromium-render-pipeline-publish/activity/ (2023-03-06-192059-production) deployed and tested on enwiki. Everything works fine.

Relevant patches:

The following response headers are still in TODO phase: etag, content-location and server.

@Jdforrester-WMF, I've marked the repo as read only and prepended the appropriate message. Let me know if you need anything else sir!

@Jgiannelos & @pmiazga, can you confirm with me that right now, Proton running in isolation follow redirects because that's the default MW way except explicitly specified to not follow redirects?

Testing locally: curl -X 'GET' http://mediawiki.development.instance:3030/en.wikipedia.beta.wmflabs.org/v1/pdf/User:XSavitarTest%2Fsandbox%2FRestbaseProton/a4/desktop -H 'accept: application/pdf' --output ~/Desktop/RedirectTest.pdf seems to reveal that Proton directly follows redirects when trying to print the article as this seems to be the normal mediawiki behavior except the redirect=no is set in the URL.

RESTBase follows redirects so what Proton gets is the article of the target page to print. From testing locally, this is still the case.

Right @hnowlan, from within the network, I can do that. Thanks! The external endpoint is not needed for now.

The difference in the response headers that need to be added or modified for direct Proton to be compatible with how RESTBase + Proton works is:

@pmiazga, the response gotten from hitting proton directly needs to be compatible with RESTBase responses. I see Proton has less stuff but some of them are already exactly the same like RESTBase responses but it lacks more information. I'll have to make sure we add more things to Proton's response headers.

It's gone -> https://codesearch.wmcloud.org/search/?q=TODO%3A%20remove%20B%2FC%20hack%20in%201.32!&i=nope&files=&excludeFiles=&repos=

@Arlolra, I can also confirm that this is working on Group 1 (example: enwikibooks) and Group 2 (example: enwikipedia) which are still both on .13 now.

I was able to reproduce this locally and it happens even if you don't convert from HTML -> Wikitext. From source mode directly to HTML causes this problem. I'm working on a fix. Let's see how it goes. Thanks for reporting @Krinkle. The regression was introduced by the DualParsoidClient: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/VisualEditor/+/844465.

No, they are not equivalent. We have to modes of accessing the parser cache, they need to be tracked separately.

@daniel, I think the below keys:

htmlinputtransformhelper.get-original.with-renderid.stash-miss.paser-cache-miss
htmlinputtransformhelper.get-original.with-renderid.stash-miss.paser-cache-hit.match
htmlinputtransformhelper.get-original.with-renderid.stash-miss.paser-cache-hit.mismatch

Note to self: T218555: Provide access to WebRequest and associated information via a service object

Now done!

In T308701#8142830, ArielGlenn wrote:

Hey Derick, would you mind moving my mention in the task description into a comment someplace, or making it so that it's not a mention by phab user name? Otherwise even if I unsubscribe, any edits to the task by you resubscribe me, and I get an email for every change you make, which at this point... I don't really need :-) Thanks a lot!