Parsoid parses on the beta cluster are currently failing right at the beginning with a config request failure with a Error: Protocol "http:" not supported. Expected "https:"error.
Is this some transient issue or did something change with the beta labs config?
The problem is that MW API is configured as 'http://', but for some reason request uses the https Agent - thus the failure.
I don't think anything has changed in the beta cluster setup to cause this, but probably some dependencies have changed. Did you recently upgrade request library? We had a similar issue in preq at some point - request changed the api and now just setting the agent to the req options is not a good way, the better way is to set agentClass and agentOptions - then request internally pools the agents. If you just set the agent instance in my experience it might be ignored with some combinations of options. See https://github.com/wikimedia/preq/blob/master/index.js#L104-L105
The error, as seen by RESTBase:
error: Config Request failure for \"https://en.wikipedia.beta.wmflabs.org/w/api.php\": Error: Protocol \"http:\" not supported. Expected \"https:\" path: /en.wikipedia.beta.wmflabs.org/v3/page/pagebundle/Main_Page/377057
Error: Config Request failure for \"https://en.wikipedia.beta.wmflabs.org/w/api.php\": Error: Protocol \"http:\" not supported. Expected \"https:\"
at ConfigRequest.ApiRequest._requestCB (/srv/deployment/parsoid/deploy-cache/revs/8ff45dbe8244a853d497c9d4d3a53acd158dbb3e/src/lib/mw/ApiRequest.js:390:15)
at self.callback (/srv/deployment/parsoid/deploy-cache/revs/8ff45dbe8244a853d497c9d4d3a53acd158dbb3e/node_modules/request/request.js:185:22)
at emitOne (events.js:96:13)
at Request.emit (events.js:188:7)
at Request.start (/srv/deployment/parsoid/deploy-cache/revs/8ff45dbe8244a853d497c9d4d3a53acd158dbb3e/node_modules/request/request.js:749:10)
at Request.end (/srv/deployment/parsoid/deploy-cache/revs/8ff45dbe8244a853d497c9d4d3a53acd158dbb3e/node_modules/request/request.js:1506:10)
at end (/srv/deployment/parsoid/deploy-cache/revs/8ff45dbe8244a853d497c9d4d3a53acd158dbb3e/node_modules/request/request.js:560:14)
at Immediate.<anonymous> (/srv/deployment/parsoid/deploy-cache/revs/8ff45dbe8244a853d497c9d4d3a53acd158dbb3e/node_modules/request/request.js:574:7)
at runCallback (timers.js:672:20)
at tryOnImmediate (timers.js:645:5)
at processImmediate [as _immediateCallback] (timers.js:617:5)I'm assuming it's this series of patches that broke it
https://github.com/wikimedia/puppet/commit/2f55fc2396c92c1a2d0463ba77c828748f3f7684
https://github.com/wikimedia/puppet/commit/974c46cd4a46fd605cec4c1617c3cff5b9c3dfbb
https://github.com/wikimedia/puppet/commit/e26579ff3c4abedd2beb9d329d48b79f755d6406
09:37 Pchelolo: arlolra: I think I found why parsoid is failing in beta 09:39 arlolra: Pchelolo: /me perks up ... I haven't actually looked yet 09:40 arlolra: I just assumed it was a configuration change 09:41 Pchelolo: when using ApiRequest, https://en.wikipedia.beta.wmflabs.org/w/api.php is provided as a uri which is conf.wiki.apiURI. http.Agent is selected based on it and https agent is assigned. then on line 291 the uri is assigned to mwApiServer===conf.parsoid.mwApiServer which is http://deployment-mediawiki-07.deployment-prep.eqiad.wmflabs/w/api.php 09:41 Pchelolo: so the protocol starts to mismatch the agent 09:41 Pchelolo: we just need to move the agent selection code way to the bottom of ApiRequest.prototype.request method and we should be fine 09:41 Pchelolo: verifying 09:43 Pchelolo: vuala, parsoid works in beta 09:44 Pchelolo: I'm not sure my approach is entirely correct, but parsoid is life-hacked on deployment-parsoid09 now and it works
I will leave it to the professionals to verify this is the right approach, but it seems correct, the mwApi and mwApiServer protocols can mismatch, so we need to assign a correct agent.
Change 464364 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid@master] Set request agent after updating uri with mwApiServer
While @Arlolra patch surely does a sensible thing, I am not sure we have an https interface to mediawiki in beta. I didn't think of this when I did the transition in production, probably. This is easy to solve though, and can go in parallel with what @Arlolra is doing (which is indeed correct anyways).
Change 464364 merged by jenkins-bot:
[mediawiki/services/parsoid@master] Set request agent after updating uri with mwApiServer
Sorry, I misinterpreted the issue. We are connecting via http even though the configuration says https, so Arlo's patch is all that's needed.
The config still doesn't seem to be working for the edge case of en.wp.org:
error: Config Request failure for \"https://en.wikipedia.org/w/api.php\": 404 path: /en.wikipedia.org/v3/page/pagebundle/Main_Page/847600508
Error: Config Request failure for \"https://en.wikipedia.org/w/api.php\": 404
at ConfigRequest.ApiRequest._requestCB (/srv/deployment/parsoid/deploy-cache/revs/894d12f4a01e5445b0716d16f70e81d0be4f78fd/src/lib/mw/ApiRequest.js:410:11)
at Request.self.callback (/srv/deployment/parsoid/deploy-cache/revs/894d12f4a01e5445b0716d16f70e81d0be4f78fd/node_modules/request/request.js:185:22)
at emitTwo (events.js:106:13)
at Request.emit (events.js:191:7)
at Request.<anonymous> (/srv/deployment/parsoid/deploy-cache/revs/894d12f4a01e5445b0716d16f70e81d0be4f78fd/node_modules/request/request.js:1157:10)
at emitOne (events.js:96:13)
at Request.emit (events.js:188:7)
at IncomingMessage.<anonymous> (/srv/deployment/parsoid/deploy-cache/revs/894d12f4a01e5445b0716d16f70e81d0be4f78fd/node_modules/request/request.js:1079:12)
at IncomingMessage.g (events.js:292:16)
at emitNone (events.js:91:20)
at IncomingMessage.emit (events.js:185:7)
at endReadableNT (_stream_readable.js:974:12)
at _combinedTickCallback (internal/process/next_tick.js:80:11)
at process._tickCallback (internal/process/next_tick.js:104:9)Change 464553 had a related patch set uploaded (by Arlolra; owner: Arlolra):
[mediawiki/services/parsoid/deploy@master] Fix enwikiprod on the beta cluster
Change 464553 merged by jenkins-bot:
[mediawiki/services/parsoid/deploy@master] Fix enwikiprod on the beta cluster