Page MenuHomePhabricator

Fix scap canary checks on beta cluster (Check failed: returned 302, expecting: 200)
Closed, ResolvedPublic

Description

At https://integration.wikimedia.org/ci/job/beta-scap-sync-world/

Scap canary checks always fail on the beta cluster:

13:30:48 10:30:48 Check 'Check endpoints for deployment-mediawiki11.deployment-prep.eqiad1.wikimedia.cloud' failed: /wiki/{title} (Main Page) is CRITICAL: Test Main Page returned the unexpected status 302 (expecting: 200); /wiki/{title} (Special Version) is CRITICAL: Test Special Version returned the unexpected status 302 (expecting: 200)

That's a redirect from http to https, mediawiki_canary_swagger_url uses http in wmcs and https in production.

Event Timeline

Krinkle renamed this task from Fix scap canary checks on beta cluster to Fix scap canary checks on beta cluster (Check failed: returned 302, expecting: 200).May 14 2021, 9:00 PM
Krinkle updated the task description. (Show Details)
Krinkle added a subscriber: Krinkle.

For other people's context (I believe Majavah knows this already) - the reason this has been left broken for so long is that neither HTTP nor HTTPS works from Scap's point of view.

When scap targets localhost with an HTTP url, you get a connection but none of the expected HTTP 200 responses (MW will redirect to HTTPS in beta, just like it does in prod).

When scap targets localhost with an HTTPS url, it would work, if not for the fact that we never get to MW because the beta appservers don't yet support valid TLS connections.

The servers got cfssl certificates in the subtask, but guess they need alternative names too for the wikis?

taavi@deployment-deploy01:~$ service-checker-swagger deployment-mediawiki11.deployment-prep.eqiad1.wikimedia.cloud https://en.wikipedia.beta.wmflabs.org -s /spec.yaml
[...]
ssl.CertificateError: hostname 'en.wikipedia.beta.wmflabs.org' doesn't match 'appservers.svc.deployment-prep.eqiad1.wikimedia.cloud'
[...]

I tried adding them (efecf3a6ff78) but wildcards are somehow not yet functional, filed T282930 for that.

Change 692318 had a related patch set uploaded (by Majavah; author: Majavah):

[operations/puppet@production] scap: Use https for beta swagger checks

https://gerrit.wikimedia.org/r/692318

Change 692318 merged by Jbond:

[operations/puppet@production] scap: Use https for beta swagger checks

https://gerrit.wikimedia.org/r/692318