Page MenuHomePhabricator
Create Task
Maniphest T334561

Add mw-on-k8s deployments to mediawiki certificates
Closed, ResolvedPublic
Actions

Description

Reaching mw-api-int.discovery.wmnet from envoy service proxy fails, we are missing the correct SANs for all mw-on-k8s deployments.

  • Add "mw-api-ext.discovery.wmnet", "mw-api-ext-ro.discovery.wmnet", "mw-api-ext.svc.eqiad.wmnet", "mw-api-ext.svc.codfw.wmnet", "mw-api-int.discovery.wmnet","mw-api-int-ro.discovery.wmnet", "mw-api-int.svc.eqiad.wmnet", "mw-api-int.svc.codfw.wmnet" to appservers-rw.discovery.wmnet stanza in mediawiki.certs.yaml
  • Add "mw-api-ext.discovery.wmnet", "mw-api-ext-ro.discovery.wmnet", "mw-api-ext.svc.eqiad.wmnet", "mw-api-ext.svc.codfw.wmnet", "mw-api-int.discovery.wmnet","mw-api-int-ro.discovery.wmnet", "mw-api-int.svc.eqiad.wmnet", "mw-api-int.svc.codfw.wmnet" to api-rw.discovery.wmnet stanza in mediawiki.certs.yaml
  • puppet cert clean appservers-rw.discovery.wmnet
  • puppet cert clean api-rw.discovery.wmnet
  • rm /srv/private/modules/secret/secrets/certificates/appservers-rw.discovery.wmnet/{appservers-rw.discovery.wmnet.crt.pem,appservers-rw.discovery.wmnet.csr.pem}
  • rm /srv/private/modules/secret/secrets/certificates/api-rw.discovery.wmnet/{api-rw.discovery.wmnet.crt.pem,api-rw.discovery.wmnet.csr.pem}
  • cergen -c 'appservers-rw.*' --generate --base-path=/srv/private/modules/secret/secrets/certificates /srv/private/modules/secret/secrets/certificates/certificate.manifests.d
  • cergen -c 'api-rw.*' --generate --base-path=/srv/private/modules/secret/secrets/certificates /srv/private/modules/secret/secrets/certificates/certificate.manifests.d
  • Commit the changes to the private repo
  • Copy /srv/private/modules/secret/secrets/certificates/appservers-rw.discovery.wmnet/appservers-rw.discovery.wmnet.crt.pem to puppet ./modules/profile/files/ssl/appservers.svc.{eqiad,codfw}.wmnet.crt
  • Copy /srv/private/modules/secret/secrets/certificates/api-rw.discovery.wmnet/api-rw.discovery.wmnet.crt.pem to puppet ./modules/profile/files/ssl/api.svc.{eqiad,codfw}.wmnet.crt
  • Commit public puppet changes
  • Run puppet on deploy2002 and redeploy mw-on-k8s with scap sync-world --k8s-only "Updating mw-on-k8s certificates"

Details

SubjectRepoBranchLines +/-
ssl: Update api.svc and appservers.svc certsoperations/puppetproduction+168 -148
Customize query in gerrit

Related Objects
Search...

Event Timeline

Clement_Goubert triaged this task as High priority.Apr 12 2023, 9:34 AM
Clement_Goubert created this task.
Clement_Goubert moved this task from Incoming 🐫 to Doing 😎 on the serviceops board.
Clement_Goubert renamed this task from Add mw-on-k8s deployments to appservers-rw.discovery.wmnet certificate SAN to Add mw-on-k8s deployments to mediawiki certificates.Apr 12 2023, 12:39 PM
Clement_Goubert updated the task description. (Show Details)
Clement_Goubert updated the task description. (Show Details)Apr 13 2023, 9:19 AM
fgiunchedi subscribed.Apr 13 2023, 10:28 AM

Double checked in private.git data and procedure LGTM

Clement_Goubert updated the task description. (Show Details)Apr 13 2023, 10:33 AM
Stashbot added a comment.Apr 13 2023, 10:39 AM

Mentioned in SAL (#wikimedia-operations) [2023-04-13T10:39:28Z] <claime> updating appservers and api certificates - T334561

Clement_Goubert updated the task description. (Show Details)Apr 13 2023, 10:40 AM
Clement_Goubert updated the task description. (Show Details)
Clement_Goubert updated the task description. (Show Details)Apr 13 2023, 10:42 AM
gerritbot added a comment.Apr 13 2023, 10:48 AM

Change 908502 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] ssl: Update api.svc and appservers.svc certs

https://gerrit.wikimedia.org/r/908502

gerritbot added a project: Patch-For-Review.Apr 13 2023, 10:48 AM
Clement_Goubert updated the task description. (Show Details)Apr 13 2023, 10:50 AM
gerritbot added a comment.Apr 13 2023, 10:51 AM

Change 908502 merged by Clément Goubert:

[operations/puppet@production] ssl: Update api.svc and appservers.svc certs

https://gerrit.wikimedia.org/r/908502

Maintenance_bot removed a project: Patch-For-Review.Apr 13 2023, 11:10 AM
Stashbot added a comment.Apr 13 2023, 11:15 AM

Mentioned in SAL (#wikimedia-operations) [2023-04-13T11:15:25Z] <claime> Re-deploying mw-on-k8s to update certificates - T334561

Clement_Goubert closed this task as Resolved.Apr 13 2023, 11:23 AM
Clement_Goubert updated the task description. (Show Details)
cgoubert@deploy2002:~$ curl -v -H Host:en.wikipedia.org https://mw-api-int.discovery.wmnet:4446/wiki/Main_page 2>&1 | grep mw-api-int
* Connected to mw-api-int.discovery.wmnet (10.2.1.81) port 4446 (#0)
*  subjectAltName: host "mw-api-int.discovery.wmnet" matched cert's "mw-api-int.discovery.wmnet"
* Connection #0 to host mw-api-int.discovery.wmnet left intact
cgoubert@deploy2002:~$ curl -v -H Host:en.wikipedia.org https://mw-api-ext.discovery.wmnet:4447/wiki/Main_page 2>&1 | grep mw-api-ext
* Connected to mw-api-ext.discovery.wmnet (10.2.1.76) port 4447 (#0)
*  subjectAltName: host "mw-api-ext.discovery.wmnet" matched cert's "mw-api-ext.discovery.wmnet"
* Connection #0 to host mw-api-ext.discovery.wmnet left intact
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL