Page MenuHomePhabricator
Create Task
Maniphest T139461

Move k8s master to k8s-master.tools.wmflabs.org
Closed, ResolvedPublic
Actions

Description

Right now it is tools-k8s-master-01.tools.eqiad.wmflabs, which causes a bunch of problems:

  1. SPOF, hard to failover
  2. It uses the puppetmaster CA to sign certificates, which means anything communicating with it over SSL needs to be on tools-puppetmaster-01 (another SPOF!)

Steps to switch:

  1. Create a public IP + assign DNS for the k8s master
  2. Set up SSL certificates using *.tools.wmflabs.org so that the k8s master works properly
  3. Set hiera variable and hand-run puppet to make sure that all the kubelets and kube-proxies switch over properly
  4. Do a find + sed on all .kube/config files

Details

SubjectRepoBranchLines +/-
tools: Don't set CA explicitly for kube2proxyoperations/puppetproduction+2 -4
tools: Use provisioned cert instead of puppet certoperations/puppetproduction+6 -4
tools: Don't specify CA explicitly for client configoperations/puppetproduction+0 -1
tools: Provision star.tools.wmflabs.org cert for k8s masteroperations/puppetproduction+6 -1
Customize query in gerrit

Related Objects

Event Timeline

yuvipanda created this task.Jul 6 2016, 1:00 PM
Restricted Application added a project: Cloud-Services. · View Herald TranscriptJul 6 2016, 1:00 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald Transcript
Stashbot subscribed.Jul 6 2016, 1:09 PM

Mentioned in SAL [2016-07-06T13:09:13Z] <yuvipanda> associated a floating IP with tools-k8s-master-01 for T139461

gerritbot subscribed.Jul 6 2016, 1:53 PM

Change 297591 had a related patch set uploaded (by Yuvipanda):
tools: Provision star.tools.wmflabs.org cert for k8s master

https://gerrit.wikimedia.org/r/297591

gerritbot added a project: Patch-For-Review.Jul 6 2016, 1:53 PM
yuvipanda mentioned this in rOPUP2af585fa3272: tools: Provision star.tools.wmflabs.org cert for k8s master.Jul 6 2016, 1:57 PM
gerritbot added a comment.Jul 6 2016, 2:05 PM

Change 297595 had a related patch set uploaded (by Yuvipanda):
tools: Use provisioned cert instead of puppet cert

https://gerrit.wikimedia.org/r/297595

yuvipanda mentioned this in rOPUPf8d42424420a: tools: Use provisioned cert instead of puppet cert.Jul 6 2016, 2:10 PM
gerritbot added a comment.Jul 6 2016, 2:42 PM

Change 297600 had a related patch set uploaded (by Yuvipanda):
tools: Don't specify CA explicitly for client config

https://gerrit.wikimedia.org/r/297600

yuvipanda mentioned this in rOPUP3df28ca02124: tools: Provision star.tools.wmflabs.org cert for k8s master.Jul 6 2016, 2:45 PM
yuvipanda mentioned this in rOPUPc5d56d30b792: tools: Don't specify CA explicitly for client config.
yuvipanda mentioned this in rOPUP30c444f4871a: tools: Use provisioned cert instead of puppet cert.
gerritbot added a comment.Jul 6 2016, 4:25 PM

Change 297611 had a related patch set uploaded (by Yuvipanda):
tools: Don't set CA explicitly for kube2proxy

https://gerrit.wikimedia.org/r/297611

yuvipanda mentioned this in rOPUPc2a200eb6284: tools: Don't set CA explicitly for kube2proxy.Jul 6 2016, 4:31 PM
gerritbot added a comment.Jul 6 2016, 5:18 PM

Change 297591 merged by Yuvipanda:
tools: Provision star.tools.wmflabs.org cert for k8s master

https://gerrit.wikimedia.org/r/297591

gerritbot added a comment.Jul 6 2016, 5:18 PM

Change 297600 merged by Yuvipanda:
tools: Don't specify CA explicitly for client config

https://gerrit.wikimedia.org/r/297600

gerritbot added a comment.Jul 6 2016, 5:19 PM

Change 297595 merged by Yuvipanda:
tools: Use provisioned cert instead of puppet cert

https://gerrit.wikimedia.org/r/297595

gerritbot added a comment.Jul 6 2016, 5:19 PM

Change 297611 merged by Yuvipanda:
tools: Don't set CA explicitly for kube2proxy

https://gerrit.wikimedia.org/r/297611

yuvipanda mentioned this in rOPUP40b2106a6813: tools: Provision star.tools.wmflabs.org cert for k8s master.Jul 6 2016, 5:23 PM
yuvipanda mentioned this in rOPUP5b6c1f18e67c: tools: Don't specify CA explicitly for client config.
yuvipanda mentioned this in rOPUP24aefaabec01: tools: Use provisioned cert instead of puppet cert.
yuvipanda mentioned this in rOPUPafd3fa804aa7: tools: Don't set CA explicitly for kube2proxy.
yuvipanda closed this task as Resolved.Jul 6 2016, 5:31 PM

Done!

• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:41 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits