The current infrastructure in EQIAD has a couple of identified Single Points of Failure that should be addressed. Those are
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | akosiaris | T162039 Prepare to service applications from kubernetes | |||
| Resolved | akosiaris | T162040 Eliminate SPOFs in the existing eqiad kubernetes infrastructure | |||
| Unknown Object (Task) | |||||
| Resolved | • Cmjohnson | T166076 rack/setup/install ganeti1005-ganeti1008 |
Task T161702 for the purchase of ganeti nodes in eqiad is being processed by me in the procurement space & is projected to result in the ordering of 4 new ganeti hosts for eqiad. (Just commenting here since that blocking sub-task isn't public any longer due to pricing being added.)
Change 351836 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/dns@master] Create kubemaster.svc.$site.wmnet
Change 352580 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] lvs: Add the kubernetes master service/cluster
Change 351836 merged by Alexandros Kosiaris:
[operations/dns@master] Create kubemaster.svc.$site.wmnet
Change 352580 merged by Alexandros Kosiaris:
[operations/puppet@production] lvs: Add the kubernetes master service/cluster
The single master (with a manual override) SPOF has been addressed. We now have 2 masters in eqiad behind an LVS service with a proper puppet certificate.
Change 359908 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/dns@master] Renumber neon.eqiad.wmnet
Change 359909 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/dns@master] Renumber etcd100{2,3,4,5,6}.eqiad.wmnet
Change 359910 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/dns@master] Renumber chlorine.eqiad.wmnet
Change 359908 merged by Alexandros Kosiaris:
[operations/dns@master] Renumber neon.eqiad.wmnet
Change 359909 merged by Alexandros Kosiaris:
[operations/dns@master] Renumber etcd100{2,3,4,5}.eqiad.wmnet
Change 359910 merged by Alexandros Kosiaris:
[operations/dns@master] Renumber chlorine.eqiad.wmnet
2/3 etcd hosts are now in a different row giving up read functionality in the worst case scenario, that is failure of row A and read/write in case of failure of row C. chlorine has been moved to row A as well. This is now successfully resolved.