Page MenuHomePhabricator

Move 70% of mediawiki external requests to mw on k8s
Closed, ResolvedPublic

Description

Move external traffic progressively at 65%, then 70% to mw on k8s.

Info from T351074: Move servers from the appserver/api cluster to kubernetes:

For every 5% of external traffic we move, we've needed to bump mw-web by 12-13 replicas and mw-api-ext by 10 replicas.

This means that for every 5% increase in traffic, we're requiring 22-23 additional replicas. Given every pod requires 5.6 CPUs it means we're going to need about 123 cores per traffic bump, or roughly 3 servers as our servers have 48 cores each.

The above calculation is per-datacenter, of course.

  • 65%
  • 70%

Event Timeline

Clement_Goubert changed the task status from Open to In Progress.Mar 22 2024, 11:21 AM
Clement_Goubert triaged this task as High priority.
Clement_Goubert created this task.

Waiting on codfw repool as part of T357547: ☂️ Northward Datacentre Switchover (March 2024) before moving forward with this increase.

Given we have increased mw-web and mw-api-ext by respectively 53 and 10 replicas to cope with handling all the appserver traffic during the codfw depool part of the switchover, the first 5% increase will in my opinion not need an associated replicas increase.

Change #1015277 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: move 65% of traffic to mw on k8s

https://gerrit.wikimedia.org/r/1015277

Mentioned in SAL (#wikimedia-operations) [2024-03-28T12:04:15Z] <claime> trafficserver: move 65% of traffic to mw on k8s - T360763

Change #1015277 merged by Clément Goubert:

[operations/puppet@production] trafficserver: move 65% of traffic to mw on k8s

https://gerrit.wikimedia.org/r/1015277

Change #1018721 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/deployment-charts@master] mw-web, mw-api-ext: Raise replicas for 70% traffic

https://gerrit.wikimedia.org/r/1018721

Change #1018723 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] trafficserver: move 70% of traffic to mw on k8s

https://gerrit.wikimedia.org/r/1018723

Change #1018721 merged by jenkins-bot:

[operations/deployment-charts@master] mw-web, mw-api-ext: Raise replicas for 70% traffic

https://gerrit.wikimedia.org/r/1018721

Change #1018723 merged by Clément Goubert:

[operations/puppet@production] trafficserver: move 70% of traffic to mw on k8s

https://gerrit.wikimedia.org/r/1018723