| Joe | |
| Jul 19 2023, 1:45 PM |
| F37149157: image.png | |
| Jul 26 2023, 11:32 AM |
We are going to migrate the requests from flink to the mediawiki api to go to mediawiki on kubernetes instead than to the traditional api cluster.
Given flink does on average about 20 rps per datacenter, we should be able to add resources in line with mw-api-ext or less.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Stalled | None | T255792 Quibble runs core:unit tests twice! | |||
| Open | None | T328919 Upgrade to PHPUnit 10 | |||
| Open | None | T338103 Micro-optimize ApiResult::isMetadataKey with str_starts_with once we support PHP8+ | |||
| Open | None | T328921 Drop PHP 7.4 support from MediaWiki | |||
| Stalled | None | T334726 Use return type `never` in Wikibase | |||
| Open | None | T328922 Drop PHP 8.0 support from MediaWiki | |||
| Stalled | None | T319055 Upgrade to psr/container 2.x | |||
| Stalled | Krinkle | T319432 Migrate WMF production from PHP 7.4 to PHP 8.1 | |||
| Open | None | T291916 Tracking task for Bullseye migrations in production | |||
| Stalled | None | T356293 Migrate MW appservers' base images to bullseye | |||
| Open | None | T290536 Serve production traffic via Kubernetes | |||
| In Progress | Clement_Goubert | T333120 Migrate internal traffic to k8s | |||
| Resolved | Joe | T342252 Migrate rdf-streaming-updater to connect to mw-on-k8s |
@dcausse not sure if you're the right person to ask, if not apologies; but I wanted to know if we're making any write request to mediawiki from the rdf updater, or we're just reading data.
In the latter case, we can make flink talk to the local mw api cluster in all datacenters, distributing resources better and reducing latencies.
We do only reads indeed and we actually initially tested with the api-ro endpoint so it should work and I'm all for it, @bking or myself can take care of this if this helps.
I'll have to do some prep work, but sure once I'm ready I'm happy to let you drive the migration :)
Change 939700 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):
[operations/puppet@production] services_proxy: add mw-api-int-async-ro
Change 939701 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):
[operations/deployment-charts@master] mw-api-int: bump replicas to 8
Change 939702 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):
[operations/deployment-charts@master] rdf-streaming-updater: move to mw-api-int, use readonly endpoint
Change 939716 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):
[operations/deployment-charts@master] mw-api-int: increase namespace limits
Change 939700 merged by Giuseppe Lavagetto:
[operations/puppet@production] services_proxy: add mw-api-int-async-ro
Change 939716 merged by jenkins-bot:
[operations/deployment-charts@master] mw-api-int: increase namespace limits
Change 939701 merged by jenkins-bot:
[operations/deployment-charts@master] mw-api-int: bump replicas to 8
Change 939702 merged by jenkins-bot:
[operations/deployment-charts@master] rdf-streaming-updater: move to mw-api-int, use readonly endpoint
Change 940870 had a related patch set uploaded (by DCausse; author: DCausse):
[operations/deployment-charts@master] rdf-streaming-updater (dse-k8s test): use mw-api-int-async-ro
Now rdf-streaming-updater uses the read-only endpoint on k8s, meaning it should also be more efficient in its API usage.
Change 940870 merged by jenkins-bot:
[operations/deployment-charts@master] rdf-streaming-updater (dse-k8s test): use mw-api-int-async-ro
Change 940886 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):
[operations/deployment-charts@master] mw-api-int: Raise php container CPU limits
Change 940886 merged by jenkins-bot:
[operations/deployment-charts@master] mw-api-int: Raise php container CPU limits
Change 940888 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):
[operations/deployment-charts@master] admin_ng: Raise max cpu per pod to 10 for mw-api-int
Change 940888 merged by jenkins-bot:
[operations/deployment-charts@master] admin_ng: Raise max cpu per pod to 10 for mw-api-int
We've been experiencing throttling on mw-api-int and raising the container's CPU limit has helped, but not fixed the issue.
I will raise the number of replicas for this deployment, however looking at the other deployments, I can see they also get throttled.
We may want to think about removing the cpu limits for the latency-sensitive mw-on-k8s deployments.
Change 941900 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):
[operations/deployment-charts@master] mw-api-int: Raise number of replicas to 10
Change 941900 abandoned by Clément Goubert:
[operations/deployment-charts@master] mw-api-int: Raise number of replicas to 10
Reason:
We've gone way beyond 10 replicas by now