Page MenuHomePhabricator

Separate deployment for wikifunctions.org
Closed, ResolvedPublic

Description

Given the specific nature of wikifunctions, the SRE team has evaluated that even opening the execution of functions to the general public constitutes a risk that is unacceptable if wikifunctions shares the php worker pool with the rest of production. Given the current (and future, AIUI) performance of the system expected to be in the order of seconds per function call, we certainly want to contain the possible blast radius of the problem.

To this end we need the following:

  • set up a new deployment of mediawiki on k8s, possibly under ingress
  • Set up the corresponding entry in service::catalog, including discovery dns
  • Modify the traffic layer routing logic to separate out wikifunctions. Probably it makes sense to do so in mw-on-k8s.lua.
  • Add service-mesh listeners for mw-wikifunctions
  • Change wikifunctions mesh listener to use the separate deployment (mw-wikifunctions-ro) instead of mw-api-int for api calls
  • Add networkpolicy rule allowing wikifunctions to connect to mw-wikifunctions

Event Timeline

Joe triaged this task as High priority.Sep 28 2023, 6:34 AM
Joe moved this task from Incoming 🐫 to this.quarter 🍕 on the serviceops-deprecated board.
Jdforrester-WMF changed the task status from Open to In Progress.Sep 28 2023, 4:33 PM
Jdforrester-WMF moved this task from To Triage to In Progress on the Abstract Wikipedia team board.

Change 965054 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] admin_ng: Add namespace for wikifunctions mediawiki deployment

https://gerrit.wikimedia.org/r/965054

Change 965055 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] Add mediawiki deployment for wikifunctions

https://gerrit.wikimedia.org/r/965055

Change 965056 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] wikifunctions: Add routing to separate mw-on-k8s

https://gerrit.wikimedia.org/r/965056

Change 965062 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/dns@master] Add mw-wikifunctions records

https://gerrit.wikimedia.org/r/965062

Change 965065 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/dns@master] Add mw-wikifunctions discovery records

https://gerrit.wikimedia.org/r/965065

Change 965086 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/puppet@production] service::catalog: Add mw-wikifunctions - 1

https://gerrit.wikimedia.org/r/965086

Change 965121 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/puppet@production] Add mw-wikifunctions to mediawiki k8s releases

https://gerrit.wikimedia.org/r/965121

Change 965054 merged by jenkins-bot:

[operations/deployment-charts@master] admin_ng: Add namespace for wikifunctions mediawiki deployment

https://gerrit.wikimedia.org/r/965054

Change 965055 merged by jenkins-bot:

[operations/deployment-charts@master] Add mediawiki deployment for wikifunctions

https://gerrit.wikimedia.org/r/965055

Change 965156 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/puppet@production] deployment_server: Add mw-wikifunctions

https://gerrit.wikimedia.org/r/965156

Change 965156 merged by JMeybohm:

[operations/puppet@production] deployment_server: Add mw-wikifunctions

https://gerrit.wikimedia.org/r/965156

Change 965158 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] Remove namespace quota and limitranger from mw-wikifunctions

https://gerrit.wikimedia.org/r/965158

Change 965158 merged by jenkins-bot:

[operations/deployment-charts@master] Remove namespace quota and limitranger from mw-wikifunctions

https://gerrit.wikimedia.org/r/965158

Change 965121 merged by JMeybohm:

[operations/puppet@production] Add mw-wikifunctions to mediawiki k8s releases

https://gerrit.wikimedia.org/r/965121

Change 965062 merged by JMeybohm:

[operations/dns@master] Add mw-wikifunctions records

https://gerrit.wikimedia.org/r/965062

Change 965086 merged by JMeybohm:

[operations/puppet@production] service::catalog: Add mw-wikifunctions - 1

https://gerrit.wikimedia.org/r/965086

Change 965175 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/puppet@production] service::catalog: Add mw-wikifunctions - 2

https://gerrit.wikimedia.org/r/965175

Change 965176 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/puppet@production] service::catalog: Add mw-wikifunctions - 3

https://gerrit.wikimedia.org/r/965176

Change 965177 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/puppet@production] service::catalog: Add mw-wikifunctions - 4

https://gerrit.wikimedia.org/r/965177

Change 965175 merged by JMeybohm:

[operations/puppet@production] service::catalog: Add mw-wikifunctions - 2

https://gerrit.wikimedia.org/r/965175

Change 965176 merged by JMeybohm:

[operations/puppet@production] service::catalog: Add mw-wikifunctions - 3

https://gerrit.wikimedia.org/r/965176

Change 965177 merged by JMeybohm:

[operations/puppet@production] service::catalog: Add mw-wikifunctions - 4

https://gerrit.wikimedia.org/r/965177

Change 965065 merged by JMeybohm:

[operations/dns@master] Add mw-wikifunctions discovery records

https://gerrit.wikimedia.org/r/965065

Change 965227 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] Add appserver, api and jobrunner SANs to mw deployments

https://gerrit.wikimedia.org/r/965227

Change 965227 merged by jenkins-bot:

[operations/deployment-charts@master] Add appserver, api and jobrunner SANs to mw deployments

https://gerrit.wikimedia.org/r/965227

Change 965471 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/puppet@production] service_proxy: Add mw-wikifunctions-ro listener

https://gerrit.wikimedia.org/r/965471

Change 965473 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] wikifunctions: Switch to use mw-wikifunctions for API calls

https://gerrit.wikimedia.org/r/965473

Change 965471 merged by JMeybohm:

[operations/puppet@production] service_proxy: Add mw-wikifunctions-ro listener

https://gerrit.wikimedia.org/r/965471

Change 965473 merged by jenkins-bot:

[operations/deployment-charts@master] wikifunctions: Switch to use mw-wikifunctions for API calls

https://gerrit.wikimedia.org/r/965473

Mentioned in SAL (#wikimedia-operations) [2023-10-12T12:16:06Z] <jayme> disable puppet on A:cp-text - T347544

Change 965056 merged by JMeybohm:

[operations/puppet@production] wikifunctions: Add routing to separate mw-on-k8s

https://gerrit.wikimedia.org/r/965056

All wikifunctions.org traffic from the edge as well as from function-orchestrator is now served by the separate k8s deployment mw-wikifunctions

Change 969131 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/cookbooks@master] Add mw-wikifunctions to MEDIAWIKI_SERVICES

https://gerrit.wikimedia.org/r/969131

Change 969131 merged by jenkins-bot:

[operations/cookbooks@master] Add mw-wikifunctions to MEDIAWIKI_SERVICES

https://gerrit.wikimedia.org/r/969131