Page MenuHomePhabricator
Create Task
Maniphest T359423

Migrate charts to Calico Network Policies
Open, MediumPublic
Actions

Description

The following charts will need to be migrated to the base.networkpolicy.egress.external-services helm template, rendering Calico network policies:

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
cfssl-issuer: Add external-services supportJMeybohmoperations/deployment-chartsmaster+33 -2
global_config: Add pki::multirootca IPs to external-servicesJMeybohmoperations/puppetproduction+17 -4
eventstreams: adopt base.external-services-networkpolicyScott Frenchoperations/deployment-chartsmaster+74 -32
datahub: replace IPs by Services in network policiesBrouberoloperations/deployment-chartsmaster+30 -43
datahub-next: restore IP-based networkpolicy to datahubsearchBrouberoloperations/deployment-chartsmaster+10 -3
datahub: fix label matching beetween pods and networkpoliciesBrouberoloperations/deployment-chartsmaster+8 -8
datahub: update datahubsearch hostname to use external-servicesBrouberoloperations/deployment-chartsmaster+1 -1
Deploy calico network policy templates to all datahub chartsBrouberoloperations/deployment-chartsmaster+257 -8
datahub-next: replace IPs by Services in network policiesBrouberoloperations/deployment-chartsmaster+19 -38
Migrate AQS2 services and image-suggestions to calico network policiesBtullisoperations/deployment-chartsmaster+181 -323
benthos: adopt securityContext and base.external-services-networkpolicyScott Frenchoperations/deployment-chartsmaster+82 -26
eventgate-*: Migrate to base.external-services-networkpolicyJMeybohmoperations/deployment-chartsmaster+80 -47
eventgate: Update mesh modulesJMeybohmoperations/deployment-chartsmaster+123 -73
charts/kserve-inference: Wire up generated network policy for LW servicesKlausmanoperations/deployment-chartsmaster+119 -3
changeprop-jobqueue: Migrate to base.external-services-networkpolicyJMeybohmoperations/deployment-chartsmaster+5 -185
changeprop: Migrate to base.external-services-networkpolicyJMeybohmoperations/deployment-chartsmaster+5 -184
changeprop: Add base.external-services-networkpolicy:1.0JMeybohmoperations/deployment-chartsmaster+63 -5
changeprop: Update mesh modulesJMeybohmoperations/deployment-chartsmaster+123 -73
Make datahub networkpolicy include/template consistentBtullisoperations/deployment-chartsmaster+3 -3
Fix whitespace in the external-services values for datahub stagingBtullisoperations/deployment-chartsmaster+1 -1
spark-history: bypass Kerberos principal hostname reverse DNS check for namenodeBrouberoloperations/deployment-chartsmaster+7 -1
spark-history: fix egress network policiesBrouberoloperations/deployment-chartsmaster+12 -4
spark-history: replace hardcoded CIDRs by service names to generate egress policiesBrouberoloperations/deployment-chartsmaster+7 -70
spark-history: add external-services egress network policy templateBrouberoloperations/deployment-chartsmaster+49 -3
Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
gerritbot added a comment.Mar 26 2024, 3:48 PM

Change #1014540 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] changeprop: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1014540

gerritbot added a comment.Mar 26 2024, 3:52 PM

Change #1014542 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] changeprop-jobqueue: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1014542

JMeybohm updated the task description. (Show Details)Mar 26 2024, 4:03 PM
brouberol updated the task description. (Show Details)Mar 26 2024, 4:06 PM
BTullis updated the task description. (Show Details)Mar 26 2024, 10:26 PM
gerritbot added a comment.Mar 26 2024, 10:27 PM

Change #1014646 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/deployment-charts@master] Fix typo in the external-services values for datahub staging

https://gerrit.wikimedia.org/r/1014646

BTullis added a comment.EditedMar 26 2024, 10:30 PM

I needed to start work on the DataHub migration because SSO broke due to the switch of IDP servers: https://sal.toolforge.org/log/UMCPdY4BhuQtenzvi5Pe

Slack thread here.

CR here: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1014065

So far, I have only migrated the CAS IDP services, but there are still several others that can be migrated too, including:

  • datahubsearch
  • mariadb (analytics_meta)

We can probably remove the LDAP policies from this deployment, now that CAS is enabled. Otherwise I would migrate that too.

gerritbot added a comment.Mar 26 2024, 10:54 PM

Change #1014646 merged by jenkins-bot:

[operations/deployment-charts@master] Fix whitespace in the external-services values for datahub staging

https://gerrit.wikimedia.org/r/1014646

BTullis updated the task description. (Show Details)Mar 26 2024, 10:58 PM
BTullis updated the task description. (Show Details)Mar 26 2024, 11:08 PM
gerritbot added a comment.Mar 27 2024, 10:52 AM

Change #1014652 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/deployment-charts@master] Make datahub networkpolicy include/template consistent

https://gerrit.wikimedia.org/r/1014652

brouberol updated the task description. (Show Details)Mar 27 2024, 12:50 PM
brouberol added a subscriber: klausman.
gerritbot added a comment.Mar 27 2024, 12:51 PM

Change #1015029 had a related patch set uploaded (by Klausman; author: Klausman):

[operations/deployment-charts@master] charts/kserve-inference: Wire up generated network policy for LW services

https://gerrit.wikimedia.org/r/1015029

gerritbot added a comment.Mar 27 2024, 1:23 PM

Change #1014652 merged by jenkins-bot:

[operations/deployment-charts@master] Make datahub networkpolicy include/template consistent

https://gerrit.wikimedia.org/r/1014652

Raine updated the task description. (Show Details)Mar 27 2024, 2:43 PM
Raine subscribed.
gerritbot added a comment.Mar 27 2024, 3:37 PM

Change #1014538 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop: Update mesh modules

https://gerrit.wikimedia.org/r/1014538

gerritbot added a comment.Mar 27 2024, 3:38 PM

Change #1014539 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop: Add base.external-services-networkpolicy:1.0

https://gerrit.wikimedia.org/r/1014539

gerritbot added a comment.Mar 27 2024, 3:38 PM

Change #1014540 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1014540

gerritbot added a comment.Mar 27 2024, 4:03 PM

Change #1014542 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop-jobqueue: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1014542

JMeybohm updated the task description. (Show Details)Mar 27 2024, 4:16 PM
BTullis mentioned this in T360531: [Commons Impact Metrics] Refactor/create helm config for AQS service accessing both Cassandra and Druid.Mar 28 2024, 11:06 AM
gerritbot added a comment.Mar 28 2024, 11:28 AM

Change #1015029 merged by jenkins-bot:

[operations/deployment-charts@master] charts/kserve-inference: Wire up generated network policy for LW services

https://gerrit.wikimedia.org/r/1015029

JMeybohm updated the task description. (Show Details)Apr 10 2024, 12:44 PM

Grouped the todo list by chart, some of those also need mesh.configuration updates due to T346638: Rename the envoy's uses_ingress option to sets_sni which could be bundled with this

gerritbot added a comment.Apr 11 2024, 11:40 AM

Change #1019007 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] eventgate: Update mesh modules

https://gerrit.wikimedia.org/r/1019007

gerritbot added a comment.Apr 11 2024, 12:02 PM

Change #1019018 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] eventgate-*: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1019018

JMeybohm updated the task description. (Show Details)Apr 12 2024, 7:32 AM
JMeybohm updated the task description. (Show Details)Apr 23 2024, 10:15 AM
gerritbot added a comment.Apr 25 2024, 8:06 AM

Change #1019007 merged by jenkins-bot:

[operations/deployment-charts@master] eventgate: Update mesh modules

https://gerrit.wikimedia.org/r/1019007

gerritbot added a comment.Apr 25 2024, 8:06 AM

Change #1019018 merged by jenkins-bot:

[operations/deployment-charts@master] eventgate-*: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1019018

JMeybohm updated the task description. (Show Details)Apr 25 2024, 9:34 AM
Maintenance_bot removed a project: Patch-For-Review.Apr 26 2024, 6:54 PM
BTullis updated the task description. (Show Details)May 9 2024, 10:18 AM
BTullis updated the task description. (Show Details)
BTullis updated the task description. (Show Details)May 9 2024, 10:50 AM
gerritbot added a comment.May 9 2024, 6:27 PM

Change #1028910 had a related patch set uploaded (by Scott French; author: Scott French):

[operations/deployment-charts@master] benthos: adopt securityContext and base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1028910

gerritbot added a project: Patch-For-Review.May 9 2024, 6:27 PM
gerritbot added a comment.May 14 2024, 3:43 PM

Change #1028910 merged by jenkins-bot:

[operations/deployment-charts@master] benthos: adopt securityContext and base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1028910

Maintenance_bot removed a project: Patch-For-Review.May 14 2024, 4:32 PM
gerritbot added a comment.May 22 2024, 11:46 AM

Change #1033405 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/deployment-charts@master] Migrate AQS2 services and image-suggestions to calico network policies

https://gerrit.wikimedia.org/r/1033405

gerritbot added a project: Patch-For-Review.May 22 2024, 11:46 AM
gerritbot added a comment.May 24 2024, 11:05 AM

Change #1033405 merged by jenkins-bot:

[operations/deployment-charts@master] Migrate AQS2 services and image-suggestions to calico network policies

https://gerrit.wikimedia.org/r/1033405

Maintenance_bot removed a project: Patch-For-Review.May 24 2024, 11:31 AM
Scott_French updated the task description. (Show Details)May 31 2024, 10:48 PM
gerritbot added a comment.Jun 1 2024, 12:23 AM

Change #1037870 had a related patch set uploaded (by Scott French; author: Scott French):

[operations/deployment-charts@master] eventstreams: adopt base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1037870

gerritbot added a project: Patch-For-Review.Jun 1 2024, 12:23 AM
BTullis closed subtask T364533: Migrate AQS and image-suggestions services to Calico Network Policies as Resolved.Jun 4 2024, 4:08 PM
JMeybohm updated the task description. (Show Details)Jun 7 2024, 3:38 PM
brouberol updated the task description. (Show Details)Jun 10 2024, 6:57 AM
gerritbot added a comment.Jun 10 2024, 7:29 AM

Change #1040874 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] datahub-next: replace IPs by Services in network policies

https://gerrit.wikimedia.org/r/1040874

gerritbot added a comment.Jun 10 2024, 7:29 AM

Change #1040875 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] datahub: replace IPs by Services in network policies

https://gerrit.wikimedia.org/r/1040875

brouberol updated the task description. (Show Details)Jun 10 2024, 7:38 AM
gerritbot added a comment.Jun 11 2024, 9:55 AM

Change #1040874 merged by Brouberol:

[operations/deployment-charts@master] datahub-next: replace IPs by Services in network policies

https://gerrit.wikimedia.org/r/1040874

gerritbot added a comment.Jun 11 2024, 2:31 PM

Change #1041671 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] datahub: update datahubsearch hostname to use external-services

https://gerrit.wikimedia.org/r/1041671

gerritbot added a comment.Jun 11 2024, 2:54 PM

Change #1041676 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] Deploy calico network policy templates to all datahub charts

https://gerrit.wikimedia.org/r/1041676

JMeybohm updated the task description. (Show Details)Jun 12 2024, 1:21 PM
gerritbot added a comment.Jun 13 2024, 8:57 AM

Change #1041676 merged by Brouberol:

[operations/deployment-charts@master] Deploy calico network policy templates to all datahub charts

https://gerrit.wikimedia.org/r/1041676

gerritbot added a comment.Jun 13 2024, 9:16 AM

Change #1042952 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] datahub-next: restore IP-based networkpolicy to datahubsearch

https://gerrit.wikimedia.org/r/1042952

gerritbot added a comment.Jun 13 2024, 9:17 AM

Change #1041671 abandoned by Brouberol:

[operations/deployment-charts@master] datahub: update datahubsearch hostname to use external-services

Reason:

https://gerrit.wikimedia.org/r/1041671

gerritbot added a comment.Jun 13 2024, 9:32 AM

Change #1042964 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] datahub: fix label matching beetween pods and networkpolicies

https://gerrit.wikimedia.org/r/1042964

gerritbot added a comment.Jun 13 2024, 9:50 AM

Change #1042964 merged by jenkins-bot:

[operations/deployment-charts@master] datahub: fix label matching beetween pods and networkpolicies

https://gerrit.wikimedia.org/r/1042964

gerritbot added a comment.Jun 13 2024, 9:51 AM

Change #1042952 merged by jenkins-bot:

[operations/deployment-charts@master] datahub-next: restore IP-based networkpolicy to datahubsearch

https://gerrit.wikimedia.org/r/1042952

gerritbot added a comment.Jun 13 2024, 10:37 AM

Change #1040875 merged by Brouberol:

[operations/deployment-charts@master] datahub: replace IPs by Services in network policies

https://gerrit.wikimedia.org/r/1040875

akosiaris updated the task description. (Show Details)Jul 9 2024, 1:55 PM
JMeybohm updated the task description. (Show Details)Aug 22 2024, 3:26 PM
JMeybohm added a subscriber: Scott_French.
brouberol updated the task description. (Show Details)Aug 22 2024, 3:37 PM
gerritbot added a comment.Aug 26 2024, 5:33 PM

Change #1037870 merged by jenkins-bot:

[operations/deployment-charts@master] eventstreams: adopt base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1037870

Scott_French updated the task description. (Show Details)Aug 26 2024, 6:16 PM
klausman updated the task description. (Show Details)Aug 27 2024, 9:03 AM
klausman updated the task description. (Show Details)Aug 27 2024, 9:11 AM
akosiaris updated the task description. (Show Details)Aug 27 2024, 4:16 PM
BTullis updated the task description. (Show Details)Aug 27 2024, 5:07 PM
gerritbot added a comment.Aug 29 2024, 12:37 PM

Change #1068754 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/puppet@production] global_config: Add pki::multirootca IPs to external-services

https://gerrit.wikimedia.org/r/1068754

gerritbot added a comment.Aug 29 2024, 12:40 PM

Change #1068768 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] cfssl-issuer: Add external-services support

https://gerrit.wikimedia.org/r/1068768

gerritbot added a comment.Aug 30 2024, 6:56 AM

Change #1068754 merged by JMeybohm:

[operations/puppet@production] global_config: Add pki::multirootca IPs to external-services

https://gerrit.wikimedia.org/r/1068754

gerritbot added a comment.Sep 2 2024, 2:39 PM

Change #1068768 merged by jenkins-bot:

[operations/deployment-charts@master] cfssl-issuer: Add external-services support

https://gerrit.wikimedia.org/r/1068768

Maintenance_bot removed a project: Patch-For-Review.Sep 2 2024, 3:31 PM
bking updated the task description. (Show Details)Sep 9 2024, 5:37 PM
bking updated the task description. (Show Details)Sep 24 2024, 1:47 PM
gmodena subscribed.Sep 25 2024, 11:01 AM
bking closed subtask T373195: Migrate Search Platform-owned helm charts to Calico Network Policies as Resolved.Nov 8 2024, 5:12 PM
Scott_French updated the task description. (Show Details)Feb 10 2026, 9:13 PM
Scott_French edited projects, added: ServiceOps new; removed: serviceops-deprecated.Feb 10 2026, 9:19 PM

Checking in, it looks like we still have a couple of services on Wikikube that require migration, most notably MediaWiki (or, IIUC, cleanup of the older duplicate network policies, as in the tegola-vector-tiles case).

@JMeybohm - Do you happen to recall whether there was an earlier effort to adopt external-services in MediaWiki, and / or what, if anything might have blocked that?

Scott_French moved this task from Inbox to Backlog on the ServiceOps new board.Feb 10 2026, 9:19 PM
JMeybohm updated the task description. (Show Details)Feb 11 2026, 8:57 AM
In T359423#11604574, @Scott_French wrote:

@JMeybohm - Do you happen to recall whether there was an earlier effort to adopt external-services in MediaWiki, and / or what, if anything might have blocked that?

AFAIK no efforts have been made so far, probably a :scooby afraid: situation.

JMeybohm added a subscriber: DPogorzelski-WMF.Feb 11 2026, 9:04 AM

@DPogorzelski-WMF could you please verify if this is (sill) relevant for T416580: Kserve helm chart/T380723: Update knative-serving+net-istio to v1.12.x on ML clusters?

Clement_Goubert subscribed.Feb 13 2026, 10:44 AM
In T359423#11606179, @JMeybohm wrote:
In T359423#11604574, @Scott_French wrote:

@JMeybohm - Do you happen to recall whether there was an earlier effort to adopt external-services in MediaWiki, and / or what, if anything might have blocked that?

AFAIK no efforts have been made so far, probably a :scooby afraid: situation.

For risk management/clarity as it may require more exploratory work than other charts, I think work to migrate mediawiki to external-services should be tracked in its own task.

Clement_Goubert created subtask T417383: Migrate mediawiki to external services.Feb 13 2026, 10:49 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits