Page MenuHomePhabricator
Create Task
Maniphest T359423

Migrate charts to Calico Network Policies
Open, MediumPublic
Actions

Description

The following charts will need to be migrated to the base.networkpolicy.egress.external-services helm template, rendering Calico network policies:

  • benthos-cache-invalidator (also needs securityContext update: T362978) @kamila
  • changeprop
  • datahub (also needs mesh.configuration and securityContext update: T346638, T362978) @BTullis
  • eventgate (also needs mesh.configuration and securityContext update: T346638, T362978) @JMeybohm
    • eventgate-analytics
    • eventgate-analytics-external
    • eventgate-logging-external
    • eventgate-main
  • eventstreams (also needs securityContext update: T362978)
    • eventstreams
    • eventstreams-internal
  • flink-app (also needs mesh.configuration and securityContext update: T346638, T362978)
    • cirrus-streaming-updater
    • mw-page-content-change-enrich
    • rdf-streaming-updater
  • flink-operator (also needs securityContext update: T362978)
  • mediawiki (also needs securityContext update: T362978)
    • mw-debug
    • all other mediawiki releases
  • tegola-vector-tiles (also needs mesh.configuration and securityContext update: T346638, T362978)
  • spark-history @brouberol
  • kserve-inference (also needs securityContext update: T362978) @klausman

Details

Show related patches Customize query in gerrit

Related Objects
Search...

Event Timeline

brouberol created this task.Mar 6 2024, 4:57 PM
Gehel triaged this task as Medium priority.Mar 20 2024, 9:04 AM
Gehel moved this task from Incoming to Toil / Automation on the Data-Platform-SRE board.
brouberol updated the task description. (Show Details)Mar 25 2024, 12:37 PM
gerritbot added a comment.Mar 25 2024, 12:44 PM

Change #1013989 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] spark-history: add external-services egress network policy template

https://gerrit.wikimedia.org/r/1013989

gerritbot added a project: Patch-For-Review.Mar 25 2024, 12:44 PM

Change #1013990 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] spark-history: replace hardcoded CIDRs by service names to generate egress policies

https://gerrit.wikimedia.org/r/1013990

gerritbot added a comment.Mar 25 2024, 12:57 PM

Change #1013989 merged by Brouberol:

[operations/deployment-charts@master] spark-history: add external-services egress network policy template

https://gerrit.wikimedia.org/r/1013989

gerritbot added a comment.Mar 25 2024, 1:06 PM

Change #1013990 merged by Brouberol:

[operations/deployment-charts@master] spark-history: replace hardcoded CIDRs by service names to generate egress policies

https://gerrit.wikimedia.org/r/1013990

gerritbot added a comment.Mar 25 2024, 1:11 PM

Change #1013997 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] spark-history: fix egress network policies

https://gerrit.wikimedia.org/r/1013997

gerritbot added a comment.Mar 25 2024, 1:22 PM

Change #1013997 merged by Brouberol:

[operations/deployment-charts@master] spark-history: fix egress network policies

https://gerrit.wikimedia.org/r/1013997

gerritbot added a comment.Mar 25 2024, 1:51 PM

Change #1014010 had a related patch set uploaded (by Brouberol; author: Brouberol):

[operations/deployment-charts@master] spark-history: bypass Kerberos principal hostname reverse DNS check for namenode

https://gerrit.wikimedia.org/r/1014010

gerritbot added a comment.Mar 26 2024, 10:26 AM

Change #1014010 merged by Brouberol:

[operations/deployment-charts@master] spark-history: bypass Kerberos principal hostname reverse DNS check for namenode

https://gerrit.wikimedia.org/r/1014010

gerritbot added a comment.Mar 26 2024, 3:48 PM

Change #1014538 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] changeprop: Update mesh modules

https://gerrit.wikimedia.org/r/1014538

gerritbot added a comment.Mar 26 2024, 3:48 PM

Change #1014539 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] changeprop: Add base.external-services-networkpolicy:1.0

https://gerrit.wikimedia.org/r/1014539

gerritbot added a comment.Mar 26 2024, 3:48 PM

Change #1014540 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] changeprop: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1014540

gerritbot added a comment.Mar 26 2024, 3:52 PM

Change #1014542 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] changeprop-jobqueue: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1014542

JMeybohm updated the task description. (Show Details)Mar 26 2024, 4:03 PM
brouberol updated the task description. (Show Details)Mar 26 2024, 4:06 PM
BTullis updated the task description. (Show Details)Mar 26 2024, 10:26 PM
gerritbot added a comment.Mar 26 2024, 10:27 PM

Change #1014646 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/deployment-charts@master] Fix typo in the external-services values for datahub staging

https://gerrit.wikimedia.org/r/1014646

BTullis added a comment.EditedMar 26 2024, 10:30 PM

I needed to start work on the DataHub migration because SSO broke due to the switch of IDP servers: https://sal.toolforge.org/log/UMCPdY4BhuQtenzvi5Pe

Slack thread here.

CR here: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1014065

So far, I have only migrated the CAS IDP services, but there are still several others that can be migrated too, including:

  • datahubsearch
  • mariadb (analytics_meta)

We can probably remove the LDAP policies from this deployment, now that CAS is enabled. Otherwise I would migrate that too.

gerritbot added a comment.Mar 26 2024, 10:54 PM

Change #1014646 merged by jenkins-bot:

[operations/deployment-charts@master] Fix whitespace in the external-services values for datahub staging

https://gerrit.wikimedia.org/r/1014646

BTullis updated the task description. (Show Details)Mar 26 2024, 10:58 PM
BTullis updated the task description. (Show Details)Mar 26 2024, 11:08 PM
gerritbot added a comment.Mar 27 2024, 10:52 AM

Change #1014652 had a related patch set uploaded (by Btullis; author: Btullis):

[operations/deployment-charts@master] Make datahub networkpolicy include/template consistent

https://gerrit.wikimedia.org/r/1014652

brouberol updated the task description. (Show Details)Mar 27 2024, 12:50 PM
brouberol added a subscriber: klausman.
gerritbot added a comment.Mar 27 2024, 12:51 PM

Change #1015029 had a related patch set uploaded (by Klausman; author: Klausman):

[operations/deployment-charts@master] charts/kserve-inference: Wire up generated network policy for LW services

https://gerrit.wikimedia.org/r/1015029

gerritbot added a comment.Mar 27 2024, 1:23 PM

Change #1014652 merged by jenkins-bot:

[operations/deployment-charts@master] Make datahub networkpolicy include/template consistent

https://gerrit.wikimedia.org/r/1014652

kamila updated the task description. (Show Details)Mar 27 2024, 2:43 PM
kamila subscribed.
gerritbot added a comment.Mar 27 2024, 3:37 PM

Change #1014538 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop: Update mesh modules

https://gerrit.wikimedia.org/r/1014538

gerritbot added a comment.Mar 27 2024, 3:38 PM

Change #1014539 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop: Add base.external-services-networkpolicy:1.0

https://gerrit.wikimedia.org/r/1014539

gerritbot added a comment.Mar 27 2024, 3:38 PM

Change #1014540 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1014540

gerritbot added a comment.Mar 27 2024, 4:03 PM

Change #1014542 merged by jenkins-bot:

[operations/deployment-charts@master] changeprop-jobqueue: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1014542

JMeybohm updated the task description. (Show Details)Mar 27 2024, 4:16 PM
BTullis mentioned this in T360531: [Commons Impact Metrics] Refactor/create helm config for AQS service accessing both Cassandra and Druid.Mar 28 2024, 11:06 AM
gerritbot added a comment.Mar 28 2024, 11:28 AM

Change #1015029 merged by jenkins-bot:

[operations/deployment-charts@master] charts/kserve-inference: Wire up generated network policy for LW services

https://gerrit.wikimedia.org/r/1015029

JMeybohm updated the task description. (Show Details)Wed, Apr 10, 12:44 PM

Grouped the todo list by chart, some of those also need mesh.configuration updates due to T346638: Rename the envoy's uses_ingress option to sets_sni which could be bundled with this

gerritbot added a comment.Thu, Apr 11, 11:40 AM

Change #1019007 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] eventgate: Update mesh modules

https://gerrit.wikimedia.org/r/1019007

gerritbot added a comment.Thu, Apr 11, 12:02 PM

Change #1019018 had a related patch set uploaded (by JMeybohm; author: JMeybohm):

[operations/deployment-charts@master] eventgate-*: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1019018

JMeybohm updated the task description. (Show Details)Fri, Apr 12, 7:32 AM
JMeybohm updated the task description. (Show Details)Tue, Apr 23, 10:15 AM
gerritbot added a comment.Thu, Apr 25, 8:06 AM

Change #1019007 merged by jenkins-bot:

[operations/deployment-charts@master] eventgate: Update mesh modules

https://gerrit.wikimedia.org/r/1019007

gerritbot added a comment.Thu, Apr 25, 8:06 AM

Change #1019018 merged by jenkins-bot:

[operations/deployment-charts@master] eventgate-*: Migrate to base.external-services-networkpolicy

https://gerrit.wikimedia.org/r/1019018

JMeybohm updated the task description. (Show Details)Thu, Apr 25, 9:34 AM
Maintenance_bot removed a project: Patch-For-Review.Fri, Apr 26, 6:54 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL