Page MenuHomePhabricator

Identify path forward for k8s deployment of prometheus-statsd-exporter
Closed, ResolvedPublic

Description

This task aims to define the optimal approach to deploy prometheus-statsd-exporter on k8s as a sidecar process to MediaWiki.

There is uncertainty about the correct direction to proceed. Assistance is requested to identify the best path forward.

This task is a placeholder to track the discussion, decision, and conversation outcomes.

Event Timeline

colewhite renamed this task from Identify path forward for k8s deployment of StatsLib and statsd-prometheus-exporter to Identify path forward for k8s deployment of prometheus-statsd-exporter.Aug 4 2023, 11:35 PM
colewhite updated the task description. (Show Details)
lmata added a subscriber: Kappakayala.

@Kappakayala, this is a task for the K8s work.

The prometheus-statsd-exporter container and configuration is already deployed as a side-car for a number of kubernetes services:

deployment-charts/charts on  master [$?⇡] 
❯ git grep statsd-exporter | cut -d/ -f1 | sort -u
apertium
api-gateway
blubberoid
changeprop
developer-portal
eventstreams
linkrecommendation
machinetranslation
mediawiki-dev
miscweb
mobileapps
recommendation-api
shellbox
termbox
thumbor
toolhub
wikifeeds

I think the best path forward would be to extract this repeated deployment to a sextant module (like we do for lamp or ingress), and use that in the mediawiki chart and all others.
@Joe opinions?

@Clement_Goubert yes that is my plan. I planned on picking up the task next week after the switchover dust settled and we have the new k8s nodes in rotation.

Change 959803 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/deployment-charts@master] modules: add base.statsd

https://gerrit.wikimedia.org/r/959803

Joe changed the task status from Open to In Progress.Sep 21 2023, 4:04 PM
Joe claimed this task.
Joe moved this task from 🌻Mediawiki to Doing 😎 on the serviceops board.

Change 959948 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/deployment-charts@master] thumbor: use base.statsd

https://gerrit.wikimedia.org/r/959948

Change 960543 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/deployment-charts@master] apertium: use base.statsd

https://gerrit.wikimedia.org/r/960543

Change 960568 had a related patch set uploaded (by Giuseppe Lavagetto; author: Giuseppe Lavagetto):

[operations/deployment-charts@master] mediawiki: add support for a prometheus-statsd-exporter

https://gerrit.wikimedia.org/r/960568

When you have your code in core, and we have merged the above patches, we can start testing your code specifically on k8s, first on the debug cluster, then everywhere.

Change 959803 merged by jenkins-bot:

[operations/deployment-charts@master] modules: add base.statsd

https://gerrit.wikimedia.org/r/959803

Change 960568 merged by jenkins-bot:

[operations/deployment-charts@master] mediawiki: add support for a prometheus-statsd-exporter

https://gerrit.wikimedia.org/r/960568

@colewhite prometheus statsd exporter can be now installed in mw on k8s with a configuration switch. Resolving this task for now.