Page MenuHomePhabricator

Create separate ArcLamp pipeline for k8s-mwdebug
Closed, ResolvedPublic

Description

Same as we did during the Buster migration.

The goal is to be able to compare profiling data with non-k8s deployment.

Event Timeline

Change 711166 had a related patch set uploaded (by Dave Pifke; author: Dave Pifke):

[operations/puppet@production] arclamp: add temporary excimer-k8s pipeline

https://gerrit.wikimedia.org/r/711166

Change 711580 had a related patch set uploaded (by Dave Pifke; author: Dave Pifke):

[operations/mediawiki-config@master] profiler: use seperate pipeline inside k8s pods

https://gerrit.wikimedia.org/r/711580

Change 711166 merged by Dzahn:

[operations/puppet@production] arclamp: add temporary excimer-k8s pipeline

https://gerrit.wikimedia.org/r/711166

Change 711580 merged by jenkins-bot:

[operations/mediawiki-config@master] profiler: use seperate pipeline inside k8s pods

https://gerrit.wikimedia.org/r/711580

Mentioned in SAL (#wikimedia-operations) [2021-08-31T23:33:02Z] <dpifke@deploy1002> Synchronized wmf-config/profiler.php: Revert excimer-k8s pipelines T288165 (duration: 01m 14s)

Change 716041 had a related patch set uploaded (by Dave Pifke; author: Dave Pifke):

[operations/mediawiki-config@master] profiler: use seperate pipeline inside k8s pods

https://gerrit.wikimedia.org/r/716041

Change 716041 merged by jenkins-bot:

[operations/mediawiki-config@master] profiler: use seperate pipeline inside k8s pods

https://gerrit.wikimedia.org/r/716041

Mentioned in SAL (#wikimedia-operations) [2021-09-07T23:07:35Z] <dpifke@deploy1002> Synchronized wmf-config/profiler.php: Config: [[gerrit:716041|profiler: use seperate pipeline inside k8s pods (T288165)]] (duration: 00m 58s)

The config patch is live, but we're not yet getting any data:

dpifke@webperf1002:/srv/xenon/logs/hourly$ ls *k8s*
ls: cannot access '*k8s*': No such file or directory

I see the subscribers active, along with the corresponding channels in Redis:

127.0.0.1:6379> pubsub channels
1) "excimer-k8s-wall"
2) "excimer-wall"
3) "excimer-k8s"
4) "excimer"

My guess is we're missing something in the mwdebug-k8s image; I'll investigate more tomorrow.

Change 719565 had a related patch set uploaded (by Dave Pifke; author: Dave Pifke):

[operations/mediawiki-config@master] pipeline: include php-excimer and php-redis

https://gerrit.wikimedia.org/r/719565

Change 719609 had a related patch set uploaded (by Dave Pifke; author: Dave Pifke):

[operations/docker-images/production-images@master] fpm-multiversion-base: add php-excimer extension

https://gerrit.wikimedia.org/r/719609

Change 719565 abandoned by Dave Pifke:

[operations/mediawiki-config@master] pipeline: include php-excimer extension

Reason:

https://gerrit.wikimedia.org/r/c/operations/docker-images/production-images/+/719609

https://gerrit.wikimedia.org/r/719565

Change 719609 merged by Legoktm:

[operations/docker-images/production-images@master] fpm-multiversion-base: add php-excimer extension

https://gerrit.wikimedia.org/r/719609

This is working now.

The sampling rate is the same as in production, so it's unlikely to produce much output for one-off requests via WikimediaDebug (which are probably better profiled via XHGui anyways), but will still produce useful data during benchmarking runs.