In T404804#11231334, @Urbanecm_WMF wrote:

I spent some time looking into how we can get more data about this. First, I was thinking how to trigger profiling code from src/Profiler.php (in the config repo) in a maintenance script context. [..]

In production, mw-experimental is the environment for livehacking. This made me ask: how can I run maintenance scripts in mw-experimental?

It turns out that with the mw-experimental-deploy credentials, I can run kubectl exec, shelling into the mw-experimental pod, running /srv/mediawiki/multiversion/MWScript.php manually. So, I made the above-described changes in mw-experimental using the documented process, and then run the maintenance script as follows:

[urbanecm@deploy2002 ~]$ kubectl exec -c mediawiki-pinkllama-app mw-experimental.codfw.pinkllama-779d86d874-znl5n -it -- php /srv/mediawiki/multiversion/MWScript.php Version.php --wiki=dewiki

[..]

@Krinkle Please have a go running P84273 on the deployment host, and let me know if it helps

In T341197: Allow deployers to get a php REPL environment inside the mw-debug pods, @Joe added mw-debug-repl to get shell.php in the debug pods. Given this is the second pod in which running shell.php makes sense, I'm wondering, should we create a more generic wrapper?

(If we end up adding an extra wrapper, we should probably name it consistently, so either rename mw-debug-repl to mw-debug-shell or the other way around)

(Noting that for some reasons [historical ones, maybe?], mw-debug-repl requires root privileges. However, it seems like this task is now possible using deploy credentials. Maybe it should be rewritten in any case?)

In T405688#11302125, @jijiki wrote:

@Krinkle Please have a go running P84273 on the deployment host, and let me know if it helps

This is great. I've used it several times already!

Managing the file system separately from the shell is a bit confusing (one shell on the worker host with writable filesystem but without MW/PHP, another shell from the deploy host in the container with MW/PHP but without writable filesystem where you then use those files created/edited in the other shell). But, it works!

I wonder if the container could be read-write (i.e. ephemeral empty writable layer over top of /srv/mediawiki in the container?). That might pose some challenges with managing outdated state and forgotten/long-running containers, but would make this a bit closer to the level of ease it used to have on mwdebug hosts. Anyway, this is great!

It would also be great to be able to run maintenance scripts on multiple wikis (i.e. an equivalent of foreachwiki or mwscript-k8s --dblist) on mw-experimental. I tried out P84273 and couldn’t see a way to do it there, just individual wikis:

www-data@mw-experimental-eqiad:/$ php /srv/mediawiki/multiversion/MWScript.php namespaceDupes enwiki

(Use case: testing a Gerrit change that potentially affects namespace aliases in many languages.)

There's no reason why something like this:

www-data@mw-experimental-eqiad:/$ for wiki in `php /srv/mediawiki/multiversion/bin/expanddblist private`; do
> echo -e "------\n$wiki\n-----"
> php /srv/mediawiki/multiversion/MWScript.php Version.php --wiki="$wiki"
> done
------
advisorswiki
-----
MediaWiki version: 1.46.0-wmf.22 (built: 14:07, 1 April 2026)
------
arbcom_cswiki
-----
MediaWiki version: 1.46.0-wmf.22 (built: 1. 4. 2026, 16:07)
------
arbcom_dewiki
-----
MediaWiki version: 1.46.0-wmf.22 (built: 16:07, 1. Apr. 2026)
------
arbcom_enwiki
-----
^C
www-data@mw-experimental-eqiad:/$

shouldn't work. But maybe we should get something that doesn't require manual for loops?

In T405688#11778865, @Lucas_Werkmeister_WMDE wrote:

It would also be great to be able to run maintenance scripts on multiple wikis (i.e. an equivalent of foreachwiki or mwscript-k8s --dblist) on mw-experimental. I tried out P84273 and couldn’t see a way to do it there, just individual wikis:

www-data@mw-experimental-eqiad:/$ php /srv/mediawiki/multiversion/MWScript.php namespaceDupes enwiki

(Use case: testing a Gerrit change that potentially affects namespace aliases in many languages.)

mw-experimental does not guarantee any reliability, and there are no plans to do so. It's fine for sanity-checking a maintenance script, but if you have needs that mw-script doesn't currently serve, please file a task for ServiceOps-Mediawiki and we'll see what we can work out.

I don’t see what reliability has to do with it? We wanted to know the impact of adding a bunch of namespace aliases in many different languages (would this “shadow” any existing pages?), and it seemed to me that the best way to do that would be to dry-run namespaceDupes on all wikis, with the patch experimentally applied, and then inspect the maintenance script’s output (and compare it with a “regular” dry-run without the patch).

In T405688#11781939, @Lucas_Werkmeister_WMDE wrote:

I don’t see what reliability has to do with it? We wanted to know the impact of adding a bunch of namespace aliases in many different languages (would this “shadow” any existing pages?), and it seemed to me that the best way to do that would be to dry-run namespaceDupes on all wikis, with the patch experimentally applied, and then inspect the maintenance script’s output (and compare it with a “regular” dry-run without the patch).

As I mentioned, mw-experimental is fine for sanity-checking, which is exactly what this use case sounds like. The "no reliability guarantees" note refers to the infrastructure: the pod or host can vanish at any time without notice.

Okay, then it sounds like mw-experimental is the right environment for what I wanted to do and I only wish it was easier to run maintenance scripts on one or more wikis there :)