MediaWiki has the mwmaint servers available as locations to run maintenance scripts, manual jobs, database queries, and interact with wikis via a REPL.
These same sorts of tasks can be done from within a Toolhub container, but that container needs to be running in an environment where the Django app can connect to the production database, elasticsearch cluster, and memcached servers. Docker is not currently available on mwmaint boxes to run the container, and the permissions for the 'toolhub' user in the Kubernetes clusters do not grant pod/exec[create delete] rights to attach to a running container.
Non-exhaustive list of things that would be easier with cli access to a running Toolhub container:
- Database schema maintenance. Django includes a fairly nice system for managing forward and backward schema migrations including the ability to add migration steps that operate at an object graph level for complex data fills. Running DB_USER=toolhub_admin DB_PASSWORD=... poetry run python3 ./manage.py migrate from a container that can connect to m5-master would make db migrations much easier than requiring SQL export by Django with manual application of the DDL changes.
- Bootstrapping the database with seed data using Django fixture files
- Bootstrapping advanced user rights with Django's REPL
- Rebuilding the elasticsearch index when switching clusters
- Debugging issues occurring in production which cannot be replicated in a dev environment (connectivity, services only used in production, dataset related issues)
From an end user perspective, being able to kubectl exec -it toolhub-main-57dcd5447f-rsqk7 -- /bin/bash from a deploy server would be an ideal solution. It would also be reasonable to sudo somehow to do this rather than having the default kube_env toolhub staging provided credentials have this level of access.
Being able to launch a container in interactive mode on a mwmaint server or equivalent and not in a Kubernetes cluster directly would also be workable, but may have complications in configuration to reach service dependencies.
A nearly worst case solution would be to write action plans and hand them to SREs with container access to implement.
A worst case solution would be to require that all one-off commands actually be baked into Helm charts and deployed.