Currently, the periodic jobs rely on a few helper scripts:
Top-level wrapper
/usr/local/bin/mw-cli-wrapper
Responsible for checking the mediawiki read-only state and primary datacenter. To be updated to T367118: Control mw-on-k8s periodic maintenance jobs with an etcd value
General helper scripts
/usr/local/bin/foreachwiki # Wraps /usr/local/bin/foreachwikiindblist /usr/local/bin/foreachwikiindblist # Wraps /usr/local/bin/expanddblist and calls php /usr/local/bin/expanddblist /usr/local/bin/mwscriptwikiset # Wraps /usr/local/bin/mwscript /usr/local/bin/mwscript # Calls php
These scripts all end up wrapping a php cli invocation using environment variables from /etc/profile.d/mediawiki.sh. These environment variables can be set in the Job Spec.
Ad-hoc scripts for a specific periodic job
/usr/local/bin/captchaloop # Wraps /usr/local/bin/mwscript /usr/local/bin/characterEditStatsTranslate # Wraps /usr/local/bin/mwscript /usr/local/bin/cirrus_build_completion_indices.sh # Wraps /usr/local/bin/expanddblist
Integration into the k8s CronJob world
For the general helper scripts (including /usr/local/bin/expanddblist), they should probably be included or injected in the container used to run the cronjobs rather than replicated using the container command in the jobTemplate.
There's a few options there, roughly listed from preferred to least preferred in my opinion:
- Have a new mediawiki-multiversion-cli image that includes those helper scripts: Cleaner as far as separation of concerns goes, but adds one more image to maintain and build. Will need to install python for /usr/local/bin/mw-cli-wrapper
- Use the production mediawiki-multiversion and inject the scripts as configmaps: Avoids building and maintaining a new image, while avoiding adding shell scripts to exposed containers. Feels kind of dirty as it's not configuration, but allows to just add the puppet definition for these scripts onto the deployment servers and for a faster feedback loop for their modification, as they would not need a new image to pick up the modifications.
- Include the scripts in the mediawiki-multiversion-debug image: Avoids building and maintaining one more image, limits exposure to a couple of containers and can (maybe?) be useful to have the scripts for debugging. However, the pods are still exposed to outside traffic, raising security concerns.
- Include the scripts in the production image:
Adding shell scripts to all traffic-exposed containers is a security issue IMO, I'm only including it for completeness.Actually, it's not that big a deal, if these get run, we have more problems than these scripts being in the image.
Decision
- Add a new helper script that replicates the existing general helper scripts and add them to our images [https://gerrit.wikimedia.org/r/c/operations/docker-images/production-images/+/1082478|1082478]