Description
Details
Status | Subtype | Assigned | Task | ||
---|---|---|---|---|---|
Resolved | daniel | T99268 RfC: Create a proper command-line runner for MediaWiki maintenance tasks | |||
Resolved | daniel | T314231 Instantiate MaintenanceScripts after extensions have registered with the autoloader | |||
Resolved | Ladsgroup | T326800 Make Wikimedia mwscript use run.php to run maintenance scripts | |||
Resolved | hashar | T326333 Update quibble jobs to run maintenance scripts via maintenance/run.php |
Event Timeline
I'm an outreachy applicant. Can I take up this task? How should I get started with this?
Hi @Dikshagupta99 and thanks for your interest! Have you taken a look at https://www.mediawiki.org/wiki/Requests_for_comment/Proper_command-line_runner_for_maintenance_tasks already? If yes, do you have a more specific question?
@WDoranWMF @kchapman @daniel Would you all be interested in promoting/mentoring this project via Google Summer of Code 2020 or Outreachy Round 20?
Hi! This project seems interesting to me and I would like to contribute to it via GSoC'20. But before that I have some queries:
- Is the command runner we are planning is similar to the way we use Git? Like we enter git on the terminal or command prompt and we see all the commands that can be run with one line description about each command.
- Can you please link the source of maintenance scripts we are planning to include?!
Thanks in advance!
@Soumyaa1804 Thanks for your interest! FYI, we are not planning to promote this project via GSoC and might end up promoting via Outreachy. We will finalize in the next few days.
Okay! I would be interested to work on this even in Outreachy if it gets promoted there and my initial application gets selected. :)
(as per the program rules need to restrict access to this project task to #Outreachy-mentors group until the application period begins)
Hi! @daniel I would like to contribute, can you please assign me some task to get started with?
@Akansha99: See https://phabricator.wikimedia.org/maniphest/query/rDGv.ANXQ4ce/#R (if Daniel does not correct me)
@srishakatux: I am wondering if step 9 on https://www.mediawiki.org/wiki/Outreachy/Participants should link to https://www.mediawiki.org/wiki/Good_first_bugs
@Aklapper Yes, that's a good point; I've added the link from the Outreachy/Participants page.
We are unlisting this project from Outreachy (Round 20). If you a potential intern, please explore other projects here https://www.mediawiki.org/wiki/Outreachy/Round_20#Ideas_for_projects.
Change 693134 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):
[mediawiki/core@master] WIP: Symfony Console based CLI for maintenance scripts
I'll be doing some more exploration around the patch above during the hackathon, if anyone's interested to code / critique / test, help is welcome :)
I noticed today that one of the first commits in the Maintenance system actually had a class-based registry, and while we lost that at some point, it connected the dots for me to something we still do today: The maintenance scripts are in the autoloader and each script has logic to avoid execution if loaded outside their own entry point. This was I believe designed to be run via a wrapper one day.
From the original commit:
$wgMaintenanceScripts = array();
Register their maintenance scripts [class name] with the system
$wgMaintenanceScripts for extensions to add their scripts to the default list.
public static function getMaintenanceScripts() { global $wgMaintenanceScripts; return $wgMaintenanceScripts + self::getCoreScripts(); }
http://mediawiki.org/wiki/Special:Code/MediaWiki/54225
https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/a1c51e18af85a9ac464c5b555921e58ec422cd11%5E%21/
If someone is willing to make small patches doing this step-by-step, I'd be happy to review it.
Last time I tried, I got lost in the way this is doing setup and loading config. Perhaps with SettingsBuilder, it can now be done in a saner way...
@Ladsgroup, @Krinkle, @kostajh: I have an experimantal chain of patches up, starting at https://gerrit.wikimedia.org/r/c/mediawiki/core/+/782569. It still needs some polishing, testing, and bikeshedding about names, but it seems to work. I'd love to hear your thoughts on the approach I'm takeing there, before I invest more time.
Change 818574 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):
[mediawiki/core@master] ExtensionRegistry: split exportExtractedData
Change 783428 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):
[mediawiki/core@master] Introduce runner entry point
I think this is mostly done, there is one last push needed to get the last piece over the line.
Change 818574 abandoned by Daniel Kinzler:
[mediawiki/core@master] ExtensionRegistry: split exportExtractedData
Reason:
Going with I6f8f9f3f7252f0024282d7b005671f28a5b3acc3 instead
Change 783428 merged by jenkins-bot:
[mediawiki/core@master] Introduce run.php for running maintenance scripts
amir@amir-ThinkPad-P1-Gen-3:/var/lib/mediawiki$ php maintenance/run.php extensions/AbuseFilter/maintenance/UpdateVarDumps.php --force ...the abuse_filter_log table is empty. amir@amir-ThinkPad-P1-Gen-3:/var/lib/mediawiki$ php maintenance/run.php MediaWiki.Extension.AbuseFilter.Maintenance.UpdateVarDumps --force ...the abuse_filter_log table is empty.
This is great, but the new warning introduced in rMW4912fe27a8e0: Maintenance scripts: show a warning when run directly. shown for old-style scripts is likely to mess up any automation and should be reverted IMO (or maybe it should try to detect whether it is writing to an interactive terminal).
Vagrant for example breaks quite spectacularly: T326522: Vagrant mediawiki::maintenance resource broken by new maintenance script output
Probably not, stderr can also cause issues, sometimes it's taken as a sign that the script failed, sometimes it's piped together with normal output etc.
It might be better than stdout, I'm not sure, but either way warnings should be limited to interactive mode. Or use the deprecation warning system, etc.
Change 876792 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):
[mediawiki/core@master] Do not add random text to maint script output when not in TTY mode
Change 876792 merged by jenkins-bot:
[mediawiki/core@master] maintenance: Do not output run.php warning when not in TTY mode
Change 693134 abandoned by Kosta Harlan:
[mediawiki/core@master] WIP: Symfony Console based CLI for maintenance scripts
Reason:
Change 1009784 had a related patch set uploaded (by Ahmon Dancy; author: Ahmon Dancy):
[operations/puppet@production] mw-xml.sh: Update maintenance script