This RFC has been approved as proposed per the TechCom meeting on 2019-08-07.

Implementation will probably be taken on by the Platform Engineering at some point, but it's currently not high priority.

@CCicalese_WMF, do you have an idea how to fit this into our processes? Writing the framework shouldn't take long, it's not much work. Should be doable in a week or two of focused work. Converting existing scripts would take longer, but would be trivial.

@daniel , we are adding this as a future initiative.

I'm an outreachy applicant. Can I take up this task? How should I get started with this?

Hi @Dikshagupta99 and thanks for your interest! Have you taken a look at https://www.mediawiki.org/wiki/Requests_for_comment/Proper_command-line_runner_for_maintenance_tasks already? If yes, do you have a more specific question?

@WDoranWMF @kchapman @daniel Would you all be interested in promoting/mentoring this project via Google Summer of Code 2020 or Outreachy Round 20?

In T99268#5808193, @srishakatux wrote:

@WDoranWMF @kchapman @daniel Would you all be interested in promoting/mentoring this project via Google Summer of Code 2020 or Outreachy Round 20?

Generally yes - which time periods are we talking about exactly?

In T99268#5808426, @daniel wrote:

which time periods are we talking about exactly?

See https://developers.google.com/open-source/gsoc/timeline

Hi! This project seems interesting to me and I would like to contribute to it via GSoC'20. But before that I have some queries:

• Is the command runner we are planning is similar to the way we use Git? Like we enter git on the terminal or command prompt and we see all the commands that can be run with one line description about each command.
• Can you please link the source of maintenance scripts we are planning to include?!

Thanks in advance!

Hi, see the /maintenance folder in mediawiki/core.

@Soumyaa1804 Thanks for your interest! FYI, we are not planning to promote this project via GSoC and might end up promoting via Outreachy. We will finalize in the next few days.

Okay! I would be interested to work on this even in Outreachy if it gets promoted there and my initial application gets selected. :)

(as per the program rules need to restrict access to this project task to Outreachy Mentors group until the application period begins)

Hi! @daniel I would like to contribute, can you please assign me some task to get started with?

@Akansha99: See https://phabricator.wikimedia.org/maniphest/query/rDGv.ANXQ4ce/#R (if Daniel does not correct me)

@srishakatux: I am wondering if step 9 on https://www.mediawiki.org/wiki/Outreachy/Participants should link to https://www.mediawiki.org/wiki/Good_first_bugs

@Aklapper Yes, that's a good point; I've added the link from the Outreachy/Participants page.

We are unlisting this project from Outreachy (Round 20). If you a potential intern, please explore other projects here https://www.mediawiki.org/wiki/Outreachy/Round_20#Ideas_for_projects.

Change 693134 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] WIP: Symfony Console based CLI for maintenance scripts

https://gerrit.wikimedia.org/r/693134

I'll be doing some more exploration around the patch above during the hackathon, if anyone's interested to code / critique / test, help is welcome :)

I noticed today that one of the first commits in the Maintenance system actually had a class-based registry, and while we lost that at some point, it connected the dots for me to something we still do today: The maintenance scripts are in the autoloader and each script has logic to avoid execution if loaded outside their own entry point. This was I believe designed to be run via a wrapper one day.

From the original commit:

$wgMaintenanceScripts = array();

Register their maintenance scripts [class name] with the system

$wgMaintenanceScripts for extensions to add their scripts to the default list.

public static function getMaintenanceScripts() {
      global $wgMaintenanceScripts;
      return $wgMaintenanceScripts + self::getCoreScripts();
  }

http://mediawiki.org/wiki/Special:Code/MediaWiki/54225
https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/a1c51e18af85a9ac464c5b555921e58ec422cd11%5E%21/

If someone is willing to make small patches doing this step-by-step, I'd be happy to review it.

Last time I tried, I got lost in the way this is doing setup and loading config. Perhaps with SettingsBuilder, it can now be done in a saner way...

@Ladsgroup, @Krinkle, @kostajh: I have an experimantal chain of patches up, starting at https://gerrit.wikimedia.org/r/c/mediawiki/core/+/782569. It still needs some polishing, testing, and bikeshedding about names, but it seems to work. I'd love to hear your thoughts on the approach I'm takeing there, before I invest more time.

It sounds like an awesome hackathon project!

Change 818574 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[mediawiki/core@master] ExtensionRegistry: split exportExtractedData

https://gerrit.wikimedia.org/r/818574

Change 783428 had a related patch set uploaded (by Daniel Kinzler; author: Daniel Kinzler):

[mediawiki/core@master] Introduce runner entry point

https://gerrit.wikimedia.org/r/783428

In T99268#7830522, @Krinkle wrote:

I noticed today that one of the first commits in the Maintenance system actually had a class-based registry, and while we lost that at some point, it connected the dots for me to something we still do today: The maintenance scripts are in the autoloader and each script has logic to avoid execution if loaded outside their own entry point. This was I believe designed to be run via a wrapper one day.

From the original commit:

$wgMaintenanceScripts = array();

Register their maintenance scripts [class name] with the system

$wgMaintenanceScripts for extensions to add their scripts to the default list.

public static function getMaintenanceScripts() {
      global $wgMaintenanceScripts;
      return $wgMaintenanceScripts + self::getCoreScripts();
  }

http://mediawiki.org/wiki/Special:Code/MediaWiki/54225
https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/a1c51e18af85a9ac464c5b555921e58ec422cd11%5E%21/

That was indeed the original idea over a decade ago :p

I think this is mostly done, there is one last push needed to get the last piece over the line.

Change 818574 abandoned by Daniel Kinzler:

[mediawiki/core@master] ExtensionRegistry: split exportExtractedData

Reason:

Going with I6f8f9f3f7252f0024282d7b005671f28a5b3acc3 instead

https://gerrit.wikimedia.org/r/818574

Change 783428 merged by jenkins-bot:

[mediawiki/core@master] Introduce run.php for running maintenance scripts

https://gerrit.wikimedia.org/r/783428

This is great, but the new warning introduced in rMW4912fe27a8e0: Maintenance scripts: show a warning when run directly. shown for old-style scripts is likely to mess up any automation and should be reverted IMO (or maybe it should try to detect whether it is writing to an interactive terminal).
Vagrant for example breaks quite spectacularly: T326522: Vagrant mediawiki::maintenance resource broken by new maintenance script output

In T99268#8507896, @Tgr wrote:

This is great, but the new warning shown for old-style scripts is likely to mess up any automation and should be reverted IMO (or maybe it should try to detect whether it is writing to an interactive terminal).
Vagrant for example breaks quite spectacularly: T326522: Vagrant mediawiki::maintenance resource broken by new maintenance script output

Would https://gerrit.wikimedia.org/r/c/mediawiki/core/+/876395 help?

In T99268#8507901, @PleaseStand wrote:

Would https://gerrit.wikimedia.org/r/c/mediawiki/core/+/876395 help?

Probably not, stderr can also cause issues, sometimes it's taken as a sign that the script failed, sometimes it's piped together with normal output etc.
It might be better than stdout, I'm not sure, but either way warnings should be limited to interactive mode. Or use the deprecation warning system, etc.

Change 876792 had a related patch set uploaded (by Gergő Tisza; author: Gergő Tisza):

[mediawiki/core@master] Do not add random text to maint script output when not in TTY mode

https://gerrit.wikimedia.org/r/876792

Change 876792 merged by jenkins-bot:

[mediawiki/core@master] maintenance: Do not output run.php warning when not in TTY mode

https://gerrit.wikimedia.org/r/876792