Page MenuHomePhabricator

Scap should not rely on extension-list, instead pass --extension-dir to mergeMessageFileList.php
Open, NormalPublic

Description

@hashar wrote:

I am pretty sure the only use case is maintenance/mergeMessageFileList.php which apparently already has some kind of detection mechanism built-in:

$possibilities = [
    "$extdir/$extname/extension.json",
    "$extdir/$extname/skin.json",
    "$extdir/$extname/$extname.php"
];
$found = false;
foreach ( $possibilities as $extfile ) {
    if ( file_exists( $extfile ) ) {

That is enabled by passing it --extensions-dir and it "nicely" errors out whenever an extension lack one. The scap task still rely on the extension-list file

Event Timeline

mmodell created this task.Feb 3 2016, 5:37 PM
mmodell claimed this task.
mmodell raised the priority of this task from to High.
mmodell updated the task description. (Show Details)
mmodell added a subscriber: mmodell.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 3 2016, 5:37 PM
Krenair added a subscriber: Krenair.Feb 3 2016, 6:19 PM

We should just get rid of extension-list...

@Legoktm: I agree, it seems unnecessary? But update-l10n breaks without it, currently... I'll dig deeper into it.

It looks like mergeMessageFileList.php can scan the extensions directory, as an alternative to passing an extension-list file, however, we need to scan extensions/* and skins/*

Should I make a way that we can pass multiple directories to the script or perhaps an option to recursively search for everything starting from the root of the mediawiki/core branch checkout?

Anyone have an opinion?

mmodell added a comment.EditedFeb 4 2016, 1:20 AM

Add support for passing multiple paths to extension-dir arg in mergeMessageFileList.php

https://gerrit.wikimedia.org/r/#/c/268337/

Change 268337 had a related patch set uploaded (by Legoktm):
Support multiple extension-dir paths to be passed to mergeMessageFileList

https://gerrit.wikimedia.org/r/268337

Final step is generating the extension list when the branch is cut.

mmodell lowered the priority of this task from High to Normal.Mar 29 2016, 10:07 PM
hashar added a subscriber: hashar.Jul 1 2016, 8:34 AM

From code review of https://gerrit.wikimedia.org/r/#/c/287565/ @hashar wrote:

Do we still need the extension-list anyway? IIRC it is only used for merging l10n messages and the logic could be added to that maintenance script instead.

For CI purposes, we have moved most if not all extensions to use ExtName/ExtName.php as the entry point, and some are migrated to extension.json / skin.json. So the lookup should be quite trivial and we could get rid of extension-list entirely.

@mmodell replied:
It would be great if we could remove the need for this, but somewhere it has to be either generated or computed on demand. My main goal was to remove the chicken-or-egg problem when adding or removing an extension during branch cutting.

I am pretty sure the only use case is maintenance/mergeMessageFileList.php which apparently already has some kind of detection mechanism built-in:
...

$possibilities = [
    "$extdir/$extname/extension.json",
    "$extdir/$extname/skin.json",
    "$extdir/$extname/$extname.php"
];
$found = false;
foreach ( $possibilities as $extfile ) {
    if ( file_exists( $extfile ) ) {

...
That is enabled by passing it --extensions-dir and it "nicely" errors out whenever an extension lack one. The scap task still rely on the extension-list file ... Worth giving that a try?


D288 would move scap to use --extensions-dir and we would no more need the extension-list file.

NOTE: --extensions-dir has never been used AFAIK. So we will want to properly test it, maybe by switching beta to it first.
hashar moved this task from To Triage to In-progress on the Deployments board.

@hashar is right, this isn't necessary if mergeMessageFileList.php can handle things without the extension list.

mmodell renamed this task from extension-list should live in the mediawiki branch rather than mediawiki-config to SCAP should not rely on extension-list, instead pass --extension-dir to mergeMessageFileList.php.Jul 18 2016, 7:08 PM
mmodell updated the task description. (Show Details)
mmodell set Security to None.
mmodell updated the task description. (Show Details)

Below is a draft comment I wrote at some point. committing it for the record

@mmodell replied:
It would be great if we could remove the need for this, but somewhere it has to be either generated or computed on demand. My main goal was to remove the chicken-or-egg problem when adding or removing an extension during branch cutting.

I am pretty sure the only use case is maintenance/mergeMessageFileList.php which apparently already has some kind of detection mechanism built-in:
...

$possibilities = [
    "$extdir/$extname/extension.json",
    "$extdir/$extname/skin.json",
    "$extdir/$extname/$extname.php"
];
$found = false;
foreach ( $possibilities as $extfile ) {
    if ( file_exists( $extfile ) ) {

...
That is enabled by passing it --extensions-dir and it "nicely" errors out whenever an extension lack one. The scap task still rely on the extension-list file ... Worth giving that a try?


D288 would move scap to use --extensions-dir and we would no more need the extension-list file.

NOTE: --extensions-dir has never been used AFAIK. So we will want to properly test it, maybe by switching beta to it first.

The TLDR is that I wrote the --extensions-dir feature years ago but it never got used afaik.

demon added a comment.Jun 12 2017, 5:00 PM

I'm not afraid of using it, I'm mostly concerned of how we'll handle beta. We currently have the extension meta repo symlink'd for easy installation. Sadly, this would make l10n regeneration impossibly slow if we only used --extensions-dir

Reedy updated the task description. (Show Details)Jun 12 2017, 5:09 PM

beta uses a clone of mediawiki/extensions but surely we could instead use the list of repos for which we cut a branch?

demon added a comment.Jun 12 2017, 9:50 PM

Possibly. But we also deploy some extensions to beta that we don't branch for prod yet. I'd rather just stop using that meta repo entirely--can we think up another solution?

Could we add a "beta-only" section in make-wmf-branch's config.json?

Could we add a "beta-only" section in make-wmf-branch's config.json?

Maybeeeee..... but beta doesn't rely on make-wmf-branch so it'd be a weird dependency. Really all this crud should move to wmf-config as scap plugins.

mmodell removed mmodell as the assignee of this task.Jul 5 2017, 4:39 PM

Some parts of the problem: T125678

Krinkle renamed this task from SCAP should not rely on extension-list, instead pass --extension-dir to mergeMessageFileList.php to Scap should not rely on extension-list, instead pass --extension-dir to mergeMessageFileList.php.May 31 2019, 8:58 PM