Page MenuHomePhabricator

MediaWiki web installer does not show extension when their dependency is missing
Open, MediumPublic

Description

Affects MW 1.32+

Summary

Given a tree with mediawiki/core and an extension having some requirements defined in extension.json (for example: WikibaseMediaInfo).

When running the Web installer, at the ?page=Options step, the extension should be shown with a disabled checkbox and hints at its dependencies. Instead the Extensions section is not even shown.

When running CLI installer with --with-extensions, the extension is skipped entirely and not added to LocalSettings.php. Which is to be expected since it is not functional due to the lack of dependency. However the installer keep processing just fine.

That has hit us on CI for which WikibaseMediaInfo lacked a dependency upon WikibaseCirrusSearch. Hence its PHPUnit tests were never run since the extension was not loaded. That has been solved by injecting the required dependency.

Expected

The WebInstaller should shows any extension that got cloned. Even if their registration is not valid.

install.php should fail when a requirement is missing instead of ignoring the extension and a status message shown to the user.

The root cause is Installer::findExtensionsByType() which just skip them. A summary would be:

$status = $this->getExtensionInfo( $type, $directory, $file );
if ( $status->isOK() ) {
    $exts[$file] = $status->value;
}
return $exts;

Which gets its informations from Installer::getExtensionInfo:

if ( $isJson ) {
    $jsonStatus = $this->readExtension( $fullJsonFile );
    if ( !$jsonStatus->isOK() ) {
        return $jsonStatus;
    }
    $info += $jsonStatus->value;
}

Since $jsonStatus is not ok, the extension is not added :-(

Reproduction

mkdir workspace2
quibble --db=sqlite \
  --workspace=/home/hashar/workspace2 --git-cache=/home/hashar/projects \
  --skip=all --skip-deps \
  mediawiki/extensions/WikibaseMediaInfo
$ grep wfLoadExtension workspace2/src/LocalSettings.php
$ 

Details

@Cparle had a nice issue today, the WikibaseMediaInfo extension was not showing in Special:Version after installing MediaWiki with install.php --with-extensions.

A way I reproduced it to run the installer with solely WikibaseMediaInfo cloned:

mkdir workspace2
quibble --db=sqlite \
  --workspace=/home/hashar/workspace2 --git-cache=/home/hashar/projects \
  --skip=all --skip-deps \
  mediawiki/extensions/WikibaseMediaInfo

The install works fine. Looking at the generated LocalSettings.php:

workspace2/src/LocalSettings.php
# Enabled skins.
# The following skins were automatically enabled:
wfLoadSkin( 'Vector' );


# End of automatically generated settings.
# Add more configuration options below.

Or in short, the extension has not been injected in the config which should have wfLoadExtension( 'WikibaseMediaInfo' );.

CI / Quibble does clone some repositories then runs install.php --with-extensions. I would eventually expect it to bail out immediately.

For the debugging session I have added a few var_dump in the installer with https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/502488/ then created a change depending on it https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseMediaInfo/+/502489/ . The console log is at https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-hhvm-docker/44086/consoleText

Installer::getExtensionInfo( type = extension, parentRelPath = extensions, name = WikibaseMediaInfo )
Installer::getExtensionInfoconfig comes from json


Notice: Array to string conversion in /workspace/src/includes/installer/Installer.php on line 1312
object(Status)#778 (7) {
  ["cleanCallback"]=>
  bool(false)
  ["ok":protected]=>
  bool(false)
  ["errors":protected]=>
  array(1) {
    [0]=>
    array(3) {
      ["type"]=>
      string(5) "error"
      ["message"]=>
      string(26) "config-extension-not-found"
      ["params"]=>
      array(1) {
        [0]=>
        string(20) "WikibaseCirrusSearch"
      }
    }
  }
  ["value"]=>
  NULL
  ["success"]=>
  array(0) {
  }
  ["successCount"]=>
  int(0)
  ["failCount"]=>
  int(0)
}
string(24) "Json status is not OK :("
string(59) "Installer::findExtensionsByType(): getExtensionInfo got us:"
object(Status)#778 (7) {
  ["cleanCallback"]=>
  bool(false)
  ["ok":protected]=>
  bool(false)
  ["errors":protected]=>
  array(1) {
    [0]=>
    array(3) {
      ["type"]=>
      string(5) "error"
      ["message"]=>
      string(26) "config-extension-not-found"
      ["params"]=>
      array(1) {
        [0]=>
        string(20) "WikibaseCirrusSearch"
      }
    }
  }
  ["value"]=>
  NULL
  ["success"]=>
  array(0) {
  }
  ["successCount"]=>
  int(0)
  ["failCount"]=>
  int(0)
}

The whole logic is in Installer::readExtension

  • WikibaseMediaInfo json is passed to the extension with a require for WikibaseCirrusSearch
  • The ExtensionRegistry is used to load the file. That raises an ExtensionDependencyError due to the missing requires
  • readExtension() is called again passing the missing extension a a hint (that causes the second var dump above?)
  • the extension is not present, thus readExtension returns Status::newFatal( 'config-extension-not-found' );
  • getExtensionInfo() returns the Status

In Installer::findExtensionsByType(), the extension is only added when getExtensionInfo() returned an OK status.

Event Timeline

hashar renamed this task from MediaWiki installer ignore extension when a requires is missing instead of failling to MediaWiki web installer do not show extension when their dependency is missing.Apr 9 2019, 9:03 PM
hashar updated the task description. (Show Details)

The extensions being hidden/skipped is due to https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/461280/ install.php: Allow extensions and skins to be specified which is in MW-1.32-release.

Note if I revert it, the WikibaseMediaInfo extension is shown but its missing dependencies are not shown:

install-wikibasemediainfo.png (183×447 px, 9 KB)

When I add WikibaseCirrusSearch and

install-wikibasemediainfo_and_requirements.png (235×1 px, 23 KB)

For install.php:

Including extensions

ExtensionDependencyError from line 293 of includes/registration/ExtensionRegistry.php: WikibaseMediaInfo requires UniversalLanguageSelector to be installed.
WikibaseMediaInfo requires WikibaseCirrusSearch to be installed.

Backtrace:
#0 includes/installer/Installer.php(1453): ExtensionRegistry->readFromQueue(array)
#1 includes/installer/Installer.php(1548): Installer->includeExtensions(SqliteInstaller)
#2 includes/installer/CliInstaller.php(140): Installer->performInstallation(array, array)
#3 maintenance/install.php(125): CliInstaller->execute()
#4 maintenance/doMaintenance.php(96): CommandLineInstaller->execute()
#5 maintenance/install.php(180): require_once(string)
#6 {main}

At first I guess we want to restore the behavior of not filtering extensions that are actually on disk. Then enhance it iso that:

  • the WebInstaller hints about the missing dependencies
  • the CLI installer bails out and shows some status messages for each missing dependencies

Change 502624 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/core@master] Revert "install.php: Allow extensions and skins to be specified"

https://gerrit.wikimedia.org/r/502624

I'm thinking about how to fix this. In the meantime, for CI, if you know the extension list, you could pass it to the --extensions option of install.php. That way you would get an error even if the extension directory is missing, which sounds like it would be a useful sanity check.

$ php7.2 maintenance/install.php --extensions=WikibaseMediaInfo --pass=... installtest Tim
Could not find the registration file for the extension "UniversalLanguageSelector"
$ echo $?
1
$ php7.2 maintenance/install.php --extensions=foo --pass=.. installtest Tim
Could not find the registration file for the extension "foo"
$ echo $?
1

Change 502624 abandoned by Hashar:
Revert "install.php: Allow extensions and skins to be specified"

Reason:
This was just for demo purposes for T220514

https://gerrit.wikimedia.org/r/502624

It took me a while but eventually I found the root cause which is in includes/installer/Installer.php

Installer::findExtensions() did a few validations but once it has found either an extension.json or a .php entry point, it would add the file to the list of found extensions:

if ( $isJson || $isPhp ) {
    // Extension exists. Now see if there are screenshots
    $exts[$file] = [];
    // then code to find screenshots

Thus, even if the extension registry consider it valid, findExtensions() would still yield the faulty extension to the list.

In your refactoring, the $exts[$file] = []; has been removed since it acts on a single extension and the logic has not been added to the new findExtensionsByType():

$exts = [];
 while ( ( $file = readdir( $extDir ) ) !== false ) {
     $status = $this->getExtensionInfo( $type, $directory, $file );
     if ( $status->isOK() ) {
         $exts[$file] = $status->value;
     }
    // Non OK extensions are thus silently ignored
 }

Maybe the non OK extensions should be returned and delegate up to the caller to sort it out? Though for install.php, we only look for the extension name, then pass them again to the extension registry so I guess the extension.json files are processed twice :-/

Another use case is for the WebInstallerOptions, which currently no more show the faulty extension (since it is not returned by findExtensions()) but it has logic to display dependencies at least :)

In your patch, that code has been moved to Installer::getExtensionInfo().


As for CI, if I switch to --skins and --extensions, I would need your patch to be backported on all supported branches. Though REL1_27 is soon EOL, we still run tests for it :-\ I am more in a favor of enhancing findExtensions() to keep invalid extensions and have the callers show the errors.

(or alternatively, move CliInstaller::validateExtensions() up to Installer. Its code is very similar to Installer::findExtensionsByType() though the first does keep the bad status while the second ignore them :)

Change 504427 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/core@master] Restore findExtensions not showing invalid extension

https://gerrit.wikimedia.org/r/504427

This bug does affect MediaWiki 1.32 and is a regression. install.php --with-extensions no more fail and silently ignore extensions that do not have fulfilled requirements

I forgot to loop Platform Engineering . This task is currently causing issues on CI since extensions might not be installed at all :-\

hashar triaged this task as High priority.Jun 5 2019, 7:03 AM
WDoranWMF lowered the priority of this task from High to Medium.Jun 11 2019, 1:03 PM
WDoranWMF subscribed.

@hashar We're going to split the extension regression issue into a separate task which we will mark as high priority.

@hashar We're going to split the extension regression issue into a separate task which we will mark as high priority.

Though both use cases are broken by the same code. But I will update the other task.

This missed the boat for the MW 1.33 release. Provisionally re-tagging to 1.34's release as well.

Change 504427 abandoned by Hashar:
Restore findExtensions not showing invalid extension

Reason:
in favor of https://gerrit.wikimedia.org/r/#/c/mediawiki/core/ /533239/

https://gerrit.wikimedia.org/r/504427

daniel subscribed.

Do we really want the installer to silently skip extensions that have missing dependencies? Shouldn't it rather show a warning?

eprodromou subscribed.

We're going to deal with this in clinic duty this week. @CCicalese_WMF to make the call on proper behaviour.

daniel renamed this task from MediaWiki web installer do not show extension when their dependency is missing to MediaWiki web installer does not show extension when their dependency is missing.Oct 1 2019, 2:37 PM

We discussed this in Clinic Duty. The related task T225512 has been resolved. This task, while good to have, is not as high a priority at this point so isn't going to be worked on by CPT in the near term. I'm putting it in the CPT Icebox.