Page MenuHomePhabricator

Add mechanism to prevent autoloading of Composer installed extensions via LocalSettings.php
Open, MediumPublic

Description

Presently extensions installed via Composer are automatically and imperatively invoked. Currently there is no mechanism to install an extension via Composer but to prevent it from being invoked. This is specifically a problem in a farmed environment with a single codebase for multiple wikis which have different configurations. It will be a nice feature have means to break autoloading via LocalSettings.php


Version: unspecified
Severity: enhancement

Details

Reference
bz59872

Event Timeline

bzimport raised the priority of this task from to Medium.Nov 22 2014, 2:33 AM
bzimport added a project: MediaWiki-General.
bzimport set Reference to bz59872.
bzimport added a subscriber: Unknown Object (MLST).
Kghbln created this task.Jan 9 2014, 6:13 PM
Kghbln awarded a token.Dec 2 2014, 5:17 PM
Kghbln set Security to None.
Kghbln added a subscriber: cicalese.

Sounds like a good idea! By the way, you can see a delightful but inconclusive previous discussion on this topic here: https://www.mediawiki.org/wiki/Talk:Requests_for_comment/Extension_management_with_Composer#A_way_to_disable_extensions_40627

Kghbln added a comment.Dec 2 2014, 6:06 PM

I thought you might like it. Thank you for linking to the discussion on this. Thus we do not have to repeat ourselves here. I'm with you.

Thanks for adding me as a subscriber, Karsten. Yaron, thanks for linking to that discussion. I had seen similar discussions, but none as thorough as that one.This is definitely an issue for our wikifarms as well.

Egel added a comment.Dec 2 2014, 7:51 PM

This is the way we (www.scoutwiki.org) are trying to cope with the composer problem. Composer is run in a separate directory.
We use this script on our farm to stop part of the autoloading:

composer.sh
echo "runnning composer.phar"
php ./composer.phar $*
patch -p0 < noautoload_files.patch
echo "changes in autoloaded files:"
diff -N ./vendor/composer/autoload_files.old ./vendor/composer/autoload_files.php
cp  ./vendor/composer/autoload_files.php ./vendor/composer/autoload_files.old

That uses

noautoload_files.patch
--- vendor/composer/autoload_real.php.orig      2014-01-20 18:32:05.139817007 +$
+++ vendor/composer/autoload_real.php   2014-01-19 21:21:06.650869379 +0100
@@ -45,3 +45,3 @@
        $loader->register(true);
-        $includeFiles = require __DIR__ . '/autoload_files.php';
+        $includeFiles =  array();
       foreach ($includeFiles as $file) {

All changes in /vendor/composer/autoload_files.php are edited to CommonSettings.php, shared by all wikis

CommonSettings.php
// composer/vendor libraries before the composer-installed extensions
if ( $wmgUseComposer ) {
	$ComposerPath = "$IP/../composer";
	require_once( "$ComposerPath/vendor/autoload.php" );
	require_once( "$ComposerPath/vendor/data-values/interfaces/Interfaces.php" );
	require_once( "$ComposerPath/vendor/data-values/data-values/DataValues.php" );
	require_once( "$ComposerPath/vendor/data-values/validators/Validators.php");
	require_once( "$ComposerPath/vendor/data-values/common/Common.php");
	require_once( "$ComposerPath/vendor/data-values/geo/Geo.php");
	require_once( "$ComposerPath/vendor/param-processor/param-processor/DefaultConfig.php");
}
# ----------------------------------------------------------------------
# Extensions

// Validator before a lot of other composer-installed extensions
if ( $wmgUseValidator ) {
	require_once( "$ComposerPath/extensions/Validator/Validator.php" );
}
if ( $wmgUseMaps ) {
	require_once( "$ComposerPath/extensions/Maps/Maps.php" );
}
// SemanticMediaWiki before other semantic extensions
if ( $wmgUseSemanticMediaWiki ) {
	require_once( "$ComposerPath/extensions/SemanticMediaWiki/SemanticMediaWiki.php");
	$smwgNamespaceIndex = 102;
    enableSemantics( $wgServerName );
}
 
// SemanticMaps after SemanticMediaWiki and Maps
if ( $wmgUseSemanticMaps ) {
	require_once( "$ComposerPath/extensions/SemanticMaps/SemanticMaps.php");
}

(this a small part of the file)

Enabling is done in InitialiseSettings.php, shared by all wikis, like:

InitialiseSettings.php
'wmgUseComposer' => array(
	'default' => false,
	'scoutwiki_fr' => true,
	'scoutwiki_it' => true,
	'scoutwiki_nl' => true,
	'scoutwiki_asso' => true,
), 
'wmgUseMaps' => array(
	'default' => false,
	'scoutwiki_fr' => true,
	'scoutwiki_it' => true,
	'scoutwiki_nl' => true,
	'scoutwiki_asso' => true,
), 
'wmgUseSemanticMaps' => array(
	'default' => false,
	'scoutwiki_fr' => true,
	'scoutwiki_nl' => true,
), 
'wmgUseSemanticMediaWiki' => array(
	'default' => false,
	'scoutwiki_fr' => true,
	'scoutwiki_nl' => true,
),

It seems the goal is not to disable autoloading of classes in the extension, but to disable the (running of the) extension. If that is true, then the solution is to not register/load the extension while loading the autoloader. But to make that separate steps. That is how https://www.mediawiki.org/wiki/Manual:Extension_registration is supposed to works. I.e. the extensions that load/register themselves when the autoloader is loaded need to be changed.

The simple way to fix this is for the extensions using composer.json to stop including the configuration that forces autoloading of the entry point. Since the RfC on this topic (T467: RfC: Extension management with Composer) was abandoned recently due to the WONTFIX of this problem upstream it seems the most reasonable course of action.

The LocalSettings related fix would be to add a global such as $wgEnableComposerExtensionLoading = false; and get all of the extension authors to wrap their autoloaded entry points in a check for it.

But if T61872#1361866 is done then instead of T61872#1361869 one can just add something to LocalSettings to load all extensions in the extensions directory to get the effect of having extensions automatically enabled once they are installed. (For people that want that. And people that don't can still disable it.)

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 5 2015, 10:44 AM

I am with @JanZerebecki and @bd808: To me it looks like disabling some extensions selectively is attacking the problem at the wrong end. I think that an extension should not be loaded (in the sense of wfLoadExtension) when it is installed with composer. In particular, this means that composer.json should not autoload the file ExtensionName.php. Rather should composer only be used to merely install the files, then the extension would be activated in LocalSettings.php using wfLoadExtension('ExtensionName') or require_once "$IP/extensions/ExtensionName/ExtensionName.php".

If this was the commonly agreed upon way to go, I think two things would need to be done: (1) update documentation to explain that extensions should not load themselves, (2) find extensions that do autoloading and change them.

I just created a quick, hacky report so find extensions that definitely activate themselves when they are installed via composer:

git clone --recursive https://gerrit.wikimedia.org/r/p/mediawiki/extensions.git
cd extensions
grep autoload */composer.json -A3 | grep '"files"' -A1 | grep -oE '[^"]*\.php' | while read line; do grep -E "wgExtensionCredits|wfLoadExtension" */$line -l; done | cut -f1 -d/

This is the result:

  • AdminLinks
  • Bootstrap
  • DataTransfer
  • GraphViz
  • ImageMap
  • Lingo
  • LocalisationUpdate
  • Maps
  • MixedNamespaceSearchSuggestions
  • NumerAlpha
  • ParserHooks
  • PubSubHubbub
  • SemanticBreadcrumbLinks
  • SemanticCite
  • SemanticExtraSpecialProperties
  • SemanticForms
  • SemanticGlossary
  • SemanticHighcharts
  • SemanticInterlanguageLinks
  • SemanticMaps
  • SemanticMetaTags
  • SemanticResultFormats
  • SemanticSifter
  • SemanticSignup
  • SemanticWatchlist
  • SideBarMenu
  • SubPageList
  • UserFunctions
  • Validator
  • WebPlatformAuth
saper added a subscriber: saper.EditedDec 16 2015, 10:09 AM

We also have this issue with skins (is this another bug?)

Skins installed via composer that register themselves via PHP are loaded automagically (nice, no need for ugly require_only("$IP...).

I have this in my composer.local.json:

{
  "repositories": [
    {
      "type": "vcs",
      "url": "git@github.com:saper/mediawiki-skins-Vector.git"
    },
    {
      "type": "vcs",
      "url": "git@github.com:saper/mediawiki-skins-Slate.git"
    },
    {
      "type": "vcs",
      "url": "git@github.com:saper/mediawiki-skins-Metrolook.git"
    }
  ],
  "require": {
    "mediawiki/vector-skin": "dev-installer-name",
    "mediawiki/slate-skin": "dev-installer-name",
    "mediawiki/metrolook-skin": "dev-installer-name",
    "mediawiki/chameleon-skin": "*"
  },
  "minimum-stabilty": "dev"
}

(One needs my local git repos to avoid T117856).

So I need to

wfLoadSkin( 'Metrolook' );
wfLoadSkin( 'Timeless' );
wfLoadSkin( 'Vector' );

but chameleon (has no skin.json entry as of now) loads just fine and shows up in the preferences.

I actually like the PHP autoload here...

@saper: the problem with autoloading are use cases where the code of many extensions/skins should be deployed, but only part of them should actually be used. See the "wikifarm" use case here and here. If extensions/skins do not autoload (as in calling wfLoadExtension themselves), then one can choose in his/her LocalSettings.php, which ones one wants (using require_once or wfLoadExtension as appropriate for the extension in question).

Is this task still relevant, now that we have extension registration? Any extensions that are still actively doing things in an autoloaded file should have tasks opened against them, and be fixed.

Yes, this is still an issue for Semantic MediaWiki and a handful of related Semantic... extensions. They are not explicitly loaded with wfLoadExtension but rather use composer-generated autoloading. My understanding is that once MediaWiki complies with (not sure that is the correct terminology) PSR-4, that community will be willing/able to adapt to extension registration.

@CCicalese_WMF But surely they're doing it wrong then? (Sorry, that's a bit harsh isn't it! SMW is certainly doing a lot of things right!) But nothing needs to change on the MediaWiki side; there's no need to introduce any new method of getting around autoloaded files carrying out actions, those files just need to be changed and every extension loaded with wfLoadExtension().

Actually, yes, you are correct that the fix proposed by this task should not be necessary. I realize now that I did not answer the question that you asked. I answered the more general question whether extensions using composer autoloading and bypassing extension registration was still a problem. I do believe that the extensions that use another autoloading mechanism and do not support extension registration should be modified to use extension registration, rendering this task unnecessary. My understanding is that the roadblock to this had to do with support for PSR-4 autoloading in MediaWiki. I'm not sure whether completion of T173799 would be sufficient to satisfy this.

mwjames added a subscriber: mwjames.EditedOct 13 2017, 12:26 AM

My understanding is that once MediaWiki complies with (not sure that is the correct terminology) PSR-4, that community will be willing/able to adapt to extension registration.

Well, I have outlined necessary steps [0] required before the project is willing to take the risk converting SMW to use extension.json as a possible registration mechanism and even then, Composer remains the method of choice for class registration and autloading that complies with PSR-4 and does not introduce some MediaWiki specific dependency during the start that have wreck the software several times in the past.

Since T152263 hasn't been merged nor does it seem to be done in near future chances are becoming very slim that extension.json is going to be supported with the upcoming 3.0 release.

The project (and hereby related extensions) will only make a conversion to extension.json on a major release which implies if it doesn't happen within the set framework for 3.0 it can only happen with 4.0.

As I said before, I'm not thrilled about extension.json as a registration system but I'm willing to implement necessary changes once the requirements are solved and workable.

There is an outstanding issue (see [0]) of pre-defined constants.

[0] https://github.com/SemanticMediaWiki/SemanticMediaWiki/pull/1732

I have been working on an LDAP suite in the past week or so with a couple of other people and, since @Osnard started work on LDAPProvider with extension.json + composer.json, I have found that there is a way to load extensions with composer.json but not load them without wfLoadExtension().

It even looks like the composer autoloading mechanism would be sufficient for dependencies. The bit that is left, as far as I can see, for T152263 is that hooks, etc (e.g. wfLoadExtension) would have to be registered.