Page MenuHomePhabricator

Unable to overwrite services using MediaWikiServices hook
Closed, ResolvedPublic

Description

docs/hooks.txt says

'MediaWikiServices': Called when a global MediaWikiServices instance is
initialized. Extensions may use this to define, replace, or wrap services.
However, the preferred way to define a new service is
the $wgServiceWiringFiles array.
$services: MediaWikiServices

However, if I try to do redefine existing service, it does not work. With some debugging I get this:

New container is constructed as 675773727
Service is defined in 675773727
New container is constructed as 974431285
Service is defined in 974431285
My hook on MediaWikiServices is called on 974431285
Service is redefined on 974431285
ServiceContainer::importWiring is ran on 974431285
Wrong (original) service is returned on 974431285

Looking at the code:

	/**
	 * Imports all wiring defined in $container. Wiring defined in $container
	 * will override any wiring already defined locally. However, already
	 * existing service instances will be preserved.
	 *
	 * @since 1.28
	 *
	 * @param ServiceContainer $container
	 * @param string[] $skip A list of service names to skip during import
	 */
	public function importWiring( ServiceContainer $container, $skip = [] ) {
		$newInstantiators = array_diff_key(
			$container->serviceInstantiators,
			array_flip( $skip )
		);

		error_log( "importWiring is ran on " . $this->id );

		$this->serviceInstantiators = array_merge(
			$this->serviceInstantiators,
			$newInstantiators
		);
	}

For array merge:

If the input arrays have the same string keys, then the later value for that key will overwrite the previous one. If, however, the arrays contain numeric keys, the later value will not overwrite the original value, but will be appended.

Based on the documentation, I think the parameters for array_merge are in wrong order.

Event Timeline

daniel triaged this task as High priority.Dec 14 2016, 11:12 PM
daniel added a project: User-Daniel.

if you are right, that would be bad :) I need to check and think, though. There is a misunderstanding somewhere. Maybe in my head :)

MediaWikiServices::importWiring does what it says in the docs. I think the problem is that the first version of the container is set up in ExtensionRegistry::loadFromQueue before hooks have been exported. MediaWikiServices::resetGlobalServices then replaces it with a new object, and even though the MediaWikiServices hook does get called for that new object, the updated service definition gets overwritten by the old one that is copied over from the early container.

So in general the order of things is:

  • extensions which do not use extension registration add their MediaWikiServices hooks
  • loadFromQueue runs, container gets created, those hooks run
  • extensions which use extension registration add their hooks
  • resetGlobalServices runs, all hooks run but those redefining a service have no effect since old service definitions override new ones.

Also, in theory some extension entry point could get the container singleton and thus trigger hook execution, somewhere halfway in the process of LocalSettings.php executing the non-extension-registration entry points.

I ran into this issue with GlobalPreferences, and am working around it by instantiating the service just after redefinition:

	public static function onMediaWikiServices( MediaWikiServices $services ) {
		$services->redefineService(
			'PreferencesFactory',
			function ( MediaWikiServices $services ) {
				return new GlobalPreferencesFactory();
			}
		);
		// Now instantiate the new Preferences.
		$services->getPreferencesFactory();
	}

Change 430012 had a related patch set uploaded (by MaxSem; owner: MaxSem):
[mediawiki/core@master] Don't initialize MediaWikiServices before extensions have been loaded

https://gerrit.wikimedia.org/r/430012

Change 430012 merged by jenkins-bot:
[mediawiki/core@master] Don't initialize MediaWikiServices before extensions have been loaded

https://gerrit.wikimedia.org/r/430012

Change 565298 had a related patch set uploaded (by TK-999; owner: TK-999):
[mediawiki/core@master] MediaWikiServices: Re-run MediaWikiServices hook when resetting instance

https://gerrit.wikimedia.org/r/565298

Change 565298 had a related patch set uploaded (by TK-999; owner: TK-999):
[mediawiki/core@master] MediaWikiServices: Re-run MediaWikiServices hook when resetting instance

https://gerrit.wikimedia.org/r/565298

I haven't yet considered all the angles, but my initial feeling is that this is hiding a problem and it will likely come at a performance cost.

To what extent can invalid/early instantiation be avoided?

What are those invalid/early instances used for?

Does that work correctly?

If so, are they really invalid? If not invalid, why does it need resetting?

Thanks for the review!

To what extent can invalid/early instantiation be avoided?

My understanding is that early instantiation no longer occurs since rMW69aecc2 on a default MediaWiki installation.

What are those invalid/early instances used for?

Does that work correctly?

The main use case I see for having to initialize the service container in LocalSettings.php is when a dynamic configuration loading system is used to infer the current wiki context and apply configuration overrides as needed, and this system requires services from the container such as WANObjectCache or the MediaWiki database abstraction layer.

If so, are they really invalid? If not invalid, why does it need resetting?

The code in Setup.php indicates that the service container reset is performed to ensure that any subsequent service accesses on the global container instance will take into account new services and service overrides registered by extensions.

// Reset the global service locator, so any services that have already been created will be
// re-created while taking into account any custom settings and extensions.
MediaWikiServices::resetGlobalInstance( new GlobalVarConfig(), 'quick' );

An alternative solution to this problem—that doesn't involve running the MediaWikiServices hook for a second time—could be to adjust the resetGlobalInstance() method such that it does not import the wiring from the old container instance. This was introduced along with resetGlobalInstance() itself in rMWbca436d. Since the new container instance will contain all services and service overrides from configured wiring files and MediaWikiServices hook handlers, it may be possible to eschew the importWiring call, which would resolve this issue without having to run the hook again.

The main use case I see for having to initialize the service container in LocalSettings.php is when a dynamic configuration loading system is used to infer the current wiki context and apply configuration overrides as needed, and this system requires services from the container such as WANObjectCache or the MediaWiki database abstraction layer.

I don't think we should allow use of any services during the setup phase. I am afraid that supporting this would create a mine field where no service can be safely used because you cannot not for certain whether someone somewhere will modify it again later. How would one know whether the "right" version of these services is made available? Or whether code executes in the right order? Or whether there are circular dependencies based you haven't reached an override yet. These would all fail in mysterious and non-obvious ways I think if we allow use of services during the setup phase.

**Good news*:* Using caching or database queries is not a problem. And, exactly because core now uses service wiring for most classes, this also means it is much easier to create your own instance of these. A service generally has all dependencies injected with no shared or leaked state outside of it, which means as long as you are able to pass in all the constructor parameters, there is nothing left to worry about.

In the case of WANObjectCache, or RDBMS, this means if you construct these services yourself, then you can use them. You then have the choice to either dispose of the instance (and let them be re-created later) or to let core re-use them by registering them as an override. Either way is fine.

So in a nut shell: Any classes you want to use during the setup phase must be constructed explicitly, not via the service container. And you can choose to give the objects back to the service container for re-use if you want to.

As an example, the EtcdConfig and APCUBagOStuff classes are constructed explicitly in wmf-config (source).

So in a nut shell: Any classes you want to use during the setup phase must be constructed explicitly, not via the service container. And you can choose to give the objects back to the service container for re-use if you want to.

So, should we simply make it impossible to use the service container before the system has been fully initialized?

Yep. I can't imagine a scenario in which that makes sense to do and isn't unreasonble to support (imho).

Anything that needs services early either is very fragile and likely accidental, or must be required to handle its own objects instead if really needed. E.g. in wmf-config we use EtcdConfig but that is and should be constructed explicitly there as standalone object tree. Not using some kind of global service.

daniel removed daniel as the assignee of this task.Dec 1 2020, 4:24 PM
daniel subscribed.

Change 651539 had a related patch set uploaded (by Daniel Kinzler; owner: Daniel Kinzler):
[mediawiki/core@master] TEST: disallow premature instantiation of services.

https://gerrit.wikimedia.org/r/651539

Yep. I can't imagine a scenario in which that makes sense to do and isn't unreasonble to support (imho).

Anything that needs services early either is very fragile and likely accidental, or must be required to handle its own objects instead if really needed. E.g. in wmf-config we use EtcdConfig but that is and should be constructed explicitly there as standalone object tree. Not using some kind of global service.

So, according to the patch above, we can just go and prohibit "premature" service creation. At least nothing we have in CI fails. I guess we'd want it behind a feature flag at first, so things will not just explode when this is deployed.

Moving to inbox for triage. Can probably go to clinic duty for review and deployment. If deploying this change results in deprecation warnings in production, further follow-up will become necessary. If that follow-up turns out to be complex, the patch may need to be reverted.

Change 651539 merged by jenkins-bot:
[mediawiki/core@master] Deprecate premature instantiation of services.

https://gerrit.wikimedia.org/r/651539

Krinkle assigned this task to daniel.