Page MenuHomePhabricator

Extension tables missing after install.php
Closed, ResolvedPublicBUG REPORT

Description

Steps to replicate the issue (include links if applicable):
Brief summary:

  • Clone MediaWiki core and OAuth
  • Install MediaWiki with --with-extensions
  • Fetch Special:OAuthListConsumers

Script:

cd /tmp
git clone --depth=1 https://gerrit.wikimedia.org/r/mediawiki/core.git
git -C core/extensions/ clone --depth=1 https://gerrit.wikimedia.org/r/mediawiki/extensions/OAuth.git
ln -s composer.local.json-sample core/composer.local.json
composer -d core/ install
php core/maintenance/run.php install --dbtype=sqlite --with-extensions --with-developmentsettings --pass=testpass0123456789 'Test Wiki' 'Test Wiki Admin'
php -S localhost:12345 & sleep 1; curl -v http://localhost:12345/core/index.php/Special:OAuthListConsumers
kill %1

What happens?:

Error 1: no such table: oauth_registered_consumer

What should have happened instead?:
The table should exist, there shouldn’t be an error.

Workaround:
Run update.php (or rather php maintenance/run.php update yadda yadda), which will create the tables.

Software version (on Special:Version page; skip for WMF-hosted wikis like Wikipedia):
MediaWiki core 8ce0a47689, OAuth d25e72b64f.

Other information (browser name/version, screenshots, etc.):
I’ve pretty much only tested the exact steps in the script so far (I first noticed it in setup-mediawiki + m3api-oauth2 but that uses basically the same steps). Someone™ should check whether this also affects other extensions, database types, and/or wikis without development settings. (I might do that later but not tonight :P)

  • It also happens without --with-developmentsettings (but then you don’t see the exact error message in the web output, of course)
  • It also happens for Wikibase (clone with --recursive, curl Special:ListProperties), so it’s probably not specific to any one extension
  • …

Event Timeline

Looking through the installer code, I don’t see anything that looks like it’s supposed to load extensions during the installation process – they just end up in the _Extensions installer var, whence LocalSettingsGenerator.php eventually puts them into LocalSettings.php (as either wfLoadExtension() or require_once lines). But the database initialization probably still runs inside a PHP process that has never loaded the extensions, so their LoadExtensionSchemaUpdates hook handlers never get run.

It’s possible that “--with-extensions requires subsequent update.php” has always been the behavior of the installer (rather than a regression), possibly even the intended behavior. But I haven’t found any documentation for this yet, neither in the option’s own documentation in the maintenance script nor in Manual:install.php (permalink).

CC @hashar who added --with-extensions to install.php in 2018.

OAuth's LoadExtensionSchemaUpdates has

if ( !Utils::isCentralWiki() ) {
	// no tables to add
	return true;
}
...
public static function isCentralWiki() {
	global $wgMWOAuthCentralWiki;
	return ( WikiMap::getCurrentWikiId() === $wgMWOAuthCentralWiki );
}

When the installer runs this hook, extension config has not been exported to global variables, so $wgMWOAuthCentralWiki is unset. Also, the default is false, it depends on a SetupAfterCache hook to write to $wgMWOAuthCentralWiki so that isCentralWiki() will return true.

So I don't think I broke it. It probably never worked.

Migrating to virtual domains should fix it (T348485). The installer tries to figure out whether a virtual domain is local before running an update, so as long as you integrate properly with that, the Utils::isCentralWiki() check should be unnecessary.

Change #1198101 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/extensions/OAuth@master] Update installer/updater to use virtual domains

https://gerrit.wikimedia.org/r/1198101

Change #1198101 merged by jenkins-bot:

[mediawiki/extensions/OAuth@master] Update installer/updater to use virtual domains

https://gerrit.wikimedia.org/r/1198101

• matmarex claimed this task.
• matmarex subscribed.

Verified that this is fixed using the commands from the task description to install a new wiki and test OAuth on it (thanks for the easy reproduction steps).

Well, that sounds like it only fixes it for the OAuth extension? What about other extensions?

If there are other extensions which have been converted to virtual domains but not to use addExtensionUpdateOnVirtualDomain(), probably better to handle them in dedicated tasks or in the relevant T348573: All Wikimedia extensions that store their data outside the main database should use a virtual database domain subtasks.

If the problem is something else, can you be more specific about it?

Well, as someone maintaining CI templates to install / set up MediaWiki (GitHub action, GitLab CI), it doesn’t sound like I can remove the update.php call yet if I want my templates to be useful for arbitrary extensions, because I don’t know if the extensions other people would like to use in their CI have been converted to virtual domains or not. (Though this is to some extent academic – I’ve advertised the CI templates a bit, but I don’t know if anyone else actually uses them.)

@LucasWerkmeister The problem you encountered is (afaik) not about extension schemas in general.

In general, extension schemas work fine in install.php and don't relate to or require use of virtrual domains.

A handful of "special" extensions that involve an external database (CentralAuth, OAuth, and GlobalUsage come to mind) have historically defined novel configuration variable for LocalSettings. Some excluded their schemas from LoadExtensionSchemaUpdates to avoid mistakes, others would blindly provision them locally but only use it when configured. This was usually because such configuration variable would be passed to LBFactory->getExternalLB which did not accept false or local/foreign wiki databases as value, so it required more code at callsites specifically for that use case and people tended not to in my experience. There hasn't been much consistency in how or whether that was done. If the chosen database is also wiki, you could run update.php on that side. This wouldn't work for setups where you use a separate database (akin to "centralauth" or "x1") but should work for things like GlobalUsage where the table tends to be a local table on another wiki.

I'm surprised OAuth worked, although as you observe, only in update.php, not install.php. Not sure that was entirely on purpose :)

With the introduction of virtual domains (T330590) such special extension can "just work" by removing the novel config in favor of a virtual domain. These work out of the box in MediaWiki, defaulting to the local wiki. Through LocalSettings, one can optionally point these anywhere (foreign wiki or separate DB) and runtime queries and schema updates both go there.

Such special extensions don't have a long tail, and they tend to require custom configuration (at least until they adopt virtual domains). My guess is that there are no arbitrary extensions where this "just works" with and not without update.php. You may have found OAuth as the only case not yet converted. Looking at T330590, there might be one other (ReadingLists: T349157).

Do you have a specific example in mind?

I see, thanks for explaining! Then I guess I’ll just remove the update.php call and see if anything breaks or anyone complains :)

Ninja edit: or rather, given that the CI templates ostensibly support older MediaWiki versions in there, I guess I’ll leave it in and accept the slight performance cost, at least until some future time where I decide to drop support for out-of-support versions. But given that explanation I’m now content with this task being resolved.

I'm surprised OAuth worked, although as you observe, only in update.php, not install.php.

Probably because update.php runs hooks while install.php doesn't, so dynamic configuration can be made to work with the former but not with the latter.