Page MenuHomePhabricator

Lookup obtained via SingleEntitySourceServices::getPrefetchingTermLookup must obey EntitySource
Open, HighPublic

Description

When the new normalized term store is active for properties, the PrefetchingTermLookup obtained via SingleEntitySourceServices::getPrefetchingTermLookup has to use the current EntitySource (and can't just default to assuming the current wiki).

This will cause SingleEntitySourceServices::getPrefetchingTermLookup to break in federation scenarios, where the PrefetchingTermLookup needs to access other wikis' databases (like the commons federation).

Event Timeline

hoo created this task.Jun 18 2019, 11:13 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 18 2019, 11:13 AM

This is what we do already:

public function getPrefetchingTermLookup() {
    if ( $this->prefetchingTermLookup === null ) {
        $this->prefetchingTermLookup = new BufferingTermLookup(
            $this->getTermIndex(),
            1000 // TODO: customize buffer sizes
        );

        if ( $this->settings->useNormalizedPropertyTerms() ) {
            $mediaWikiServices = MediaWikiServices::getInstance();
            $logger = LoggerFactory::getInstance( 'Wikibase' );

            $repoDbDomain = $this->entitySource->getDatabaseName();
            $loadBalancerFactory = $mediaWikiServices->getDBLoadBalancerFactory();
            $loadBalancer = $loadBalancerFactory->getMainLB( $repoDbDomain );

            $propertyTermLookup = new PrefetchingPropertyTermLookup(
                $loadBalancer,
                new DatabaseTermIdsResolver(
                    new DatabaseTypeIdsStore(
                        $loadBalancer,
                        $mediaWikiServices->getMainWANObjectCache(),
                        $repoDbDomain,
                        $logger
                    ),
                    $loadBalancer,
                    $logger
                )
            );
            $this->prefetchingTermLookup = new ByTypeDispatchingPrefetchingTermLookup(
                [ 'property' => $propertyTermLookup ],
                $this->prefetchingTermLookup
            );
        }
    }
    return $this->prefetchingTermLookup;
}

we are using the same entitySource we pass to old store to create a load balancer here, should that be enough @hoo ?

I'll try to setup local separate instance of MediaInfo and test our changes with federation (both types/versions of it) first.. and see if the code isn't enough for it

alaa_wmde triaged this task as High priority.Wed, Jul 3, 9:19 AM

High cause it blocks migration of terms

hoo added a comment.Wed, Jul 3, 1:23 PM

I think we also need to pass domain in the call to ILoadBalancer::getConnection in PrefetchingPropertyTermLookup::getDbr (remember, when a specific domain is requested, you also need to mark the connection with LoadBalancer::reuseConnection later on).

I think we also need to pass domain in the call to ILoadBalancer::getConnection in PrefetchingPropertyTermLookup::getDbr (remember, when a specific domain is requested, you also need to mark the connection with LoadBalancer::reuseConnection later on).

@hoo what are domains actually?

I still cannot reproduce the issue locally as setting up a second instance of mediawiki with mediainfo extension configured with federation is taking too long.

hoo added a comment.Wed, Jul 3, 4:26 PM

I think we also need to pass domain in the call to ILoadBalancer::getConnection in PrefetchingPropertyTermLookup::getDbr (remember, when a specific domain is requested, you also need to mark the connection with LoadBalancer::reuseConnection later on).

@hoo what are domains actually?

Databases for what we care.

Change 522231 had a related patch set uploaded (by Alaa Sarhan; owner: Alaa Sarhan):
[mediawiki/extensions/Wikibase@master] Add getPrefetchingTermLookup to WikibaseServices

https://gerrit.wikimedia.org/r/522231

Change 522232 had a related patch set uploaded (by Alaa Sarhan; owner: Alaa Sarhan):
[mediawiki/extensions/Wikibase@master] Pass db domain down to DatabaseTermIdsResolver for dbr construction.

https://gerrit.wikimedia.org/r/522232

Change 522231 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add getPrefetchingTermLookup to WikibaseServices

https://gerrit.wikimedia.org/r/522231

Change 522232 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Pass db domain down to DatabaseTermIdsResolver for dbr construction.

https://gerrit.wikimedia.org/r/522232

I still get error in beta cluster:

A database query error has occurred. Did you forget to run your application's database schema updater after upgrading? 
Query: SELECT  wbpt_property_id,wbpt_term_in_lang_id  FROM `wbt_property_terms`    WHERE wbpt_property_id = '694'  
Function: Wikibase\Lib\Store\Sql\Terms\PrefetchingPropertyTermLookup::prefetchTerms
Error: 1146 Table 'enwiki.wbt_property_terms' doesn't exist (172.16.4.147:3306)
 
#0 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(1573): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#1 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(1162): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#2 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(1801): Wikimedia\Rdbms\Database->query(string, string)
#3 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/Terms/PrefetchingPropertyTermLookup.php(79): Wikimedia\Rdbms\Database->select(string, array, array, string)
#4 /srv/mediawiki/php-master/extensions/Wikibase/data-access/src/ByTypeDispatchingPrefetchingTermLookup.php(53): Wikibase\Lib\Store\Sql\Terms\PrefetchingPropertyTermLookup->prefetchTerms(array, array, array)
#5 /srv/mediawiki/php-master/extensions/Wikibase/data-access/src/ByTypeDispatchingPrefetchingTermLookup.php(53): Wikibase\DataAccess\ByTypeDispatchingPrefetchingTermLookup->prefetchTerms(array, array, array)
#6 /srv/mediawiki/php-master/extensions/Wikibase/data-access/src/ByTypeDispatchingPrefetchingTermLookup.php(89): Wikibase\DataAccess\ByTypeDispatchingPrefetchingTermLookup->prefetchTerms(array, array, array)
#7 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/EntityTermLookupBase.php(52): Wikibase\DataAccess\ByTypeDispatchingPrefetchingTermLookup->getTermsOfType(Wikibase\DataModel\Entity\PropertyId, string, array)
#8 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/LanguageFallbackLabelDescriptionLookup.php(47): Wikibase\Lib\Store\EntityTermLookupBase->getLabels(Wikibase\DataModel\Entity\PropertyId, array)
#9 /srv/mediawiki/php-master/extensions/Wikibase/client/includes/Usage/UsageTrackingLanguageFallbackLabelDescriptionLookup.php(72): Wikibase\Lib\Store\LanguageFallbackLabelDescriptionLookup->getLabel(Wikibase\DataModel\Entity\PropertyId)
#10 /srv/mediawiki/php-master/extensions/Wikibase/client/includes/DataAccess/Scribunto/WikibaseLanguageDependentLuaBindings.php(60): Wikibase\Client\Usage\UsageTrackingLanguageFallbackLabelDescriptionLookup->getLabel(Wikibase\DataModel\Entity\PropertyId)
#11 /srv/mediawiki/php-master/extensions/Wikibase/client/includes/DataAccess/Scribunto/Scribunto_LuaWikibaseLibrary.php(586): Wikibase\Client\DataAccess\Scribunto\WikibaseLanguageDependentLuaBindings->getLabel(string)
#12 /srv/mediawiki/php-master/extensions/Scribunto/includes/engines/LuaSandbox/Engine.php(391): Wikibase\Client\DataAccess\Scribunto\Scribunto_LuaWikibaseLibrary->getLabel(string)
#13 [internal function]: Scribunto_LuaSandboxCallback->__call(string, array)
#14 /srv/mediawiki/php-master/extensions/Scribunto/includes/engines/LuaSandbox/Engine.php(314): LuaSandboxFunction->call(LuaSandboxFunction)
#15 /srv/mediawiki/php-master/extensions/Scribunto/includes/engines/LuaCommon/LuaCommon.php(296): Scribunto_LuaSandboxInterpreter->callFunction(LuaSandboxFunction, LuaSandboxFunction)
#16 /srv/mediawiki/php-master/extensions/Scribunto/includes/engines/LuaCommon/LuaCommon.php(983): Scribunto_LuaEngine->executeFunctionChunk(LuaSandboxFunction, PPTemplateFrame_Hash)
#17 /srv/mediawiki/php-master/extensions/Scribunto/includes/common/Hooks.php(128): Scribunto_LuaModule->invoke(string, PPTemplateFrame_Hash)
#18 /srv/mediawiki/php-master/includes/parser/Parser.php(3594): ScribuntoHooks::invokeHook(Parser, PPFrame_Hash, array)
#19 /srv/mediawiki/php-master/includes/parser/Parser.php(3301): Parser->callParserFunction(PPFrame_Hash, string, array)
#20 /srv/mediawiki/php-master/includes/parser/PPFrame_Hash.php(254): Parser->braceSubstitution(array, PPFrame_Hash)
#21 /srv/mediawiki/php-master/includes/parser/Parser.php(3115): PPFrame_Hash->expand(PPNode_Hash_Tree, integer)
#22 /srv/mediawiki/php-master/includes/parser/Parser.php(1424): Parser->replaceVariables(string)
#23 /srv/mediawiki/php-master/includes/parser/Parser.php(555): Parser->internalParse(string)
#24 /srv/mediawiki/php-master/includes/content/WikitextContent.php(365): Parser->parse(string, Title, ParserOptions, boolean, boolean, NULL)
#25 /srv/mediawiki/php-master/includes/content/AbstractContent.php(555): WikitextContent->fillParserOutput(Title, NULL, ParserOptions, boolean, ParserOutput)
#26 /srv/mediawiki/php-master/includes/Revision/RenderedRevision.php(266): AbstractContent->getParserOutput(Title, NULL, ParserOptions, boolean)
#27 /srv/mediawiki/php-master/includes/Revision/RenderedRevision.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(WikitextContent, boolean)
#28 /srv/mediawiki/php-master/includes/Revision/RevisionRenderer.php(199): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string)
#29 /srv/mediawiki/php-master/includes/Revision/RevisionRenderer.php(148): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#30 [internal function]: Closure$MediaWiki\Revision\RevisionRenderer::getRenderedRevision#2(MediaWiki\Revision\RenderedRevision, array)
#31 /srv/mediawiki/php-master/includes/Revision/RenderedRevision.php(197): call_user_func(Closure$MediaWiki\Revision\RevisionRenderer::getRenderedRevision#2;8004, MediaWiki\Revision\RenderedRevision, array)
#32 /srv/mediawiki/php-master/includes/Storage/DerivedPageDataUpdater.php(1289): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#33 [internal function]: MediaWiki\Storage\DerivedPageDataUpdater->getCanonicalParserOutput()
#34 /srv/mediawiki/php-master/includes/edit/PreparedEdit.php(104): call_user_func(array)
#35 /srv/mediawiki/php-master/includes/edit/PreparedEdit.php(119): MediaWiki\Edit\PreparedEdit->getOutput()
#36 /srv/mediawiki/php-master/includes/Storage/DerivedPageDataUpdater.php(1267): MediaWiki\Edit\PreparedEdit->__get(string)
#37 /srv/mediawiki/php-master/includes/page/WikiPage.php(2019): MediaWiki\Storage\DerivedPageDataUpdater->getPreparedEdit()
#38 /srv/mediawiki/php-master/extensions/SpamBlacklist/includes/SpamBlacklistHooks.php(31): WikiPage->prepareContentForEdit(WikitextContent)
#39 /srv/mediawiki/php-master/includes/Hooks.php(174): SpamBlacklistHooks::filterMergedContent(DerivativeContext, WikitextContent, Status, string, User, boolean)
#40 /srv/mediawiki/php-master/includes/Hooks.php(202): Hooks::callHook(string, array, array, NULL)
#41 /srv/mediawiki/php-master/includes/EditPage.php(1766): Hooks::run(string, array)
#42 /srv/mediawiki/php-master/includes/EditPage.php(2221): EditPage->runPostMergeFilters(WikitextContent, Status, User)
#43 /srv/mediawiki/php-master/includes/EditPage.php(1596): EditPage->internalAttemptSave(NULL, boolean)
#44 /srv/mediawiki/php-master/includes/api/ApiEditPage.php(378): EditPage->attemptSave(NULL)
#45 /srv/mediawiki/php-master/includes/api/ApiMain.php(1583): ApiEditPage->execute()
#46 /srv/mediawiki/php-master/includes/api/ApiMain.php(500): ApiMain->executeAction()
#47 /srv/mediawiki/php-master/extensions/VisualEditor/includes/ApiVisualEditorEdit.php(74): ApiMain->execute()
#48 /srv/mediawiki/php-master/extensions/VisualEditor/includes/ApiVisualEditorEdit.php(400): ApiVisualEditorEdit->saveWikitext(Title, string, array)
#49 /srv/mediawiki/php-master/includes/api/ApiMain.php(1583): ApiVisualEditorEdit->execute()
#50 /srv/mediawiki/php-master/includes/api/ApiMain.php(531): ApiMain->executeAction()
#51 /srv/mediawiki/php-master/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()
#52 /srv/mediawiki/php-master/api.php(86): ApiMain->execute()
#53 /srv/mediawiki/w/api.php(3): include(string)
#54 {main}

It seems PrefetchingPropertyTermLookup needs the DB domain as well.

Change 523951 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Pass db domain to Prefetching(Item|Property)TermLookup

https://gerrit.wikimedia.org/r/523951

I assume this is fixed now (didn't test, though).

Change 523951 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Pass db domain to Prefetching(Item|Property)TermLookup

https://gerrit.wikimedia.org/r/523951

Tested, still explodes on beta cluster (To reproduce, edit https://en.wikipedia.beta.wmflabs.org/w/index.php?title=Italic_title&veaction=editsource&action=edit and add {{#invoke:Wikidata test|hello}} and then try to save)

that's another place that need to be patched now .. will do the same solution (and will look for all read-related classes and apply that to them to to avoid this exploding if we ever use them in clients/federation)

I found this error if that's any of help:

#0 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(1573): Wikimedia\Rdbms\Database->getQueryExceptionAndLog(string, integer, string, string)
#1 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(1162): Wikimedia\Rdbms\Database->reportQueryError(string, integer, string, string, boolean)
#2 /srv/mediawiki/php-master/includes/libs/rdbms/database/Database.php(1801): Wikimedia\Rdbms\Database->query(string, string)
#3 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/Sql/Terms/PrefetchingPropertyTermLookup.php(90): Wikimedia\Rdbms\Database->select(string, array, array, string)
#4 /srv/mediawiki/php-master/extensions/Wikibase/data-access/src/ByTypeDispatchingPrefetchingTermLookup.php(53): Wikibase\Lib\Store\Sql\Terms\PrefetchingPropertyTermLookup->prefetchTerms(array, array, array)
#5 /srv/mediawiki/php-master/extensions/Wikibase/data-access/src/ByTypeDispatchingPrefetchingTermLookup.php(53): Wikibase\DataAccess\ByTypeDispatchingPrefetchingTermLookup->prefetchTerms(array, array, array)
#6 /srv/mediawiki/php-master/extensions/Wikibase/data-access/src/ByTypeDispatchingPrefetchingTermLookup.php(89): Wikibase\DataAccess\ByTypeDispatchingPrefetchingTermLookup->prefetchTerms(array, array, array)
#7 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/EntityTermLookupBase.php(52): Wikibase\DataAccess\ByTypeDispatchingPrefetchingTermLookup->getTermsOfType(Wikibase\DataModel\Entity\PropertyId, string, array)
#8 /srv/mediawiki/php-master/extensions/Wikibase/lib/includes/Store/LanguageFallbackLabelDescriptionLookup.php(47): Wikibase\Lib\Store\EntityTermLookupBase->getLabels(Wikibase\DataModel\Entity\PropertyId, array)
#9 /srv/mediawiki/php-master/extensions/Wikibase/client/includes/Usage/UsageTrackingLanguageFallbackLabelDescriptionLookup.php(72): Wikibase\Lib\Store\LanguageFallbackLabelDescriptionLookup->getLabel(Wikibase\DataModel\Entity\PropertyId)
#10 /srv/mediawiki/php-master/extensions/Wikibase/client/includes/DataAccess/Scribunto/WikibaseLanguageDependentLuaBindings.php(60): Wikibase\Client\Usage\UsageTrackingLanguageFallbackLabelDescriptionLookup->getLabel(Wikibase\DataModel\Entity\PropertyId)
#11 /srv/mediawiki/php-master/extensions/Wikibase/client/includes/DataAccess/Scribunto/Scribunto_LuaWikibaseLibrary.php(586): Wikibase\Client\DataAccess\Scribunto\WikibaseLanguageDependentLuaBindings->getLabel(string)
#12 /srv/mediawiki/php-master/extensions/Scribunto/includes/engines/LuaSandbox/Engine.php(391): Wikibase\Client\DataAccess\Scribunto\Scribunto_LuaWikibaseLibrary->getLabel(string)
#13 [internal function]: Scribunto_LuaSandboxCallback->__call(string, array)
#14 /srv/mediawiki/php-master/extensions/Scribunto/includes/engines/LuaSandbox/Engine.php(314): LuaSandboxFunction->call(LuaSandboxFunction)
#15 /srv/mediawiki/php-master/extensions/Scribunto/includes/engines/LuaCommon/LuaCommon.php(296): Scribunto_LuaSandboxInterpreter->callFunction(LuaSandboxFunction, LuaSandboxFunction)
#16 /srv/mediawiki/php-master/extensions/Scribunto/includes/engines/LuaCommon/LuaCommon.php(983): Scribunto_LuaEngine->executeFunctionChunk(LuaSandboxFunction, PPTemplateFrame_Hash)
#17 /srv/mediawiki/php-master/extensions/Scribunto/includes/common/Hooks.php(128): Scribunto_LuaModule->invoke(string, PPTemplateFrame_Hash)
#18 /srv/mediawiki/php-master/includes/parser/Parser.php(3594): ScribuntoHooks::invokeHook(Parser, PPFrame_Hash, array)
#19 /srv/mediawiki/php-master/includes/parser/Parser.php(3301): Parser->callParserFunction(PPFrame_Hash, string, array)
#20 /srv/mediawiki/php-master/includes/parser/PPFrame_Hash.php(254): Parser->braceSubstitution(array, PPFrame_Hash)
#21 /srv/mediawiki/php-master/includes/parser/Parser.php(3115): PPFrame_Hash->expand(PPNode_Hash_Tree, integer)
#22 /srv/mediawiki/php-master/includes/parser/Parser.php(1424): Parser->replaceVariables(string)
#23 /srv/mediawiki/php-master/includes/parser/Parser.php(555): Parser->internalParse(string)
#24 /srv/mediawiki/php-master/includes/content/WikitextContent.php(365): Parser->parse(string, Title, ParserOptions, boolean, boolean, NULL)
#25 /srv/mediawiki/php-master/includes/content/AbstractContent.php(555): WikitextContent->fillParserOutput(Title, NULL, ParserOptions, boolean, ParserOutput)
#26 /srv/mediawiki/php-master/includes/Revision/RenderedRevision.php(266): AbstractContent->getParserOutput(Title, NULL, ParserOptions, boolean)
#27 /srv/mediawiki/php-master/includes/Revision/RenderedRevision.php(234): MediaWiki\Revision\RenderedRevision->getSlotParserOutputUncached(WikitextContent, boolean)
#28 /srv/mediawiki/php-master/includes/Revision/RevisionRenderer.php(199): MediaWiki\Revision\RenderedRevision->getSlotParserOutput(string)
#29 /srv/mediawiki/php-master/includes/Revision/RevisionRenderer.php(148): MediaWiki\Revision\RevisionRenderer->combineSlotOutput(MediaWiki\Revision\RenderedRevision, array)
#30 [internal function]: Closure$MediaWiki\Revision\RevisionRenderer::getRenderedRevision#2(MediaWiki\Revision\RenderedRevision, array)
#31 /srv/mediawiki/php-master/includes/Revision/RenderedRevision.php(197): call_user_func(Closure$MediaWiki\Revision\RevisionRenderer::getRenderedRevision#2;8004, MediaWiki\Revision\RenderedRevision, array)
#32 /srv/mediawiki/php-master/includes/Storage/DerivedPageDataUpdater.php(1289): MediaWiki\Revision\RenderedRevision->getRevisionParserOutput()
#33 [internal function]: MediaWiki\Storage\DerivedPageDataUpdater->getCanonicalParserOutput()
#34 /srv/mediawiki/php-master/includes/edit/PreparedEdit.php(104): call_user_func(array)
#35 /srv/mediawiki/php-master/includes/edit/PreparedEdit.php(119): MediaWiki\Edit\PreparedEdit->getOutput()
#36 /srv/mediawiki/php-master/includes/Storage/DerivedPageDataUpdater.php(1267): MediaWiki\Edit\PreparedEdit->__get(string)
#37 /srv/mediawiki/php-master/includes/page/WikiPage.php(2019): MediaWiki\Storage\DerivedPageDataUpdater->getPreparedEdit()
#38 /srv/mediawiki/php-master/extensions/SpamBlacklist/includes/SpamBlacklistHooks.php(31): WikiPage->prepareContentForEdit(WikitextContent)
#39 /srv/mediawiki/php-master/includes/Hooks.php(174): SpamBlacklistHooks::filterMergedContent(DerivativeContext, WikitextContent, Status, string, User, boolean)
#40 /srv/mediawiki/php-master/includes/Hooks.php(202): Hooks::callHook(string, array, array, NULL)
#41 /srv/mediawiki/php-master/includes/EditPage.php(1766): Hooks::run(string, array)
#42 /srv/mediawiki/php-master/includes/EditPage.php(2221): EditPage->runPostMergeFilters(WikitextContent, Status, User)
#43 /srv/mediawiki/php-master/includes/EditPage.php(1596): EditPage->internalAttemptSave(NULL, boolean)
#44 /srv/mediawiki/php-master/includes/api/ApiEditPage.php(378): EditPage->attemptSave(NULL)
#45 /srv/mediawiki/php-master/includes/api/ApiMain.php(1583): ApiEditPage->execute()
#46 /srv/mediawiki/php-master/includes/api/ApiMain.php(500): ApiMain->executeAction()
#47 /srv/mediawiki/php-master/extensions/VisualEditor/includes/ApiVisualEditorEdit.php(74): ApiMain->execute()
#48 /srv/mediawiki/php-master/extensions/VisualEditor/includes/ApiVisualEditorEdit.php(400): ApiVisualEditorEdit->saveWikitext(Title, string, array)
#49 /srv/mediawiki/php-master/includes/api/ApiMain.php(1583): ApiVisualEditorEdit->execute()
#50 /srv/mediawiki/php-master/includes/api/ApiMain.php(531): ApiMain->executeAction()
#51 /srv/mediawiki/php-master/includes/api/ApiMain.php(502): ApiMain->executeActionWithErrorHandling()
#52 /srv/mediawiki/php-master/api.php(86): ApiMain->execute()
#53 /srv/mediawiki/w/api.php(3): include(string)
#54 {main}

that's the callstack of that same error right?

that's the callstack of that same error right?

No, they are different, this one is PrefetchingPropertyTermLookup we need to pass dbDomain there

okaydoke will go over all of them anyway then

alaa_wmde removed alaa_wmde as the assignee of this task.Thu, Jul 18, 11:29 AM
alaa_wmde added a subscriber: alaa_wmde.

My hypotheses: After hours of investigating I realized the big reason is that federation (i.e. $wmgUseEntitySourceBasedFederation) is enabled for all of beta cluster while it's only properly configured for commons and that's why the same thing works just fine in beta commons (https://commons.wikimedia.beta.wmflabs.org/wiki/Artturi). (I just realized that we disabled proprety term migration for commons in beta cluster because it practically blew up the whole website) I'm going to make configuration of beta cluster similar to production as other configurations are similar and then check enwiki in beta cluster to see if it's fixed for good or not.

Change 524296 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[operations/mediawiki-config@master] labs: Make $wmgUseEntitySourceBasedFederation similar to production

https://gerrit.wikimedia.org/r/524296

Change 524296 merged by jenkins-bot:
[operations/mediawiki-config@master] labs: Make $wmgUseEntitySourceBasedFederation similar to production

https://gerrit.wikimedia.org/r/524296

My hypotheses: After hours of investigating I realized the big reason is that federation (i.e. $wmgUseEntitySourceBasedFederation) is enabled for all of beta cluster while it's only properly configured for commons and that's why the same thing works just fine in beta commons (https://commons.wikimedia.beta.wmflabs.org/wiki/Artturi). (I just realized that we disabled proprety term migration for commons in beta cluster because it practically blew up the whole website) I'm going to make configuration of beta cluster similar to production as other configurations are similar and then check enwiki in beta cluster to see if it's fixed for good or not.

I was wrong. It didn't fix anything

So today spent few hours tweaking mediawiki-docker-dev to make instantiate two instances: a repo and a client (so far the actual configuration are not bundled) T228551: Draft docker-compose with two instances repo/client of wikibase
Seems to work to some degree, but I seem to miss some configuration + I really need someone who understands how federation works fully as that's turning into a very deep rabbit hole in my head

So after setting up Scribunto to do the same test that @Ladsgroup done on beta, and it seems just to work.

->

Q1 exists only in repo instance (take not of the url):

and does not exist in client instance (take note of the url):

And my current client configurations is the following:

1<?php
2
3require_once __DIR__ . '/.docker/LocalSettings.php';
4
5wfLoadExtension( 'UniversalLanguageSelector' );
6wfLoadExtension( 'Interwiki' );
7$wgGroupPermissions['sysop']['interwiki'] = true;
8
9wfLoadExtension( 'Scribunto' );
10$wgScribuntoDefaultEngine = 'luastandalone';
11
12wfLoadExtension( 'WikiEditor' );
13wfLoadExtension( 'SyntaxHighlight_GeSHi' );
14wfLoadExtension( 'CodeEditor' );
15$wgDefaultUserOptions['usebetatoolbar'] = 1;
16$wgScribuntoUseGeSHi = true;
17
18wfLoadSkin( 'Vector' );
19wfLoadExtension( 'Elastica' );
20wfLoadExtension( 'CirrusSearch' );
21
22$wgSearchType = 'CirrusSearch';
23
24// $wgEnableWikibaseRepo = true;
25// $wgEnableWikibaseClient = true;
26require_once "$IP/extensions/Wikibase/repo/Wikibase.php";
27require_once "$IP/extensions/Wikibase/repo/ExampleSettings.php";
28require_once "$IP/extensions/Wikibase/client/WikibaseClient.php";
29require_once "$IP/extensions/Wikibase/client/ExampleSettings.php";
30
31wfLoadExtension( 'WikibaseCirrusSearch' );
32wfLoadExtension( 'WikibaseMediaInfo' );
33
34$wgMediaInfoEnableFilePageDepicts = true;
35
36$wgMediaInfoProperties = [
37 'depicts' => 'P1',
38 'inherits' => 'P2'
39];
40
41$wgDepictsQualifierProperties = [
42 'features' => 'P3'
43];
44
45$wgWBRepoSettings['entityNamespaces'] = [];
46
47$wgWBRepoSettings['foreignRepositories'] = [
48 'd' => [
49 'repoDatabase' => 'repo',
50 'baseUri' => 'http://repo.web.mw.localhost:80/',
51 'supportedEntityTypes' => [ 'item', 'property' ],
52 'prefixMapping' => [],
53 'entityNamespaces' => [ 'item' => 0, 'property' => 120 ]
54 ],
55];
56
57$wgWBClientSettings['repoUrl'] = "http://default.web.mw.localhost:8098";
58$wgWBClientSettings['repoDatabase'] = "repo";
59$wgWBClientSettings['repoArticlePath'] = "/mediawiki/index.php/$1";
60$wgWBClientSettings['entityNamespaces'] = [
61 'item' => 0,
62 'property' => 120,
63];
64$wgWBClientSettings['repositories'] = [
65 '' => [
66 'repoDatabase' => 'repo',
67 'entityNamespaces' => [
68 'item' => 0,
69 'property' => 120,
70 'lexeme' => 146,
71 ],
72 'baseUri' => 'http://default.web.mw.localhost:8098/mediawiki',
73 'prefixMapping' => [ '' => '' ],
74 ],
75 'commons' => [
76 'repoDatabase' => 'default',
77 'entityNamespaces' => [ 'mediainfo' => '6/mediainfo' ],
78 'baseUri' => 'http://default.web.mw.localhost:8099/mediawiki',
79 'prefixMapping' => [],
80 ],
81];
82
83$wgEnableUploads = true;
84$wgHashedUploadDirectory = true;
85$wgUploadDirectory = 'images/';
86$wgUploadPath = 'images/';