Page MenuHomePhabricator

Allow accessing Wikibase entities from multiple (Wikibase) databases
Open, HighPublic

Description

Following the discussion in T211800, assuming for now a single entity type can only be provided by single Wikibase instance, make it possible for wikibase to access entities provided by multiple sources.
For now, only direct database database access is considered.

The general idea would be to define "sources" that are providing entities of particular type. Services providing access to entity data will be defined per source (i,e. a particular service is configured to access the database of given source/instance/repository). Wikibase will determine which source-specific service to use to access data of particular entity based on the entity-type vs source configuration.

Entity ID prefixes are not considered for now, as long as there is no technical need (state January 2019) to have a single entity type provided by multiple Wikibase instances/sources./databases.

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 485855 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Added PropertyInfoLookup to *EntitySourceServices

https://gerrit.wikimedia.org/r/485855

Change 485862 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Use *EntitySourceServices as a drop-in replacement for repo-based WikibaseServices

https://gerrit.wikimedia.org/r/485862

Change 487148 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added test for MultipleEntitySourceServices::getEntityRevisionLookup

https://gerrit.wikimedia.org/r/487148

Change 485671 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added ByTypeDispatchingEntityInfoBuilder

https://gerrit.wikimedia.org/r/485671

Change 487369 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added useEntitySourceBasedFederation flag to DataAccessSettings

https://gerrit.wikimedia.org/r/487369

Change 487372 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added dummy EntitySource class exploding whenever used

https://gerrit.wikimedia.org/r/487372

Change 487373 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added EntitySource to SqlEntityInfoBuilder

https://gerrit.wikimedia.org/r/487373

Change 485672 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added EntityInfoBuilder to *EntitySourceServices

https://gerrit.wikimedia.org/r/485672

Change 487967 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Added WikibaseClient::getItemTermIndex

https://gerrit.wikimedia.org/r/487967

Change 487494 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added EntitySource to TermSqlIndex

https://gerrit.wikimedia.org/r/487494

Change 488008 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/WikibaseMediaInfo@master] Adjust WikiPageEntityMetaDataLookup constructor call and re-enabled the test

https://gerrit.wikimedia.org/r/488008

Change 485690 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added TermSearchInteractoryFactory to *EntitySourceServices

https://gerrit.wikimedia.org/r/485690

Change 485827 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added ByTypeDispatchingPrefetchingTermLookup

https://gerrit.wikimedia.org/r/485827

Change 485828 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added PrefetchingTermLookup to *EntitySourceServices

https://gerrit.wikimedia.org/r/485828

Change 487636 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Temporarily skip tests that call WikiPageEntityMetaDataLookup's constructor

https://gerrit.wikimedia.org/r/487636

Change 487607 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added EntitySource to WikiPageEntityMetaDataLookup

https://gerrit.wikimedia.org/r/487607

Change 485829 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added ByTypeDispatchingEntityPrefetcher

https://gerrit.wikimedia.org/r/485829

Change 488040 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Added concept base URI to Entity Source and Definitions

https://gerrit.wikimedia.org/r/488040

Change 488008 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Adjust WikiPageEntityMetaDataLookup constructor call and re-enabled the test

https://gerrit.wikimedia.org/r/488008

Change 488050 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Use concept base URIs from source definitions when using source config instead of repository prefix one

https://gerrit.wikimedia.org/r/488050

Change 485830 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added EntityPrefetcher to *EntitySourceServices

https://gerrit.wikimedia.org/r/485830

Change 487645 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added EntitySource to PropertyInfoTable

https://gerrit.wikimedia.org/r/487645

Change 485855 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added PropertyInfoLookup to *EntitySourceServices

https://gerrit.wikimedia.org/r/485855

Change 485861 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] MultipleEntitySourceServices implements WikibaseServices interface

https://gerrit.wikimedia.org/r/485861

Change 485862 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Use MultipleEntitySourceServices as a drop-in replacement for repo-based WikibaseServices

https://gerrit.wikimedia.org/r/485862

Change 488092 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Take EntitySources into account in EntityContentFactory::getTitleForId

https://gerrit.wikimedia.org/r/488092

Change 487967 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added WikibaseClient::getItemTermIndex

https://gerrit.wikimedia.org/r/487967

Change 488040 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added concept base URI to Entity Source and Definitions

https://gerrit.wikimedia.org/r/488040

Change 488050 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Use concept base URIs from source definitions when using source config instead of repository prefix one

https://gerrit.wikimedia.org/r/488050

Change 488092 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Take EntitySources into account in EntityContentFactory::getTitleForId

https://gerrit.wikimedia.org/r/488092

Change 488500 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Add interwiki prefix to Entity Source

https://gerrit.wikimedia.org/r/488500

Change 488924 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Use prefix of the entity source when generating Title objects in EntityContentFactory

https://gerrit.wikimedia.org/r/488924

Change 488966 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Added EntitySource to WikiPageEntityStore

https://gerrit.wikimedia.org/r/488966

Change 489385 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] wbsearchentities: Show name of entity source name in "repository" part of API response

https://gerrit.wikimedia.org/r/489385

Change 489388 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Do not use ForeignEntityValidator when using entity-source based access to other Wikibase instance entities

https://gerrit.wikimedia.org/r/489388

Change 489447 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Made RdfVocabulary work with EntitySources

https://gerrit.wikimedia.org/r/489447

Change 489449 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Fixed entity-source-based-access branch of WikibaseRepo::getEntityTypeToRepositoryMapping

https://gerrit.wikimedia.org/r/489449

Change 489454 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Made RepoLinker aware of entity sources

https://gerrit.wikimedia.org/r/489454

Change 488500 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Add interwiki prefix to Entity Source

https://gerrit.wikimedia.org/r/488500

Change 488924 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Use prefix of the entity source when generating Title objects in EntityContentFactory

https://gerrit.wikimedia.org/r/488924

Change 488966 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added EntitySource to WikiPageEntityStore

https://gerrit.wikimedia.org/r/488966

Change 489385 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] wbsearchentities: Show name of entity source name in "repository" part of API response

https://gerrit.wikimedia.org/r/489385

Change 489698 had a related patch set uploaded (by Addshore; owner: Addshore):
[operations/mediawiki-config@master] Wikibase.php, add conditional setting of useEntitySourceBasedFederation

https://gerrit.wikimedia.org/r/489698

Change 489701 had a related patch set uploaded (by Addshore; owner: Addshore):
[operations/mediawiki-config@master] BETA: wmgUseEntitySourceBasedFederation true

https://gerrit.wikimedia.org/r/489701

Change 489698 merged by jenkins-bot:
[operations/mediawiki-config@master] Wikibase.php, add conditional setting of useEntitySourceBasedFederation

https://gerrit.wikimedia.org/r/489698

Change 489388 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Do not use ForeignEntityValidator when using entity-source based access to other Wikibase instance entities

https://gerrit.wikimedia.org/r/489388

Change 489449 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Fixed entity-source-based-access branch of WikibaseRepo::getEntityTypeToRepositoryMapping

https://gerrit.wikimedia.org/r/489449

Change 489701 merged by jenkins-bot:
[operations/mediawiki-config@master] BETA: wmgUseEntitySourceBasedFederation true

https://gerrit.wikimedia.org/r/489701

The EntitySource stuff is now turned on on beta

Change 489771 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/WikibaseQualityConstraints@master] Temporarily skip tests creating RdfVocabulary instance

https://gerrit.wikimedia.org/r/489771

Change 489454 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Made RepoLinker aware of entity sources

https://gerrit.wikimedia.org/r/489454

Change 490060 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Added a dedicated config setting for entity sources, and a service processing it

https://gerrit.wikimedia.org/r/490060

Change 490104 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[operations/mediawiki-config@master] Added wmgWikibaseEntitySources setting for defining Wikibase "entity sources"

https://gerrit.wikimedia.org/r/490104

Change 490108 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[operations/mediawiki-config@master] DNM Define Wikibase "entity sources" on beta commons

https://gerrit.wikimedia.org/r/490108

Change 490145 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Moved out logic on creating entity source definitions from backwards compatible settings out of WikibaseRepo/Client

https://gerrit.wikimedia.org/r/490145

Change 490633 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[operations/mediawiki-config@master] Added wmgWikibaseRepoLocalEntitySourceName to define the "local" source of Wikibase Repo

https://gerrit.wikimedia.org/r/490633

Change 490634 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[operations/mediawiki-config@master] Added wmgUseEntitySourceBasedFederation setting to switch the federation mechanism

https://gerrit.wikimedia.org/r/490634

Change 490658 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[operations/mediawiki-config@master] DNM Use "entity source based federation" on Beta Commons

https://gerrit.wikimedia.org/r/490658

Change 490658 abandoned by WMDE-leszek:
DNM Use "entity source based federation" on Beta Commons

Reason:
not needed, done already

https://gerrit.wikimedia.org/r/490658

Change 490634 abandoned by WMDE-leszek:
Added wmgUseEntitySourceBasedFederation setting to switch the federation mechanism

https://gerrit.wikimedia.org/r/490634

Change 491534 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Documented entity-source-related config options.

https://gerrit.wikimedia.org/r/491534

Discovered T216686 which might be caused by this endeavour.

Change 492263 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Fixed WikibaseRepo::getLocalRepoWikiPageMetaDataAccessor when using "entity source based federation"

https://gerrit.wikimedia.org/r/492263

Change 492263 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Fixed WikibaseRepo::getLocalRepoWikiPageMetaDataAccessor when using "entity source based federation"

https://gerrit.wikimedia.org/r/492263

Change 494052 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] WikibaseRepo::newValidatorBuilders: do not rely on RepositoryDefinitions when the "entity source based federation" is in use.

https://gerrit.wikimedia.org/r/494052

Change 494055 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] WIP Made WikibaseClient::getRepoItemUriParser use data from entity source definitions, when "entity source based federation" in use

https://gerrit.wikimedia.org/r/494055

Change 494650 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Constructing GenericServices: use entity source defintions when applicable

https://gerrit.wikimedia.org/r/494650

Change 494651 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Use (conditionally) EntitySourceDefinitions in WikibaseRepo::getEnabledEntityTypes

https://gerrit.wikimedia.org/r/494651

Change 489794 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] WIP Add tests for RDF mapping for the properties defined in "foreign" entity source ase

https://gerrit.wikimedia.org/r/489794

Change 490060 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Added a dedicated config setting for entity sources, and a service processing it

https://gerrit.wikimedia.org/r/490060

Change 491534 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Documented entity-source-related config options.

https://gerrit.wikimedia.org/r/491534

Change 490104 had a related patch set uploaded (by Jforrester; owner: WMDE-leszek):
[operations/mediawiki-config@master] Added wmgWikibaseEntitySources setting for defining Wikibase "entity sources"

https://gerrit.wikimedia.org/r/490104

Change 490633 had a related patch set uploaded (by Jforrester; owner: WMDE-leszek):
[operations/mediawiki-config@master] Added wmgWikibaseRepoLocalEntitySourceName to define the "local" source of Wikibase Repo

https://gerrit.wikimedia.org/r/490633

Change 490108 had a related patch set uploaded (by Jforrester; owner: WMDE-leszek):
[operations/mediawiki-config@master] DNM Define Wikibase "entity sources" on beta commons

https://gerrit.wikimedia.org/r/490108

Change 489447 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Made RdfVocabulary work with EntitySources

https://gerrit.wikimedia.org/r/489447

Change 489794 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Add tests for RDF mapping for the properties defined in "foreign" entity source ase

https://gerrit.wikimedia.org/r/489794

Change 517889 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/WikibaseQualityConstraints@master] Temporarily skip tests instantiating RdfVocabulary object from Wikibase

https://gerrit.wikimedia.org/r/517889

Change 517022 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Add RDF namespace prefixes to the entity source config

https://gerrit.wikimedia.org/r/517022

Change 517021 had a related patch set uploaded (by WMDE-leszek; owner: WMDE-leszek):
[mediawiki/extensions/Wikibase@master] Prefix RDF (turtle) namespaces with configurable prefixes instead of using a source/repository suffix

https://gerrit.wikimedia.org/r/517021