Page MenuHomePhabricator

Uses of wb_terms SQL table to be migrated away from
Closed, ResolvedPublic


This is a "tracking" mega ticket about the uses of wb_terms table of Wikibase. It is covering the work towards migrating Wikibase away from using this unmaintainable table.

Each of the use cases will be looked at individually, with dedicated tickets, sprints, etc.

Note: while some of uses are considered there, not all of the below are intended to be covered in Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing) !

Note: the order of uses does NOT reflect their importance, priority, urgency, etc.

Search using API, and Special:Search

  • Find entities (items, properties) that have “terms” (in case of items/properties: label, or alias) matching the search term, possibly limit to the data in language given
  • Status: not started
    • Already existing alternative not using wb_terms: use ElasticSearch/CirrusSearch
    • Still missing: search "index" for non-Cirrus installation (somehow simplified and more search-specific than wb_terms)

Display on wiki pages

  • Display entities when referenced from other entities (i.e. used in statements), or when the entity page is referenced as a link
  • For items and properties gets labels and descriptions, possibly in multiple languages, given the “language fallback chain” functionality of Wikibase
  • Status: worked on
    • Displaying entity references as link to entity page being explored as T196882
      • Link formatting is done in two ways in Wikibase. Using Wikibase\Lib\EntityIdLinkFormatter and related formatters (e.g. links in statements), but also implementations of Wikibase\Repo\Hooks\Formatters\EntityLinkFormatter (in listings like special pages)

"Prefetching" of entity "terms"

  • Somewhat similar to the “Display on wiki pages” use case.
  • There are implementations/services that do some “caching” of labels etc of the given item, and/or do lookup for batch of item/property IDs, to minimize the number of database queries.
  • Status: worked on

Client Description Query API

  • Returns a description of the item related to the wiki page via a sitelink (in user’s language)
  • Status: not started

Client Page Terms Query API

  • Returns label, and/or description, and/or aliases of the item related to the wiki page via a sitelink (in user’s language)
  • Status: not started

PropertySuggester extension

  • Find property with a label and/or alias matching the string input by the user.
  • Status: not started
    • There is the alternative solution that does not use wb_terms, but uses ElasticSearch/CirrusSearch-based search instead
    • Still missing: option for non-CirrusSearch installations (probably similar to the general "Search" use case)

ArticlePlaceholder extension

  • Hooks in Special:Search to offer a placeholder when no search results are found
  • Status: not started

Conflict Detection

  • Finding items/properties with duplicate labels, and/or descriptions, and/or aliases. TBD: clarify how user actually does that, i.e. where
  • Status: not started

Lua: get data of the entity

  • Lua functions return the label or description of the item/property (TBD: what about Lexemes? Media Info entitities?), in a given language, or in the language of the local wiki. Also including the “language fallback” functionality of Wikibase
  • Functions:
    • mw.wikibase.getLabel
    • mw.wikibase.getLabelWithLang
    • mw.wikibase.getLabelByLang
    • mw.wikibase.getDescription
    • mw.wikibase.getDescriptionWithLang
  • Status: not started

Lua: formatting entity references

  • Functions:
    • mw.wikibase.renderSnak
    • mw.wikibase.formatValue
    • mw.wikibase.renderSnaks
    • mw.wikibase.formatValues
  • Status: not started

Lua: Find property by label

  • Lua functions allow finding the ID of the property by giving the property label (which is supposed unique for properties)
  • Function: mw.wikibase.resolvePropertyId


  • Page listing items with the same label in the given language
  • Status: not started


  • Special page for finding items/properties without a label or description
  • Status: not started

Planned/No longer using: Wikidata Concept Monitor (WDCM)

  • Used to get English labels of items to be displayed from wb_terms. Now uses MW API.
  • However, fetching a large number of labels from the MW API - even in a monolingual context - takes a considerable time in the WDCM update cycle.
  • Planned: to make the WDCM multilingual, there would be a need to get a label of item/items in the given language, using the language fallback functionality.
  • See: T203466
  • Status: not started

Planned: Item “preview” through Popus extension

  • While hovering an item link, short information/”preview” would be displayed.
  • See T111231
  • Status: not started

Planned: Item labels in Echo notifications

  • Notifications related to edits to Wikibase entities should contain their "labels", not just IDs
  • See: T116762
  • Status: not started

Toolforge tool authors

  • wb_terms data of is replicated on tool forge, and made available for third-party tools
  • Actual use cases vary.
  • Status: needs planning
    • In T197161 a significant list of tools has been collected. Uses and requirements vary, thus we need to analyze what tools do, and look for alternatives for "clusters" of similar tools.

Event Timeline

WMDE-leszek renamed this task from Uses of wb_terms SQL table to be migrated away to Uses of wb_terms SQL table to be migrated away from.Jul 5 2018, 2:41 PM
Addshore claimed this task.

Going to go ahead and close this now.
Anything remaining is under T208425