Page MenuHomePhabricator
Create Task
Maniphest T114902

Remove numeric entity IDs from database schema
Closed, ResolvedPublic
Actions

Description

If we want to be able to use non-numeric entity IDs, we need to change our database schema to no longer use integer columns for representing entity IDs. The following tables are still using integer IDs:

  • wb_entity_per_page - killed
  • wb_items_per_site - not doing
  • wb_terms - being killed

Related Objects
Search...

StatusSubtypeAssignedTask
 OpenNoneT30599 Deadlock tracking bug (tracking)
 ResolvedhooT111535 Wikibase\Repo\Store\SQL\EntityPerPageTable::{closure} creating high number of deadlocks
 DeclinedNoneT51982 Add missing wb_entity_per_page entries on LinksUpdate
 InvalidLydia_PintscherT70176 EntityPerPageTable class should be usable from the client
 ResolvedLadsgroupT67333 Wikibase\EntityPerPageTable::getItemsWithoutSitelinks slow query with large LIMIT offset
 ResolvedAddshoreT114902 Remove numeric entity IDs from database schema
 ResolvedLadsgroupT95685 Drop wb_entity_per_page table
 Resolved• adrianheineT134873 Make EntityRevisionLookup not rely on entity_per_page
 ResolveddanielT140758 Investigate where wb_entity_per_page table is used and needed
 ResolvedLadsgroupT140876 In SqlEntityInfoBuilder, replace the usage of wb_entity_per_page with using the page and redirect tables
 ResolvedLadsgroupT140877 In PropertyInfoTableBuilder, replace the usage of wb_entity_per_page with using the page and redirect tables
 ResolvedLadsgroupT140890 Drop EntityPerPage service
 ResolvedhooT140888 Re-implement listEntities() in EntityPerPageIdPager (and rename to SqlEntityIdPager)
 ResolvedhooT140891 Factor EntityPerPage::getEntitiesWithoutTerm out into it's own service
 ResolvedhooT147638 Introduce a setting for entity types supported by Special:EntitiesWithoutLabel/ Special:EntitiesWithoutDescription
 ResolvedhooT140892 Factor EntityPerPage::getEntitiesWithoutSitelinks out into it's own service
 DeclinedNoneT114903 Migrate wb_terms to using prefixed entity IDs instead of numeric IDs
 ResolvedLydia_PintscherT159718 Evaluate how to best add a column for full entity ID to wb_terms without affecting wikidata.org users
 ResolvedNoneT159851 Add a column for full entity ID to wb_terms table
 ResolvedWMDE-leszekT168708 Populating full entity column in wb_terms table when running MW update.php should be fast and run only once
 DeclinedNoneT169720 Remove column storing numeric entity ID in wb_terms once it has become redundant
 DeclinedNoneT169721 Remove term_entity_id during database schema update when term_full_entity_id is used instead
 DeclinedNoneT169722 Remove term_entity_id column from wb_terms table when no longer needed on WMF cluster
 ResolvedLadsgroupT188995 Clean up the code base for term_entity_id
 ResolvedaudeT162533 Script for populating term_full_entity_id column in wb_terms table
 ResolvedMarosteguiT162539 Deploy schema change for adding term_full_entity_id column to wb_terms table
 ResolvedLadsgroupT162673 Adjust code reading from wb_terms table to use full entity ID
 ResolvedAndrewT167114 Open view for term_full_entity_id in wb_terms table in labs
 ResolvedLadsgroupT171460 Populate term_full_entity_id on www.wikidata.org
 ResolvedaudeT171461 Populate term_full_entity_id on test.wikidata.org
 ResolvedLadsgroupT167229 Change configuration of Wikidata in production to write term_full_entity_id
 ResolvedLadsgroupT165197 Change configuration of test Wikidata to write term_full_entity_id
 ResolvedLadsgroupT172776 Property labels missing on some items
 ResolvedMarosteguiT177601 Deploy dropping wb_entity_per_page table
 ResolvedAndrewT178661 Drop wb_entity_per_page views in Wiki Replicas
 DeclinedNoneT114904 Migrate wb_items_per_site to using prefixed entity IDs instead of numeric IDs

Event Timeline

daniel created this task.Oct 7 2015, 4:31 PM
daniel raised the priority of this task from to Needs Triage.
daniel updated the task description. (Show Details)
daniel subscribed.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 7 2015, 4:31 PM
daniel added a subtask: T95685: Drop wb_entity_per_page table.Oct 7 2015, 4:32 PM
daniel added projects: Wikidata, MediaWiki-extensions-WikibaseRepository, DBA.Oct 7 2015, 4:35 PM
daniel added a subscriber: hoo.
daniel added subscribers: JanZerebecki, aude.Oct 7 2015, 4:49 PM
Ricordisamoa subscribed.Oct 7 2015, 5:27 PM
Liuxinyu970226 subscribed.Dec 11 2015, 12:19 PM
Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptDec 11 2015, 12:19 PM
Liuxinyu970226 set Security to None.Dec 11 2015, 12:19 PM
matej_suchanek added a project: Tracking-Neverending.Dec 21 2015, 8:53 PM
• jcrespo mentioned this in T111769: [Bug] EntityUsageTable::touchUsageBatch slow query.Feb 5 2016, 1:50 PM
daniel added a subscriber: • jcrespo.Feb 5 2016, 2:28 PM
Danny_B renamed this task from [tracking] Remove numeric entity IDs from database schema to Remove numeric entity IDs from database schema (tracking).May 27 2016, 5:46 PM
Danny_B moved this task from Tag to Should be Goal instead on the Tracking-Neverending board.Jul 11 2016, 6:55 PM
• Phabricator_maintenance added a project: Goal.Aug 13 2016, 8:41 PM
• Phabricator_maintenance renamed this task from Remove numeric entity IDs from database schema (tracking) to Remove numeric entity IDs from database schema.Aug 13 2016, 11:22 PM
• Phabricator_maintenance removed a project: Tracking-Neverending.
thiemowmde triaged this task as Low priority.Sep 5 2016, 3:48 PM
thiemowmde subscribed.

I believe we all know about these and we have migration paths for all of them.

Lydia_Pintscher closed subtask T95685: Drop wb_entity_per_page table as Resolved.Oct 24 2017, 3:50 PM
Marostegui moved this task from Triage to Blocked external/Not db team on the DBA board.Nov 17 2017, 2:04 PM
Marostegui removed a project: DBA.Sep 12 2018, 4:53 PM
Addshore moved this task from incoming to needs discussion or investigation on the Wikidata board.Sep 17 2018, 8:14 AM
Addshore added a project: [DEPRECATED] wdwb-tech.
Addshore changed the status of subtask T114903: Migrate wb_terms to using prefixed entity IDs instead of numeric IDs from Open to Stalled.Sep 20 2018, 10:10 PM
Addshore added a project: Technical-Debt.Oct 31 2018, 4:57 PM
Addshore closed subtask T114903: Migrate wb_terms to using prefixed entity IDs instead of numeric IDs as Declined.Mar 25 2019, 4:18 PM
Addshore subscribed.Jun 22 2019, 9:48 PM

wb_terms is being killed, the new tables for storing terms do use numeric ids, but each table is specific to items / properties, which are fine with having numeric ids. so we don't need to fix that area any further.
https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/cd4957f2f0d5f54d565f065bcd134a1acbc33234/repo/sql/AddNormalizedTermsTablesDDL.sql

Addshore updated the task description. (Show Details)Jun 22 2019, 9:48 PM
matej_suchanek updated the task description. (Show Details)Jun 24 2019, 8:24 AM
Addshore closed this task as Resolved.Jun 30 2019, 6:27 PM
Addshore claimed this task.

Going to mark this as resolved.
We need not fix items per site (at least we have no reason to right now), items right now will always be representable as an integer and a prefix so can be represented as just an int in a table specific to them.
This is the same reasoning we have for only using ints in the new refactored terms storage tables where the table is specific to items.

Restricted Application added a project: User-Addshore. · View Herald TranscriptJun 30 2019, 6:27 PM
Addshore updated the task description. (Show Details)Jun 30 2019, 6:27 PM
Addshore closed subtask T114904: Migrate wb_items_per_site to using prefixed entity IDs instead of numeric IDs as Declined.
Addshore mentioned this in T114904: Migrate wb_items_per_site to using prefixed entity IDs instead of numeric IDs.
daniel unsubscribed.Jun 30 2019, 9:15 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits