Page MenuHomePhabricator
Create Task
Maniphest T67333

Wikibase\EntityPerPageTable::getItemsWithoutSitelinks slow query with large LIMIT offset
Closed, ResolvedPublic
Actions

Description

This query form appears in the slow log for wikidatawiki on WMF S5:

SELECT /* Wikibase\EntityPerPageTable::getItemsWithoutSitelinks ? */ epp_entity_id AS entity_id FROM `wb_entity_per_page` LEFT JOIN `wb_items_per_site` ON ((ips_item_id = epp_entity_id)) WHERE (ips_site_page IS NULL) AND epp_entity_type = 'item' ORDER BY epp_page_id DESC LIMIT 1940000, 51;

Offset 1940000... wow! :-)

Can we cap the possible limit offset, or handle pagination differently?

Details

Reference
bz65333

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedLadsgroupT67333 Wikibase\EntityPerPageTable::getItemsWithoutSitelinks slow query with large LIMIT offset
 ResolvedLadsgroupT95685 Drop wb_entity_per_page table
 Resolved• adrianheineT134873 Make EntityRevisionLookup not rely on entity_per_page
 ResolveddanielT140758 Investigate where wb_entity_per_page table is used and needed
 ResolvedLadsgroupT140876 In SqlEntityInfoBuilder, replace the usage of wb_entity_per_page with using the page and redirect tables
 ResolvedLadsgroupT140877 In PropertyInfoTableBuilder, replace the usage of wb_entity_per_page with using the page and redirect tables
 ResolvedLadsgroupT140890 Drop EntityPerPage service
 ResolvedhooT140888 Re-implement listEntities() in EntityPerPageIdPager (and rename to SqlEntityIdPager)
 ResolvedhooT140891 Factor EntityPerPage::getEntitiesWithoutTerm out into it's own service
 ResolvedhooT147638 Introduce a setting for entity types supported by Special:EntitiesWithoutLabel/ Special:EntitiesWithoutDescription
 ResolvedhooT140892 Factor EntityPerPage::getEntitiesWithoutSitelinks out into it's own service
 DeclinedNoneT114903 Migrate wb_terms to using prefixed entity IDs instead of numeric IDs
 ResolvedLydia_PintscherT159718 Evaluate how to best add a column for full entity ID to wb_terms without affecting wikidata.org users
 ResolvedNoneT159851 Add a column for full entity ID to wb_terms table
 ResolvedWMDE-leszekT168708 Populating full entity column in wb_terms table when running MW update.php should be fast and run only once
 DeclinedNoneT169720 Remove column storing numeric entity ID in wb_terms once it has become redundant
 DeclinedNoneT169721 Remove term_entity_id during database schema update when term_full_entity_id is used instead
 DeclinedNoneT169722 Remove term_entity_id column from wb_terms table when no longer needed on WMF cluster
 ResolvedLadsgroupT188995 Clean up the code base for term_entity_id
 ResolvedaudeT162533 Script for populating term_full_entity_id column in wb_terms table
 ResolvedMarosteguiT162539 Deploy schema change for adding term_full_entity_id column to wb_terms table
 ResolvedLadsgroupT162673 Adjust code reading from wb_terms table to use full entity ID
 ResolvedAndrewT167114 Open view for term_full_entity_id in wb_terms table in labs
 ResolvedLadsgroupT171460 Populate term_full_entity_id on www.wikidata.org
 ResolvedaudeT171461 Populate term_full_entity_id on test.wikidata.org
 ResolvedLadsgroupT167229 Change configuration of Wikidata in production to write term_full_entity_id
 ResolvedLadsgroupT165197 Change configuration of test Wikidata to write term_full_entity_id
 ResolvedLadsgroupT172776 Property labels missing on some items
 ResolvedMarosteguiT177601 Deploy dropping wb_entity_per_page table
 ResolvedAndrewT178661 Drop wb_entity_per_page views in Wiki Replicas

Event Timeline

• bzimport raised the priority of this task from to Medium.Nov 22 2014, 3:11 AM
• bzimport added projects: MediaWiki-extensions-WikibaseRepository, patch-welcome.
• bzimport set Reference to bz65333.
• bzimport added a subscriber: Unknown Object (MLST).
• Springle created this task.May 15 2014, 6:30 AM
Lydia_Pintscher added a project: Wikidata.Dec 1 2014, 2:43 PM
Lydia_Pintscher removed a subscriber: Unknown Object (MLST).
Lydia_Pintscher removed a subscriber: Unknown Object (MLST).
Lydia_Pintscher added a subscriber: daniel.Mar 17 2015, 12:55 PM
Ricordisamoa subscribed.Aug 22 2015, 10:19 AM
SELECT page_id, page_title FROM page_props INNER JOIN page ON page_id = pp_page WHERE pp_propname = 'wb-sitelinks' AND pp_sortkey = 0;
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 22 2015, 10:19 AM
JanZerebecki added a subtask: T95685: Drop wb_entity_per_page table.Sep 10 2015, 6:31 PM
JanZerebecki removed a project: patch-welcome.
JanZerebecki set Security to None.
JanZerebecki moved this task from incoming to hold on the Wikidata board.Sep 10 2015, 6:34 PM
matej_suchanek added a project: Performance Issue.Jul 20 2016, 7:55 AM
matej_suchanek updated the task description. (Show Details)
Lydia_Pintscher closed subtask T95685: Drop wb_entity_per_page table as Resolved.Oct 24 2017, 3:50 PM
Addshore closed this task as Resolved.Jun 14 2019, 2:44 PM
Addshore assigned this task to Ladsgroup.
Addshore subscribed.

The entity per page table is gone (see T95685) so this task / issue can be resolved.

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptJun 14 2019, 2:44 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits