Page MenuHomePhabricator
Create Task
Maniphest T384455

Increase entityAccessLimit for WikibaseClient wikis
Closed, ResolvedPublic
Actions

Description

Increase entityAccessLimit for WikibaseClient wikis

The current default is 400. (The reason for this number is because back then in 2015, the highest number of entity access for all wikis is just a little below 400). However, as of now, there are more pages that requires a higher entityAccessLimit to generate the page that they wanted. Maybe we could increase it in incremental phase and check for it's performance issue after every increment. According to the long tail graph on how the number of entities of the pages in Wikipedia is accessed, even a small amount of increased in the entityAccessLimit (e.g. from 400 to 500) can have a huge impact.

Story

We need to increase the entityAccessLimit from RestrictedEntityLookup in order for WikibaseClient users to access more different entities.

Acceptance Criteria

Increase the entityAccessLimit so that at least some pages can be able to access more entities from WD and the amount of "Too many Wikidata entities accessed" error can be reduced.

Resources

2015 patch for the current 400 entityAccessLimit
2015 usage tracking

Details

SubjectRepoBranchLines +/-
Increase entityAccessLimit from 400 to 500 for all wikis except commons.operations/mediawiki-configmaster+3 -8
Customize query in gerrit

Related Objects
Search...

Event Timeline

seanleong-WMDE created this task.Jan 22 2025, 11:34 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 22 2025, 11:34 AM
seanleong-WMDE moved this task from Kanban Board to Backlog on the Wikidata Integration in Wikimedia projects board.Jan 22 2025, 11:35 AM
seanleong-WMDE edited projects, added Wikidata Integration in Wikimedia projects; removed Wikidata Integration in Wikimedia projects (Kanban Board).
seanleong-WMDE added a parent task: T381098: Investigate Optimization of Wikidata Entity Usage Limit on Wikipedia Article Pages.Jan 22 2025, 3:34 PM
seanleong-WMDE updated the task description. (Show Details)Jan 23 2025, 6:47 AM
seanleong-WMDE updated the task description. (Show Details)Jan 23 2025, 6:50 AM
seanleong-WMDE updated the task description. (Show Details)Jan 23 2025, 8:02 AM
seanleong-WMDE added a project: Performance-Team.
thiemowmde edited projects, added SRE, Wikimedia-Site-requests; removed Performance-Team.Feb 4 2025, 8:52 AM
thiemowmde updated the task description. (Show Details)
thiemowmde subscribed.
Marostegui added a project: serviceops.Feb 5 2025, 12:06 PM
Marostegui subscribed.

I am tagging serviceops here to see if this is something they can help with.

thiemowmde added subscribers: Ladsgroup, larissagaulia.Mar 19 2025, 10:44 AM

The Wikidata Integration in Wikimedia projects team essentially does have two questions here:

  • We plan to increase the limit from 400 to 500 entities per page. We assume this will gradually increase the load on the primary entity store on the wikidata.org server and possibly the load on secondary stores like the terms tables. While we don't expect any new problems to arrise we are interested if anyone from the SRE teams does have concerns?
  • We want to let the responsible SRE team(s) know in advance that and when this change will happen so we can quickly react in case it is necessary. Undoing the config change will be no problem, when needed.
Ladsgroup added a comment.Mar 19 2025, 3:30 PM

Hii, from DBA point of view, I ask that please hold off bumping the number in commons (if anything, reduce it even, it's quite big there). In most other wikis, it should be fine. In the long-term we should move wbc_ table to x1 but that doesn't block this task

seanleong-WMDE added a comment.Fri, Mar 21, 10:45 AM

Hii, got it! We will create a patch to bump the limit for other wikis to 500 and only exclude the commons for now. Thanks, @Ladsgroup!

gerritbot added a comment.Mon, Mar 24, 5:19 AM

Change #1130346 had a related patch set uploaded (by Seanleong-wmde; author: Seanleong-wmde):

[operations/mediawiki-config@master] Increase entityAccessLimit from 400 to 500 for all wikis except commons.

https://gerrit.wikimedia.org/r/1130346

gerritbot added a project: Patch-For-Review.Mon, Mar 24, 5:19 AM
seanleong-WMDE claimed this task.Mon, Mar 31, 8:37 AM
seanleong-WMDE moved this task from Backlog to Kanban Board on the Wikidata Integration in Wikimedia projects board.
seanleong-WMDE edited projects, added Wikidata Integration in Wikimedia projects (Kanban Board); removed Wikidata Integration in Wikimedia projects.
jijiki removed a project: serviceops.Wed, Apr 2, 11:06 AM
jijiki subscribed.

removing serviceops, please re-add is there is something we can help with

gerritbot added a comment.Wed, Apr 2, 12:10 PM

Change #1133317 had a related patch set uploaded (by Seanleong-wmde; author: Seanleong-wmde):

[operations/mediawiki-config@master] Increase entityAccessLimit from 400 to 500 for all wikis except commons.

https://gerrit.wikimedia.org/r/1133317

gerritbot added a comment.Wed, Apr 2, 1:53 PM

Change #1133317 had a related patch set uploaded (by Seanleong-wmde; author: Seanleong-wmde):

[operations/mediawiki-config@master] Increase entityAccessLimit from 400 to 500 for all wikis except commons.

https://gerrit.wikimedia.org/r/1133317

JoelyRooke-WMDE moved this task from In Progress to In Review on the Wikidata Integration in Wikimedia projects (Kanban Board) board.Thu, Apr 3, 7:26 AM
gerritbot added a comment.Tue, Apr 8, 1:22 PM

Change #1133317 merged by jenkins-bot:

[operations/mediawiki-config@master] Increase entityAccessLimit from 400 to 500 for all wikis except commons.

https://gerrit.wikimedia.org/r/1133317

Stashbot mentioned this in T389429: Investigate whether it’s intentional / correct that default CirrusSearch setups run cirrusSearchElasticaWrite as separate jobs.Tue, Apr 8, 1:22 PM

Mentioned in SAL (#wikimedia-operations) [2025-04-08T13:22:49Z] <lucaswerkmeister-wmde@deploy1003> Started scap sync-world: Backport for [[gerrit:1133317|Increase entityAccessLimit from 400 to 500 for all wikis except commons. (T384455)]], [[gerrit:1134064|Remove unused config vars (T389429)]], [[gerrit:1134691|Fix EntitySchema propertyType on Test Wikidata (T371196)]]

Stashbot mentioned this in T371196: The EntitySchema type URI is missing from the Wikibase ontology.Tue, Apr 8, 1:22 PM
Stashbot added a comment.Tue, Apr 8, 1:30 PM

Mentioned in SAL (#wikimedia-operations) [2025-04-08T13:30:20Z] <lucaswerkmeister-wmde@deploy1003> lucaswerkmeister-wmde, ebernhardson, seanleong-wmde: Backport for [[gerrit:1133317|Increase entityAccessLimit from 400 to 500 for all wikis except commons. (T384455)]], [[gerrit:1134064|Remove unused config vars (T389429)]], [[gerrit:1134691|Fix EntitySchema propertyType on Test Wikidata (T371196)]] synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)

Stashbot added a comment.Tue, Apr 8, 1:38 PM

Mentioned in SAL (#wikimedia-operations) [2025-04-08T13:38:19Z] <lucaswerkmeister-wmde@deploy1003> Finished scap sync-world: Backport for [[gerrit:1133317|Increase entityAccessLimit from 400 to 500 for all wikis except commons. (T384455)]], [[gerrit:1134064|Remove unused config vars (T389429)]], [[gerrit:1134691|Fix EntitySchema propertyType on Test Wikidata (T371196)]] (duration: 15m 30s)

seanleong-WMDE moved this task from In Review to Done ( set to "resolved" after prod / comm / ac testing) on the Wikidata Integration in Wikimedia projects (Kanban Board) board.Wed, Apr 9, 10:45 AM
seanleong-WMDE closed this task as Resolved.Wed, Apr 9, 10:50 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits