Page MenuHomePhabricator
Create Task
Maniphest T269960

Schema properties in client code loads the whole item in every page view
Closed, ResolvedPublic
Actions

Assigned To
Ladsgroup
Authored By
Ladsgroup
Dec 11 2020, 7:34 PM
Tags
Referenced Files
F33945418: image.png
Dec 14 2020, 10:30 AM
F33945415: image.png
Dec 14 2020, 10:30 AM
F33945420: image.png
Dec 14 2020, 10:30 AM
Subscribers
Addshore
Aklapper
darthmon_wmde
Jdforrester-WMF
karapayneWMDE
Ladsgroup
Lucas_Werkmeister_WMDE
View All 11 Subscribers
Tokens
"Mountain of Wealth" token, awarded by wkandek."Party Time" token, awarded by phuedx."Pirate Logo" token, awarded by Addshore."Burninate" token, awarded by Jdforrester-WMF.

Description

The code added as part of T198946: Add Schema property 'sameAs' pointing to Wikidata entries (the Go Fish SEO work) in I02bc43f16cc53. Loads the whole item of in client wikis (like English Wikipedia) in every page view request (like opening article of Alan Turing makes it load the whole item of Q2751). It even bypasses ParserCache (Tested using XHGui). FlameGraphs say it consumes 13% resources of all requests involving Wikibase (most requests) and I'm pretty sure it puts a heavy load on memcached and ES.

The offending code is in Wikibase\Client\Hooks\SkinAfterBottomScriptsHandler::createSchemaElement (being called from Wikibase\Client\ClientHooks::onSkinAfterBottomScripts)

Details

SubjectRepoBranchLines +/-
Avoid loading the whole entity when it only needs description.mediawiki/extensions/MobileFrontendmaster+13 -36
Avoid loading the whole entity when it only needs description.mediawiki/extensions/MobileFrontendwmf/1.37.0-wmf.9+13 -36
Avoid loading the whole item in every client page viewmediawiki/extensions/Wikibasewmf/1.36.0-wmf.21+21 -42
Avoid loading the whole item in every client page viewmediawiki/extensions/Wikibasemaster+21 -42
Customize query in gerrit

Related Objects
Search...

Event Timeline

Ladsgroup created this task.Dec 11 2020, 7:34 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 11 2020, 7:34 PM
Ladsgroup mentioned this in T198946: Add Schema property 'sameAs' pointing to Wikidata entries.Dec 11 2020, 7:35 PM
Krinkle added a project: Performance-Team (Radar).Dec 11 2020, 7:35 PM
gerritbot added a comment.Dec 11 2020, 8:17 PM

Change 648338 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@master] Avoid loading the whole item in every client page view

https://gerrit.wikimedia.org/r/648338

gerritbot added a project: Patch-For-Review.Dec 11 2020, 8:17 PM
gerritbot added a comment.Dec 11 2020, 11:36 PM

Change 648338 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Avoid loading the whole item in every client page view

https://gerrit.wikimedia.org/r/648338

ReleaseTaggerBot added a project: MW-1.36-notes (1.36.0-wmf.22; 2020-12-15).Dec 12 2020, 12:01 AM
Maintenance_bot removed a project: Patch-For-Review.Dec 12 2020, 12:10 AM
Jdforrester-WMF awarded a token.Dec 12 2020, 7:06 PM
Jdforrester-WMF subscribed.
Addshore awarded a token.Dec 12 2020, 9:12 PM
gerritbot added a comment.Dec 14 2020, 8:55 AM

Change 648283 had a related patch set uploaded (by Ladsgroup; owner: Ladsgroup):
[mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.21] Avoid loading the whole item in every client page view

https://gerrit.wikimedia.org/r/648283

gerritbot added a project: Patch-For-Review.Dec 14 2020, 8:55 AM
gerritbot added a comment.Dec 14 2020, 9:53 AM

Change 648283 merged by Ladsgroup:
[mediawiki/extensions/Wikibase@wmf/1.36.0-wmf.21] Avoid loading the whole item in every client page view

https://gerrit.wikimedia.org/r/648283

ReleaseTaggerBot edited projects, added MW-1.36-notes (1.36.0-wmf.21; 2020-12-08); removed MW-1.36-notes (1.36.0-wmf.22; 2020-12-15).Dec 14 2020, 10:00 AM
Maintenance_bot removed a project: Patch-For-Review.Dec 14 2020, 10:10 AM
Stashbot added a comment.Dec 14 2020, 10:12 AM

Mentioned in SAL (#wikimedia-operations) [2020-12-14T10:12:52Z] <ladsgroup@deploy1001> Synchronized php-1.36.0-wmf.21/extensions/Wikibase/client/includes: [[gerrit:648283|Avoid loading the whole item in every client page view (T269960)]] (duration: 00m 25s)

Ladsgroup added a comment.Dec 14 2020, 10:30 AM

https://grafana.wikimedia.org/d/000000316/memcache?viewPanel=22&orgId=1&from=now-1h&to=now

This basically shows a 100MB/s drop in memcache traffic

image.png (913×1 px, 116 KB)

Also, a 2% increase in ratio of requests served below 250ms and 30ms drop in 95th percentile

image.png (913×1 px, 98 KB)

image.png (913×1 px, 108 KB)

Ladsgroup closed this task as Resolved.Dec 14 2020, 11:14 AM
Ladsgroup claimed this task.

This is done I assume?

Restricted Application added a project: User-Ladsgroup. · View Herald TranscriptDec 14 2020, 11:14 AM
Maintenance_bot moved this task from Incoming to Done on the User-Ladsgroup board.Dec 14 2020, 11:15 AM
phuedx awarded a token.Dec 14 2020, 4:50 PM
• wkandek awarded a token.Dec 16 2020, 6:26 PM
• wkandek subscribed.
Ladsgroup added a comment.Jun 21 2021, 11:24 AM

This is happening in MobileFrontend as well....

Ladsgroup reopened this task as Open.Jun 21 2021, 11:26 AM

I fix this today.

Michael subscribed.Jun 21 2021, 11:38 AM
gerritbot added a comment.Jun 21 2021, 11:41 AM

Change 700531 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):

[mediawiki/extensions/MobileFrontend@master] Avoid loading the whole entity when it only needs description.

https://gerrit.wikimedia.org/r/700531

gerritbot added a project: Patch-For-Review.Jun 21 2021, 11:42 AM
gerritbot added a comment.Jun 21 2021, 12:59 PM

Change 700344 had a related patch set uploaded (by Ladsgroup; author: Ladsgroup):

[mediawiki/extensions/MobileFrontend@wmf/1.37.0-wmf.9] Avoid loading the whole entity when it only needs description.

https://gerrit.wikimedia.org/r/700344

gerritbot added a comment.Jun 21 2021, 1:23 PM

Change 700344 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@wmf/1.37.0-wmf.9] Avoid loading the whole entity when it only needs description.

https://gerrit.wikimedia.org/r/700344

gerritbot added a comment.Jun 21 2021, 1:23 PM

Change 700531 merged by jenkins-bot:

[mediawiki/extensions/MobileFrontend@master] Avoid loading the whole entity when it only needs description.

https://gerrit.wikimedia.org/r/700531

Stashbot added a comment.Jun 21 2021, 1:28 PM

Mentioned in SAL (#wikimedia-operations) [2021-06-21T13:28:58Z] <ladsgroup@deploy1002> Synchronized php-1.37.0-wmf.9/extensions/MobileFrontend/includes/ExtMobileFrontend.php: Backport: [[gerrit:700344|Avoid loading the whole entity when it only needs description. (T269960)]] (duration: 00m 58s)

karapayneWMDE subscribed.Jun 21 2021, 1:47 PM
ReleaseTaggerBot added a project: MW-1.37-notes (1.37.0-wmf.9; 2021-06-07).Jun 21 2021, 2:00 PM
Maintenance_bot removed a project: Patch-For-Review.Jun 21 2021, 2:10 PM
Krinkle moved this task from Limbo to Watching on the Performance-Team (Radar) board.Jun 28 2021, 6:32 PM
Ladsgroup closed this task as Resolved.Jun 28 2021, 6:38 PM

This is done. Unfortunately (or fortantely) the second issue didn't have much performance issues given the fact that short description from wikidata is enabled on some wikis and it's bound to ParserCache (unlike the first issue).

Krinkle added a parent task: T277788: Save Timing improvements (2021-2022).Dec 16 2021, 7:06 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL