Maniphest T201930

Track entity page performance
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Aleksey_WMDE
	Aug 14 2018, 2:29 PM

Description

Due to the changes of ItemId formatting on entity page, we need to be sure that we do not degrade performance (very much).

To track the performance we need to have performance test suite and run it every time we change the number of ItemIds rendered using "new code".

Performance analysis is done using jmeter config: https://phabricator.wikimedia.org/diffusion/EWBA/browse/master/jmeter/test_wd_entity_page_performance.jmx
For information how to run jMeter analysis refer to Readme: https://phabricator.wikimedia.org/diffusion/EWBA/browse/master/jmeter/README.md

Details

Subject	Repo	Branch	Lines +/-
EntityParserOutputGenerator, track generation times	mediawiki/extensions/Wikibase	master	+376 -262
ControlledFallbackEntityIdFormatter, track unique value formats	mediawiki/extensions/Wikibase	wmf/1.33.0-wmf.8	+8 -0
ControlledFallbackEntityIdFormatter, track unique value formats	mediawiki/extensions/Wikibase	master	+8 -0
FormatterCache, wrap in hash cache	mediawiki/extensions/Wikibase	master	+21 -2
Record cache misses for "FormatterCache"	mediawiki/extensions/Wikibase	master	+185 -5
Test Wikidata.org entity page performance	mediawiki/extensions/Wikibase	master	+519 -0

Customize query in gerrit

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		Addshore	T201831 Deploy item/property link formatter that uses cache instead of wb_terms DB table
		Resolved		Addshore	T201930 Track entity page performance

Event Timeline

• Aleksey_WMDE created this task.Aug 14 2018, 2:29 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 14 2018, 2:29 PM

• Aleksey_WMDE added a project: Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing).Aug 14 2018, 2:29 PM

summary.csv663 BDownload

aggregate.csv556 BDownload

These are preliminary test results using following data set:

query.csv2 KBDownload

Change 452670 had a related patch set uploaded (by Aleksey Bekh-Ivanov (WMDE); owner: Aleksey Bekh-Ivanov (WMDE)):
[mediawiki/extensions/Wikibase@master] Test Wikidata.org entity page performance

https://gerrit.wikimedia.org/r/452670

gerritbot added a project: Patch-For-Review.Aug 14 2018, 2:36 PM

• Aleksey_WMDE added a subscriber: WMDE-leszek.Aug 14 2018, 2:38 PM

To summarize current results for "Get without cache":

Number of samples: 314
Average response time, ms: 350
Std. deviation, ms: 117
Median response time, ms: 309
90% percentile, ms: 524
Min, ms: 38 Seems wrong, probably hit the cache. Should be at least greater than 100-150
Max, ms: 1002

• Aleksey_WMDE moved this task from To Do to Doing on the Wikidata-Ugly-Cat-Trailblaze (wb_terms trail blazing) board.Aug 14 2018, 4:51 PM

More fine grained results

summary.csv6 KBDownload

aggregate.csv5 KBDownload

Change 452670 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Test Wikidata.org entity page performance

https://gerrit.wikimedia.org/r/452670

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-08-21 (1.32.0-wmf.18)).Aug 17 2018, 8:00 AM

Results after enabling new formatting for Q1 only

summary-Q1.csv6 KBDownload

aggregate-Q1.csv5 KBDownload

Results after enabling new formatting for Q1-Q100

summary-Q100.csv6 KBDownload

aggregate-Q100.csv5 KBDownload

Results after enabling new formatting for Q1-Q10 000

summary-Q10000.csv6 KBDownload

aggregate-Q10000.csv5 KBDownload

WMDE-leszek triaged this task as Medium priority.Sep 22 2018, 5:11 PM

WMDE-leszek edited projects, added Wikidata-Campsite; removed Patch-For-Review.

WMDE-leszek updated the task description. (Show Details)

WMDE-leszek added a subscriber: Addshore.

Lydia_Pintscher added a project: Wikidata.Sep 29 2018, 9:27 PM

Addshore added a parent task: T201831: Deploy item/property link formatter that uses cache instead of wb_terms DB table.Oct 16 2018, 1:31 PM

Addshore added a project: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)).Oct 16 2018, 1:37 PM

Addshore moved this task from incoming to in progress on the Wikidata board.Oct 16 2018, 1:49 PM

Addshore removed a project: Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)).Nov 1 2018, 11:14 AM

Addshore edited projects, added Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)); removed Wikidata-Campsite.Nov 1 2018, 11:16 AM

Ran the numbers yesterday. Will upload results shortly.

Addshore assigned this task to WMDE-leszek.Nov 12 2018, 5:42 PM

Addshore moved this task from To Do (prioritised from top to bottom) to Doing on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.

Addshore mentioned this in T201831: Deploy item/property link formatter that uses cache instead of wb_terms DB table.

Sorry for the blocking you @Addshore.
I've analyzed the results, and noticed some slowdown, as expected. Arguably acceptable. Before starting to discuss those results though I have to mention that I noticed that the last analysis was run on more than 50% smaller data set (number of samples dropped by ~55%). Which most likely means I stopped collecting data too early. For which congrats to me.

Re-running the test now. Will report back here once finished.

Re-run the test. Still quite some requests failing or so. Numbers of data way less off now though.
Numbers attached.

Still not 100% happy to say let's move on, as the test sets not entirely same (in terms of number of requests compared). Seems OK-ish to go, maybe.

summary-Q100k.csv6 KBDownload

aggregate-Q100k.csv5 KBDownload

Change 473626 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] Record cache misses for "FormatterCache"

https://gerrit.wikimedia.org/r/473626

gerritbot added a project: Patch-For-Review.Nov 14 2018, 9:39 PM

I did some more checking of the performance of this code in comparison to what we already have on the mwdebug servers using P7805 (and eventually a modified version with less logging)

The data I have can be found @ https://docs.google.com/spreadsheets/d/11lbFz80tehy_Aw580wJoNVS9f7KRt49-hxy_PzlYL3g/edit?usp=sharing

It seems to suggest that new is slower than old, even while all of the info is in the cache.
This was determined while looking at EntityContent::getParserOutput which should encompass all of the logic and db calls etc for creating the parser output (including any batched label lookup etc)

Some profiling links for the new formatter:

And for the old one:

In the profiling Wikibase\Lib\Formatters\ControlledFallbackEntityIdFormatter::formatEntityId is a good thing to look at

Change 473695 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] FormatterCache, wrap in hash cache

https://gerrit.wikimedia.org/r/473695

Change 473626 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Record cache misses for "FormatterCache"

https://gerrit.wikimedia.org/r/473626

ReleaseTaggerBot added a project: MW-1.33-notes (1.33.0-wmf.6; 2018-11-27).Nov 15 2018, 10:00 AM

Change 473695 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] FormatterCache, wrap in hash cache

https://gerrit.wikimedia.org/r/473695

Change 474169 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] EntityParserOutputGenerator, track generation times

https://gerrit.wikimedia.org/r/474169

I just knocked together https://grafana.wikimedia.org/d/u5wAugyik/wikibase-formattercache and the hit rate looks pretty good in production :)

Addshore claimed this task.Dec 18 2018, 10:36 AM

Restricted Application added a project: User-Addshore. · View Herald TranscriptDec 18 2018, 10:36 AM

Assigning to me so that I can work on the final patch that is still here to do with entity parser output time generation tracking.

Change 480467 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] ControlledFallbackEntityIdFormatter, track unique value formats

https://gerrit.wikimedia.org/r/480467

Change 480474 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@wmf/1.33.0-wmf.8] ControlledFallbackEntityIdFormatter, track unique value formats

https://gerrit.wikimedia.org/r/480474

Change 480467 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] ControlledFallbackEntityIdFormatter, track unique value formats

https://gerrit.wikimedia.org/r/480467

Change 480474 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.33.0-wmf.8] ControlledFallbackEntityIdFormatter, track unique value formats

https://gerrit.wikimedia.org/r/480474

Mentioned in SAL (#wikimedia-operations) [2018-12-18T13:12:12Z] <addshore@deploy1001> Synchronized php-1.33.0-wmf.8/extensions/Wikibase/lib/includes/Formatters/ControlledFallbackEntityIdFormatter.php: T201930 ControlledFallbackEntityIdFormatter, track unique value formats (duration: 00m 46s)

The new tracking indicates a 5 - 15% cache miss rate for the new formatter, which seems pretty good.
Combining this hit rate with what was described in T201930#4748332 this new formatter will, in the long run, work out very nicely for reducing wb_terms load and probably also reducing render time.

ReleaseTaggerBot edited projects, added MW-1.33-notes (1.33.0-wmf.12; 2019-01-08); removed MW-1.33-notes (1.33.0-wmf.6; 2018-11-27).Dec 18 2018, 2:01 PM

Addshore moved this task from Doing to Peer Review on the Wikidata-Campsite (Wikidata-Campsite-Iteration-∞ (On Hold)) board.Dec 19 2018, 3:26 PM