Page MenuHomePhabricator

Track entity page performance
Closed, ResolvedPublic

Description

Due to the changes of ItemId formatting on entity page, we need to be sure that we do not degrade performance (very much).

To track the performance we need to have performance test suite and run it every time we change the number of ItemIds rendered using "new code".

Performance analysis is done using jmeter config: https://phabricator.wikimedia.org/diffusion/EWBA/browse/master/jmeter/test_wd_entity_page_performance.jmx
For information how to run jMeter analysis refer to Readme: https://phabricator.wikimedia.org/diffusion/EWBA/browse/master/jmeter/README.md

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 14 2018, 2:29 PM

These are preliminary test results using following data set:

Change 452670 had a related patch set uploaded (by Aleksey Bekh-Ivanov (WMDE); owner: Aleksey Bekh-Ivanov (WMDE)):
[mediawiki/extensions/Wikibase@master] Test Wikidata.org entity page performance

https://gerrit.wikimedia.org/r/452670

To summarize current results for "Get without cache":

  • Number of samples: 314
  • Average response time, ms: 350
  • Std. deviation, ms: 117
  • Median response time, ms: 309
  • 90% percentile, ms: 524
  • Min, ms: 38 Seems wrong, probably hit the cache. Should be at least greater than 100-150
  • Max, ms: 1002

More fine grained results

Change 452670 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Test Wikidata.org entity page performance

https://gerrit.wikimedia.org/r/452670

Results after enabling new formatting for Q1 only

Results after enabling new formatting for Q1-Q100


Results after enabling new formatting for Q1-Q10 000


WMDE-leszek triaged this task as Normal priority.Sep 22 2018, 5:11 PM
WMDE-leszek edited projects, added Wikidata-Campsite; removed Patch-For-Review.
WMDE-leszek updated the task description. (Show Details)
WMDE-leszek added a subscriber: Addshore.
Addshore moved this task from incoming to in progress on the Wikidata board.Oct 16 2018, 1:49 PM

Ran the numbers yesterday. Will upload results shortly.

Sorry for the blocking you @Addshore.
I've analyzed the results, and noticed some slowdown, as expected. Arguably acceptable. Before starting to discuss those results though I have to mention that I noticed that the last analysis was run on more than 50% smaller data set (number of samples dropped by ~55%). Which most likely means I stopped collecting data too early. For which congrats to me.

Re-running the test now. Will report back here once finished.

Re-run the test. Still quite some requests failing or so. Numbers of data way less off now though.
Numbers attached.

Still not 100% happy to say let's move on, as the test sets not entirely same (in terms of number of requests compared). Seems OK-ish to go, maybe.


Change 473626 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] Record cache misses for "FormatterCache"

https://gerrit.wikimedia.org/r/473626

Addshore added a comment.EditedNov 14 2018, 9:42 PM

I did some more checking of the performance of this code in comparison to what we already have on the mwdebug servers using P7805 (and eventually a modified version with less logging)

The data I have can be found @ https://docs.google.com/spreadsheets/d/11lbFz80tehy_Aw580wJoNVS9f7KRt49-hxy_PzlYL3g/edit?usp=sharing

It seems to suggest that new is slower than old, even while all of the info is in the cache.
This was determined while looking at EntityContent::getParserOutput which should encompass all of the logic and db calls etc for creating the parser output (including any batched label lookup etc)

Some profiling links for the new formatter:

And for the old one:

In the profiling Wikibase\Lib\Formatters\ControlledFallbackEntityIdFormatter::formatEntityId is a good thing to look at

Change 473695 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] FormatterCache, wrap in hash cache

https://gerrit.wikimedia.org/r/473695

Change 473626 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] Record cache misses for "FormatterCache"

https://gerrit.wikimedia.org/r/473626

Change 473695 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] FormatterCache, wrap in hash cache

https://gerrit.wikimedia.org/r/473695

Change 474169 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] EntityParserOutputGenerator, track generation times

https://gerrit.wikimedia.org/r/474169

I just knocked together https://grafana.wikimedia.org/d/u5wAugyik/wikibase-formattercache and the hit rate looks pretty good in production :)

Restricted Application added a project: User-Addshore. · View Herald TranscriptDec 18 2018, 10:36 AM

Assigning to me so that I can work on the final patch that is still here to do with entity parser output time generation tracking.

Change 480467 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@master] ControlledFallbackEntityIdFormatter, track unique value formats

https://gerrit.wikimedia.org/r/480467

Change 480474 had a related patch set uploaded (by Addshore; owner: Addshore):
[mediawiki/extensions/Wikibase@wmf/1.33.0-wmf.8] ControlledFallbackEntityIdFormatter, track unique value formats

https://gerrit.wikimedia.org/r/480474

Change 480467 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] ControlledFallbackEntityIdFormatter, track unique value formats

https://gerrit.wikimedia.org/r/480467

Change 480474 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@wmf/1.33.0-wmf.8] ControlledFallbackEntityIdFormatter, track unique value formats

https://gerrit.wikimedia.org/r/480474

Mentioned in SAL (#wikimedia-operations) [2018-12-18T13:12:12Z] <addshore@deploy1001> Synchronized php-1.33.0-wmf.8/extensions/Wikibase/lib/includes/Formatters/ControlledFallbackEntityIdFormatter.php: T201930 ControlledFallbackEntityIdFormatter, track unique value formats (duration: 00m 46s)

The new tracking indicates a 5 - 15% cache miss rate for the new formatter, which seems pretty good.
Combining this hit rate with what was described in T201930#4748332 this new formatter will, in the long run, work out very nicely for reducing wb_terms load and probably also reducing render time.

Change 474169 merged by jenkins-bot:
[mediawiki/extensions/Wikibase@master] EntityParserOutputGenerator, track generation times

https://gerrit.wikimedia.org/r/474169

Addshore removed Addshore as the assignee of this task.Jan 4 2019, 1:36 PM

Stalled waiting for the deploy next week so that we can make a dashboard.

Graph is there so I guess we can close this?

Addshore closed this task as Resolved.Jan 14 2019, 10:05 AM
Addshore claimed this task.