Page MenuHomePhabricator

Benchmarker should support per-iteration non-counted setup
Closed, ResolvedPublic

Description

In https://gerrit.wikimedia.org/r/c/mediawiki/core/+/452278 I mistakenly assumed that 'setup' would get called for each iteration. My goal there was to randomize the input in each loop so I could avoid any caches.

Event Timeline

Legoktm created this task.Aug 15 2018, 9:40 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 15 2018, 9:40 AM

Most benchmarkers I've used do not have this option. It might be worth considering whether the iterated code can be changed in a way to produce the expected results "within" the current model. I don't have many ideas off-hand, but one thing might be to have a way to disable the (static?) caches. Would that suffice?

Either way, I'd support addition of an option to perform setup per iteration. Perhaps a callable option, setupEach?

I don't have many ideas off-hand, but one thing might be to have a way to disable the (static?) caches. Would that suffice?

That would be ideal, and could be done with injection, but it's not really possible to disable the Title static cache, since it's not injectable.

Change 453936 had a related patch set uploaded (by Legoktm; owner: Legoktm):
[mediawiki/core@master] benchmarker: Implement setupEach for per-iteration setup

https://gerrit.wikimedia.org/r/453936

Change 453936 merged by jenkins-bot:
[mediawiki/core@master] benchmarker: Implement setupEach for per-iteration setup

https://gerrit.wikimedia.org/r/453936

Legoktm closed this task as Resolved.Aug 21 2018, 7:16 AM