Page MenuHomePhabricator

Evaluate phpbench for benchmarking and guarding against performance regressions
Open, Needs TriagePublic

Description

From the project page:

PHPBench is a benchmark runner for PHP analogous to PHPUnit but for performance rather than correctness.

https://github.com/phpbench/phpbench

I'm curious how/if it could fit into our existing profiling/benchmarking tools. It could be interesting to use it in CI to guard against performance regressions.

Event Timeline

Change 722594 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] Add phpbench test for LinkRecommendationFilter

https://gerrit.wikimedia.org/r/722594

Change 726681 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/quibble@master] Add support for phpbench

https://gerrit.wikimedia.org/r/726681

Change 726681 merged by jenkins-bot:

[integration/quibble@master] Add support for phpbench

https://gerrit.wikimedia.org/r/726681

Change 726880 had a related patch set uploaded (by Jforrester; author: Kosta Harlan):

[integration/config@master] jjb: Bump Quibble jobs to 1.1.0

https://gerrit.wikimedia.org/r/726880

Mentioned in SAL (#wikimedia-releng) [2021-10-07T13:04:47Z] <hashar> Updating Jenkins jobs for Quibble 1.1.0 # T291549

Change 726880 merged by jenkins-bot:

[integration/config@master] jjb: Bump almost all Quibble jobs to 1.1.0

https://gerrit.wikimedia.org/r/726880

Change 727429 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/core@master] [WIP] Add phpbench to dev requirements

https://gerrit.wikimedia.org/r/727429

Change 727443 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[integration/quibble@master] phpbench: Run composer install first

https://gerrit.wikimedia.org/r/727443

Change 726653 had a related patch set uploaded (by Kosta Harlan; author: Kosta Harlan):

[mediawiki/extensions/GrowthExperiments@master] Add composer entry for phpbench

https://gerrit.wikimedia.org/r/726653

Change 727443 merged by jenkins-bot:

[integration/quibble@master] phpbench: Run composer install first

https://gerrit.wikimedia.org/r/727443

dpifke added a subscriber: dpifke.

If this experiment turns out to be useful, Performance Team would definitely like to schedule some time to figure out how it fits in with existing tools.

Thanks @dpifke. The patches attached here are to facilitate experimentation (it helps to have CI be able to run your patch if you want to play around with the tool); I'm drafting a blog post that explains the tool and how to use it but doesn't make any suggestions about best practices or patterns to use around what type of code to test, types of assertions to make, etc. If you'd like to review the post or chat (here in phab or elsewhere) before I send that around, I'd be happy to. I was thinking that once the explanatory post is available, folks could experiment with building on top of the core patch or make their own extension patches, then we could have some evaluation to decide when/if/how to document and promote this as something in our toolkit. Does that sound OK?

Thanks @dpifke. The patches attached here are to facilitate experimentation (it helps to have CI be able to run your patch if you want to play around with the tool); I'm drafting a blog post that explains the tool and how to use it but doesn't make any suggestions about best practices or patterns to use around what type of code to test, types of assertions to make, etc. If you'd like to review the post or chat (here in phab or elsewhere) before I send that around, I'd be happy to. I was thinking that once the explanatory post is available, folks could experiment with building on top of the core patch or make their own extension patches, then we could have some evaluation to decide when/if/how to document and promote this as something in our toolkit. Does that sound OK?

https://phabricator.wikimedia.org/phame/post/view/257/benchmarking_mediawiki_with_phpbench/ if you'd like to review / comment on it (cc @Krinkle)

I get a permission error trying to view the linked draft.

I get a permission error trying to view the linked draft.

I can't figure out how to share the phab link so here's the PDF of that post.