This task aims to collect possible performance improvements that would also speed up our selenium suite. Extracted from T225730 to keep it more focused.
In T225730#10390381, @Daimona wrote:I have generated updated profile for the gate jobs, using Amir's patch above and uploaded them on people.wm.org. See T225730#9033209 for how to visualize them, e.g. https://performance.wikimedia.org/excimer/speedscope/#profileURL=https://people.wikimedia.org/~daimona/T225730-profile-2024-12-09/selenium-wmf-cpu-profile.log.
I am particularly interested in the selenium jobs, now that PHPUnit is run in parallel in most places. I haven't analyzed them in depth for now, but at first glance it does look like we're spending a decent amount of time in load.php, and particularly parsing LESS files. I'm not sure if that is expected, but 10% of the total time seems a lot. I'm also somewhat surprised to see APCUBagOStuff::doGet amongst the functions with the larger self times. At first I thought it might just be that we call it a lot (no way of knowing from a sampled profile), but then it shows that there seems to be a single caller responsible for the majority of the spent time, and that is ExtensionRegistry::loadFromQueue. I'm not sure what to make of it, for the time being.
In T225730#10393477, @thiemowmde wrote:Thanks for the fresh profiles!
- I already spend some time in our (forked) Less.php library and managed to improve it quite a bit, most notably via https://gerrit.wikimedia.org/r/939717. I'm not surprised it's still one of the most expensive things we do at startup. I might have another look.
- I also noticed that Wikimedia-Minify shows up relatively high. This is not surprising as it's also an actual parser that processes every bit of JavaScript we have one character at a time. While it's heavily optimized already there is certainly still something to improve.
- I also wonder if we can change MediaWiki-ResourceLoader's FileContentsHasher so we don't need to actually read so many files just to calculate a hash but can use the file's timestamps (and only the timestamp) instead?
- Why is ConvertibleTimestamp::setTimestamp so expensive? This doesn't seem right.
In T225730#10394368, @Daimona wrote:In T225730#10393477, @thiemowmde wrote:
- I already spend some time in our (forked) Less.php library and managed to improve it quite a bit, most notably via https://gerrit.wikimedia.org/r/939717. I'm not surprised it's still one of the most expensive things we do at startup. I might have another look.
I think it makes sense that parsing LESS would take some time, but I'm surprised by this being 10% of the overall time spent in tests. My main (implicit) question is whether we are parsing the same things over and over again, or if there are otherwise missed caching opportunities. If not, and parsing LESS is just this slow, then I do agree that it would be worth improving its performance.
- I also noticed that Wikimedia-Minify shows up relatively high. This is not surprising as it's also an actual parser that processes every bit of JavaScript we have one character at a time. While it's heavily optimized already there is certainly still something to improve.
Yup, here too I mostly wanted to confirm that we're making good use of caches.
- Why is ConvertibleTimestamp::setTimestamp so expensive? This doesn't seem right.
I suppose it might be called a lot of times; we can't know with Excimer. It's not a trivial setter either, so I could see it taking some time if it's called a lot. Worth looking into this though, to see what the number of calls really is, and if anything can be done about it.
