I reviewed multiple flame graphs where the entry point is index.php and I found one low-hanging fruit which we already solved and most likely improved the overall execution by ~5%. But it's difficult to spot thigs that could be improved with small fixes. The execution overall is pretty slow to the amount of running parts in our system and to improve the overall time most likely we should simplify things, and reduce the amount of extensions/hooks we use for common requests.

One obvious finding is that we should check the profiler output on a quarterly basis to verify the health of the system. Thanks to this ticket I noticed one trivial mistake which was affecting all Vector2022 requests. With little bit of time and practice - it should be easy to skim through the profiler report to find outstanding items.

Currently, the timing of our request is:

• ~15% of time takes initializing the environment (WebStart)
• 40~50% takes fetching/loading/parsing article
• 30~40% takes rendering
  • LightAndCandy - the mustache renderer takes only ~2-3% of the render time.
  • most of the rendering time is spent on extensions ( fetching notifications, handling Messages (UI texts), loading data from Wikibase)

One thing that I noticed is that our rendering still has to process/fetch lots of data. When during execution - "we have article - let's render it" -> it's just the beginning of another heavy lifting. Once we start rendering, we call other extensions to inject data. SkinMustache::generateHTML feels like it's just a renderer -> but to get the HTML we need to fetch plenty of data which makes execution confusing. It also prevents us from moving more focus into frontend-ready solutions as before we don't start rendering the HTML - we don't have everything that we need.

I would advocate working a little bit around SkinTemplate::outputPage and thinking about better architecture on how to render pages. At first glance, 40% of rendering time seems excessive, but after looking into details we do plenty of heavy lifting - that shouldn't be a part of renderer flow. Furthermore - rendering mixed with business logic will cause plenty of challenges if we decide to push more responsibilities to the browser.

Overall stats for components (some could include others, they add to over 100%) on the Article page

• ~20% DB Queries
• ~10% handling Message (including fetching them from cache)
• ~35% Hooks
• ~8% Discussion Tools extension
• ~9% Wikibase

Profiler files for possible future analysis:

But for best results please use the current stats available at: https://performance.wikimedia.org/php-profiling/ and https://grafana.wikimedia.org/d/rLMucuf4k/mediawiki-entrypoint-profiling?orgId=1

In T350593#9359078, @pmiazga wrote:

Profiler files for possible future analysis:

profiler_featured_article.json569 KBDownload

profiler_special_blank.json391 KBDownload

profiler_special_userlogin.json267 KBDownload

I'm viewing these by downloading the files and drag-n-drop'ing them into
https://performance.wikimedia.org/excimer/speedscope/

I notice that each of these has ~40% in ProfilerXhprof. I think that means you accidentally enabled "Inline profile" or "XHGui" at the same time as "Excimer UI". This tends to cause a 100-200% slowdown, as well as heavily skew time towards small functions due to constant function call overhead. It's important to profile with Tideways-Xhprof turned off, and only use Excimer.

Links to profiler/flamegrapsh:
Special:BlankPage - https://performance.wikimedia.org/excimer/profile/dd2f547999078ddb
Featured Article - Interstate_40_in_Tennnessee - https://performance.wikimedia.org/excimer/profile/17c43b402fb432bb
Special:History for featured article - https://performance.wikimedia.org/excimer/profile/e4eb9b4ac30c4654.
Special:Login with Excimer only - https://performance.wikimedia.org/excimer/profile/33bd9baa56e1ddbe
Special:Login with xhprof data: https://performance.wikimedia.org/excimer/profile/eeba2c8d7eb63c29