Page MenuHomePhabricator

Get typical upper module build time under 1 second
Closed, ResolvedPublic



  • p99 and upper: 200ms (min in last 24h), 1min (max in last 24h)

This means that every day, every minute, at least one module build takes 200ms. Note that we often a dozen or more modules in a single HTTP request. Not to mention the startup module, which has to compute many more (though for the common case of file modules we have optimisations in place that bypass the module build).

The highest point is even over a minute.

  • p95: 1.9ms (highest point in last 3 months)

On the other hand, our p95 has not gone above 2ms in over 3 months. This is quite an accomplishment and largely due to our content-based caching for minification and Less compilation. There are many different urls to the same resource essentially (N wikis * N user languages * N skins), and most of these don't vary, so once computed, they're re-used until the resource changes.

However this hasn't changed the fact that Less compilation and other code involved in module build still seems pretty darn slow in case of cache miss.

Over a 24 hour period:


Plenty of modules that are consistently slow (not just peaks).


Graph: Build times for skins_vector_styles

Event Timeline

Krinkle created this task.Jul 29 2016, 7:54 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 29 2016, 7:54 PM
Krinkle renamed this task from Get upper module build time under 1 second to Get typical upper module build time under 1 second.Jul 29 2016, 7:55 PM
Krinkle triaged this task as Medium priority.Aug 1 2016, 8:02 PM
Krinkle moved this task from Inbox to Backlog on the MediaWiki-ResourceLoader board.
Krinkle moved this task from Inbox to Backlog: Small & Maintenance on the Performance-Team board.
Krinkle claimed this task.Dec 7 2016, 6:30 PM
Krinkle lowered the priority of this task from Medium to Low.
Krinkle closed this task as Resolved.May 3 2017, 3:32 AM

Closing in favour of T164248: HTTP responses from app servers sometimes stall for >1s.

Build times have improved by 20-60% over the past year due to numerous improvements in different areas. But the spikes still exist, albeit with lower frequency nowadays.