Prior art:
- T229836: Track module registry size over time
- Grafana: Startup manifest dashboard
- Grafana: RL per-module dashboard.
- Grafana: RL modules overview dashboard.
- Source: WikimediaMaintenance: blameStartupRegistry.php stats.
We have a per-module breakdown on backend cache-miss build latency in production, and a cache-hit/cache-miss rate, and we have a per-module breakdown of startup allocation cost in bytes for module bundle metadata.
What we don't yet have is a breakdown of the size of the module output itself.
High level approaches to consider:
- Measure on-demand when building a module during a cache-miss in production, from live load.php requests. This would match what we do today for measuring latencies. This makes sense for latencies and cache hit/miss ratios, as those must reflect what actually happens in production in order to be useful, these vary by current database load, memcached effectiveness, and may vary by wiki configuration, skins and user language preference.
I looked into this, and steered away from it because the size measurement would vary too much by request context and user preferences. The exact size for any given user context can be measured directly by developers through a trivial inspection in DevTools or with curl. The dashboard I imagine would be more useful if it gives a stable approximation based on equal comparison. In particular there is variance between modules loaded via stylesheets and via JS responses.
One of the counter-intuitive things about caching is that the more popular something is, the less often you have a cache miss. (For example, Thumbor, our thumbnailing service, is known to be primarily a 404 generator that ocasionally produces an image, because images are strongly cached whereas 404s are revalidated more often)
This means that the size numbers would be disproportionately skewed by rarely requested contexts that may produce larger variants. One way to avoid that would be to only send stats to Graphite if the current request is "canonical", e.g. a stylesheet request for a styles module, with the wikis' default skin/language. This however means modules may never get the right metrics or very irregularly if they are primarily lazy-loaded. It's not uncommon for a styles module to be requested via JS.
- Measure synthetically from a cronjob. This matches what we do today for startup allocation cost because approximating that is relatively expensive (so best deferred), doesn't vary by factors that require live webserver context, and may benefit from being measured at a low but consistent rate from a cronjob so that the size allocation has a predictable measurement frequency.
This would be a bit more code upfront, but would then give a stablecomparison over time, with the drawback that it woud not perfectly capture every variant of a module bundle.
