Page MenuHomePhabricator

Add simple tracking for loading time of community configuration forms
Open, Needs TriagePublic

Description

To ensure we don't let our editors wait too long while opening a CommunityConfiguration edit form, we should track the time until that is available.

Acceptance criteria:

  • the time until the form is ready is tracked per provider
  • there exists a panel on Grafana showing percentiles

Open questions:

  • what should be our start time? I think the mwStartup performanceMark makes sense and removes some of the things that are not under our control anyway
  • which percentiles do we want to show on Grafana? 50%, 95%, 99.5%?
  • what exactly should the metric name be?
    • probably the Grafana board should a variable for the provider?
    • do we want to split by wiki?

Notes:

  • One other candidate for a metric to track in practice, Largest Contentful Paint (LCP), turns out to not be a great choice, because it does not seem to actually identify the form as the "largest image or text block" and instead only measures the time for the server-rendered description (<p>-element) that sits above the form.
  • "the form is ready" would probably be the end of the IIFE in init.js, after the createApp method returned
  • using mw.track.timing to record this
  • if we use the mwStartup mark, then performance.measure('CommunityConfigurationEditFormLoaded', 'mwStartup').duration should give us time we want to track

Event Timeline

Is there any recommended tool for getting performance metrics in CI? Thinking of something similar to lighthouse. In the past it was discarded but the scope was only accessibility not performance (T301790). I guess we're looking into something that plays well with selenium tests (or maybe cypress?).

Is there any recommended tool for getting performance metrics in CI? Thinking of something similar to lighthouse. In the past it was discarded but the scope was only accessibility not performance (T301790). I guess we're looking into something that plays well with selenium tests (or maybe cypress?).

There is https://www.npmjs.com/package/@cypress-audit/lighthouse, but I'm not sure how much that gets us. Which specific endpoint do you have in mind? As mentioned in the description, LCP does not seem to measure what we want.

On the other hand, if the use performance.mark('some-identifier') at the end of init.js, then that should be easily accessible in browser tests. Though, I would advise against actually making assertions based on that. This would seem to either require a limit so high to be meaningless, or be a sure-fire way to get flaky tests. I don't think browser tests are optimized for consistent performance. However, I could see us writing to Graphite from CI with a specialized key. That would at least allow us to maybe spot some trends.