Per T129564#2170016, we need to add the time to display results to the TestSearchSatisfaction2 schema, so that we can switch over to the new schema on the dashboards. This will measure the time taken to display the results from the user's perspective. That's important to understand the experience users have, even though it does contain things we can't really control (e.g. latency spikes, dodgy connections, and so on). In T129564#2196304, @EBernhardson estimated that this would be only a few hours work.
|mediawiki/extensions/WikimediaEvents : master||search satisfaction: use domInteractive instead of domComplete|
|mediawiki/extensions/WikimediaEvents : master||Add time to display results logging for search satisfaction|
- Mentioned In
- T134282: Fix log error in Search schema: "u'comp_suggest' is not one of ['prefix', 'fulltext', 'prefixmergedwithfulltext']"
T129564: Switch Desktop data collection for dashboards to use TestSearchSatisfaction2 instead of Search schema
- Mentioned Here
- T129564: Switch Desktop data collection for dashboards to use TestSearchSatisfaction2 instead of Search schema
params.msToDisplayResults = window.performance.timing.domInteractive - window.performance.timing.navigationStart;
navigationStart is a good starting point as it'll start counting from the beginning of when a link was clicked, form submitted, or however the navigation started (including redirects etc.)
However, afaik domInteractive does not relate to "what the user sees".
domInteractive means the browser has received the end of the HTML response and parsed the HTML string into a DOM. Paints may happen both before or after that, depending on different factors. I've not yet found any reasonably reliable way to measure visibility timing client-side (aside from "firstPaint", as implemented in Chrome).
You can make a recording in WebPageTest and check at what time code the first result is visible.
Modern browsers are not blocked on downloading and parsing the entire HTML response in order to render and paint elements on the screen. For slow and/or large page views, browsers will often start to render and paint some elements much earlier. At the same time, there's also no reason to assume the browser will have painted anything by the time the DOM is ready. Painting is often blocked on sub resources (e.g. blocking stylesheets).
@Krinkle thanks for your feedback, so I understand that there are no easy/reliable ways to extract this timing information :(
Note that we work mostly on the backend and we don't (for now at least) implement/test new UI features on special search.
So I tend to think that "backend time" is what I worry much and is what we want to compare when running A/B test.
From what I understood domInteractive seems to be a good compromise:
- timing information available when the instrumentation code starts
- relatively close to backend time
- relatively close to what the user sees (SpeciaSearch seems to be simple enough)
On the other hand It may be wrong when we'll start to implement new UI features...