Page MenuHomePhabricator

Add time to display results to logging in Schema:TestSearchSatisfaction2
Closed, ResolvedPublic

Description

Per T129564#2170016, we need to add the time to display results to the TestSearchSatisfaction2 schema, so that we can switch over to the new schema on the dashboards. This will measure the time taken to display the results from the user's perspective. That's important to understand the experience users have, even though it does contain things we can't really control (e.g. latency spikes, dodgy connections, and so on). In T129564#2196304, @EBernhardson estimated that this would be only a few hours work.

Details

Related Gerrit Patches:
mediawiki/extensions/WikimediaEvents : mastersearch satisfaction: use domInteractive instead of domComplete
mediawiki/extensions/WikimediaEvents : masterAdd time to display results logging for search satisfaction

Event Timeline

Deskana created this task.Apr 12 2016, 9:54 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 12 2016, 9:54 PM
Deskana renamed this task from Add time to display results to Schema:TestSearchSatisfaction2 to Add time to display results to logging in Schema:TestSearchSatisfaction2.Apr 12 2016, 9:54 PM
Deskana triaged this task as Medium priority.
Deskana moved this task from needs triage to Up Next on the Discovery-Search board.
Restricted Application added a subscriber: TerraCodes. · View Herald TranscriptApr 19 2016, 8:06 PM

Change 290268 had a related patch set uploaded (by EBernhardson):
Add time to display results logging for search satisfaction

https://gerrit.wikimedia.org/r/290268

Change 290268 merged by jenkins-bot:
Add time to display results logging for search satisfaction

https://gerrit.wikimedia.org/r/290268

debt added a subscriber: debt.Jun 14 2016, 8:14 PM

Hi @EBernhardson - is this update complete?

Thanks!

Change 296640 had a related patch set uploaded (by EBernhardson):
search satisfaction: use domInteractive instead of domComplete

https://gerrit.wikimedia.org/r/296640

Change 296640 merged by jenkins-bot:
search satisfaction: use domInteractive instead of domComplete

https://gerrit.wikimedia.org/r/296640

Krinkle added a subscriber: Krinkle.EditedJul 1 2016, 2:31 AM
@Deskana wrote

This will measure the time taken to display the results from the user's perspective.

modules/ext.wikimediaEvents.searchSatisfaction.js
params.msToDisplayResults = window.performance.timing.domInteractive - window.performance.timing.navigationStart;

navigationStart is a good starting point as it'll start counting from the beginning of when a link was clicked, form submitted, or however the navigation started (including redirects etc.)

However, afaik domInteractive does not relate to "what the user sees".

domInteractive means the browser has received the end of the HTML response and parsed the HTML string into a DOM. Paints may happen both before or after that, depending on different factors. I've not yet found any reasonably reliable way to measure visibility timing client-side (aside from "firstPaint", as implemented in Chrome).

You can make a recording in WebPageTest and check at what time code the first result is visible.

Modern browsers are not blocked on downloading and parsing the entire HTML response in order to render and paint elements on the screen. For slow and/or large page views, browsers will often start to render and paint some elements much earlier. At the same time, there's also no reason to assume the browser will have painted anything by the time the DOM is ready. Painting is often blocked on sub resources (e.g. blocking stylesheets).

See also https://grafana.wikimedia.org/dashboard/db/performance-metrics#navtiming-stack

dcausse added a subscriber: dcausse.Jul 8 2016, 2:37 PM

@Krinkle thanks for your feedback, so I understand that there are no easy/reliable ways to extract this timing information :(

Note that we work mostly on the backend and we don't (for now at least) implement/test new UI features on special search.
So I tend to think that "backend time" is what I worry much and is what we want to compare when running A/B test.
From what I understood domInteractive seems to be a good compromise:

  • timing information available when the instrumentation code starts
  • relatively close to backend time
  • relatively close to what the user sees (SpeciaSearch seems to be simple enough)

On the other hand It may be wrong when we'll start to implement new UI features...

debt closed this task as Resolved.Jul 21 2016, 6:12 PM