Page MenuHomePhabricator

Make it easy to compare WebPageTest results on alerts
Closed, ResolvedPublic

Description

The original idea comes from T151197 and @Jdlrobson

but periodically I'm asked to report how mobile performance compares to desktop (for example when we were working on lazy loading images). I think making easier to fire off comparison tests for that purpose would be great - some kind of compare pages script that given multiple URLs reports on differences between them e.g. bytes changed/first paint.

We have good way of comparing run in WebPageTest today, but we should make it more accessible and maybe the result page could be more compelling,

Event Timeline

BTW

I'm asked to report how mobile performance compares to desktop

that is just wrong way of thinking :(

I should clarify that as my choice of words were not great. During the work on lazy loading images it was important to compare desktop with mobile to show the amount of bytes we saved by looking at reduction in image bytes.

Changes on the desktop site are unfortunately much harder but by comparing the experiences we can use data to make that case more strongly.

Krinkle triaged this task as Medium priority.
Krinkle updated the task description. (Show Details)
Krinkle subscribed.
Krinkle renamed this task from Make it easier compare results to Make it easier compare WebPageTest results.Sep 12 2017, 3:28 PM

Two separate issues:

  1. Discovery path from Grafana dashboard to WebPageTest url
  1. Discovery path from two WebPageTest urls to the built-in WebPageTest comparison tool. (Yes, it has one)

This task is about both of these.

The separate idea of being able to compare in-depth two arbitrary WebPageTest runs and/or two timeline profiles and find regressions automatically (e.g. two locally generates ones, or two profiles downloaded from a WPT run). Primarily for the purpose of finding regressions between Jenkins runs on each commit, or when working locally and find a way to automatically fail the build or identify potential regressions. That is tracked as T153069.

Imarlier subscribed.

Create an upstream issue/PR asking for some sort of link or whatever.

Peter renamed this task from Make it easier compare WebPageTest results to Make it easy to compare WebPageTest results on alerts.Jun 6 2019, 9:40 AM

I think we should do this differently: On alerts we should pre-populate a link with runs before/after a alert, so you as a user just have a link to click on that automatically fills in the right values into the compare page.

Gilles claimed this task.

It's much easier to find before/after runs in Grafana now.