Page MenuHomePhabricator

Change WebPageTest dashboard for Wikidata and reading to use Browsertime data
Closed, ResolvedPublic

Description

We have enough data to do to the switch, let me prepare that and talk to the teams.

Event Timeline

The Wikidata dashboard has two new rows with the new data source mimic the old setup.

I've changed the data sources for the https://grafana.wikimedia.org/d/000000205/reading-web-performance dashboard.

I've changed so we use the data from our Browsertime tests instead so we can close down WebPageTest. The reason is that the open source version of WebPageTest hasn't been updated in years and we just had luck that it hasn't been broken (it actually has been broken but I've made some small fixes through the years).

There's a couple of changes but first, wow, I hadn't look at the reading web performance dashboard in a while and I really like the description text that has been added to each section! It's really useful, I'm gonna add the same to some of our dashboards, thanks for the inspiration!

The data in the dashboard should almost be the same, some metrics will differ. There are two reasons for that:

  • We run our tests emulating 4g in the new setup (instead of "3gfast"). The reason for this is that we a year ago tried to sync first paint in our tests compared to the 75 percentile of data that we get from the Chrome User Experience report. That matches but that makes our tests runs faster. But as long we compare the same thing it shouldn't matter. In the future we gonna recalibrate our test speed since we now also have the 75 percentile of TTFB.
  • The other reason is that is can differs when the tests ends (when we stop the tests). WebPageTest ends after 2 seconds of no network traffic, Browsertime ends 2 seconds after the load event end. That is configurable but I think the current config is ok, let me know if you do not agree!

Else everything should be the same, the thing I've changed is the number of metrics used for "sparkling mode" (the small graph behind the metric), that needed to be decreased since we do not run Browsertime tests as often as the WebPageTest ones (at the moment). If I hadn't done that, the sparkling mode will not work when looking at data for 7 days.

Please let me know if you see something strange @nray ! Next step will be to add some metrics from our device lab running on real mobile phones, but I want to keep that running for another month before we add those so we are sure they are working as they should.

Last things to do is to remove the old graphs from the Wikidata dashboard and then I'm done.

Both are changed.