Page MenuHomePhabricator

Test if there's a better way to catch size changes for logged in users in WPT
Closed, ResolvedPublic

Description

Today when we test a logged in user, the timings are good but the size changes in assets could be harder to spot. As Ori said, it could be that we terminates the test too early. Lets look into what's default today and change that.

See T112401#1791282 for what we want to make sure to catch.

Event Timeline

Peter created this task.Nov 7 2015, 7:21 PM
Peter updated the task description. (Show Details)
Peter raised the priority of this task from to Needs Triage.
Peter claimed this task.
Peter added a project: Performance-Team.
Peter added subscribers: Peter, ori.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptNov 7 2015, 7:21 PM
Peter updated the task description. (Show Details)Nov 7 2015, 7:24 PM
Peter set Security to None.
Peter added a comment.Nov 7 2015, 7:33 PM

Ok I think (but I couldn't find it now) that by default WebPageTest shuts down a test if nothing happens on the network for 2s, but lets check the waterfalls of the runs, we should be able to see if we are missing requests those requests:

In the API we can configure for minimum test time how long time a test should run, I haven't used it before but lets look into that.

Peter added a comment.Nov 7 2015, 7:47 PM

I think the problem could be SPDY and WebPageTest, the last three days I filled an issue per day with SPDY and Pat tweeted a reminder that Chrome soon will shutdown SPDY :)

The waterfall graphs for a logged in user looks totally screwed:
http://wpt.wmftest.org/result/151107_WY_5G/1/details/

Things changed with a WPT release and the latest Chrome, I don't think it always has been this way. I think a way to "fix" it for now is to increase the minimum time, need to do some testing ...

Peter added a comment.Nov 8 2015, 1:27 PM

I've tested this now and it looks better when running on IE 11 (HTTP 1.1). I will add two additional tests that runs on IE and then I'll make the graph from that data. But lets keep Chrome so we can see the difference when we will do the switch. Will also make an issue at Github for WebPageTest.

Peter added a comment.Nov 8 2015, 1:41 PM

Adding this as reference to what it looks like when Chrome/WPT makes waterfalls graph wrong:

Peter moved this task from Inbox to Doing on the Performance-Team board.Nov 8 2015, 1:42 PM

Change 251812 had a related patch set uploaded (by Krinkle):
Get correct assets size for logged in users

https://gerrit.wikimedia.org/r/251812

Peter closed this task as Resolved.Nov 10 2015, 7:04 AM

This is fixed now and looks good in Grafana, lets skip the fix.

Change 251812 abandoned by Phedenskog:
Get correct assets size for logged in users

Reason:
This has been fixed in WebPageTest, so no need to run with IE.

https://gerrit.wikimedia.org/r/251812