Page MenuHomePhabricator

[Bug] lastVisualChange osciallating by ~1s and fullyLoaded oscillating by ~.5s on Obama enwiki prod desktop ~Nov 4 2018
Closed, ResolvedPublic

Description

lastVisualChange and fullyLoaded are spiking up and down significantly for the Barack Obama article around Nov 4:

grafana.wikimedia.org_dashboard_db_mobile-2g_orgId=1&panelId=90&fullscreen.png (891×1 px, 429 KB)

A similar issue is present on mobile for visualComplete99 around Nov 2:

grafana.wikimedia.org_dashboard_db_mobile-2g_orgId=1&panelId=32&fullscreen.png (862×1 px, 486 KB)

A very modest increase to CSS has occurred on Nov 5 but hopefully it's independent of these spikes:

grafana.wikimedia.org_dashboard_db_mobile-2g_orgId=1&panelId=89&fullscreen.png (846×1 px, 106 KB)

A select comparison on wpt.wmftest.org doesn't seem to show significant differences for these dates. Have there been recent changes to our WebpageTest instance?

Event Timeline

Restricted Application changed the subtype of this task from "Deadline" to "Task". · View Herald TranscriptNov 7 2018, 9:30 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Jdlrobson added subscribers: Krinkle, Jdlrobson.

@Krinkle I'm assuming this is either something to do with the metrics or infrastructure. Please let me know if that's not the case and if there's anything we need to look into our end.

@Peter Could you take this one instead?

When I look at these kind of things I always first look at screenshots/videos. There's two reasons: Either there's content that do the change (running a banner etc) or it's bug in the tool, for example Chrome could have change changed the size of the small loading bar at the bottom of screen.

In this case I could see we have the same thing on both FF and Chrome (https://grafana.wikimedia.org/dashboard/db/webpagetest?orgId=1) so it must be content.

Since it was happening almost all the time, I've just browsed through the result of searching (yep finding the result has room for improvements):

Screen Shot 2018-11-12 at 10.41.00 PM.png (384×2 px, 126 KB)

And here is one result that shows why we have a late last visual change http://wpt.wmftest.org/result/181112_9Q_FE/3/screen_shot/#step_1

It's the "This November is the Wikipedia Asian month".

Last Visual Change is kind of a shaky metrics for us to use, since it's impacted on when we run banners. Let me talk to @Imarlier so we can setup some sync where also talk about our new dashboards and what we can use there (and why).

Peter removed Peter as the assignee of this task.Nov 12 2018, 9:46 PM

Last Visual Change is kind of a shaky metrics for us to use, since it's impacted on when we run banners. Let me talk to @Imarlier so we can setup some sync where also talk about our new dashboards and what we can use there (and why).

@Peter thanks for the thorough investigation and a sync up sounds great! I didn't know banners could impact this metric in such a dramatic way so that's good to know!

Jdlrobson claimed this task.

Sounds like we're done here.