Page MenuHomePhabricator

Add 2G tests for mobile on the mobile dashboard
Closed, ResolvedPublic

Description

Got the question from Jon why we don't test on 2G for mobile and we should of course. When I first implemented it many URL:s failed on 2G but we can increase the timeouts (I think). There's a couple things we need to do:

  • Discuss where to put the connectivity in the Graphite namespace (yep I missed that the first time). Want to talk about it with @Krinkle before we do it.
  • Test some runs using 2G and see the result (and fine tune so it works). Could be that we should separate this on a different Jenkins job so that it will not fail other tests.
  • Add the new runs to the dashboard

Details

Related Gerrit Patches:
performance/WebPageTest : masterTest Barack Obama on 2G for for mobile

Event Timeline

Peter created this task.Dec 9 2015, 7:16 AM
Peter claimed this task.
Peter raised the priority of this task from to Needs Triage.
Peter updated the task description. (Show Details)
Peter added a project: Performance-Team.
Peter added subscribers: Peter, Krinkle, Jdlrobson.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptDec 9 2015, 7:16 AM
Peter added a comment.Dec 9 2015, 12:07 PM

Ok, I think we can do like this:

  • first add the Obama article so we test it for mobile
  • add -2G or -3G to the name part of the namespace.

Setting 2G in the WebPageTestAPI doesn't work (or directly to our instance, but we can set use a custom setting like this (this is 2GSlow in the WebPageTest world):

--bandwidthDown 35000 --bandwidthUp 32000 --latency 1300 --connectivity custom

Should we keep the test we have today and add new ones for 2G so we keep track of both 2G and 3G for phones?

Change 258112 had a related patch set uploaded (by Phedenskog):
Test Barack Obama on 2G for for mobile

https://gerrit.wikimedia.org/r/258112

Gilles moved this task from Inbox to Doing on the Performance-Team board.Dec 14 2015, 7:48 PM
Krinkle triaged this task as Normal priority.Dec 21 2015, 7:51 PM
Krinkle set Security to None.

Change 258112 merged by jenkins-bot:
Test Barack Obama on 2G for for mobile

https://gerrit.wikimedia.org/r/258112

Awesome news!
How long till we should see these on https://grafana.wikimedia.org/dashboard/db/mobile-webpagetest or will they show up elsewhere?

Also this reminds me - I need to setup graphs for mobile beta asap. Am hoping @Krinkle's tech talk on my too watch list will help me with that.

Peter added a comment.Jan 21 2016, 7:40 PM

sorry that it took some time! I've added the graph today, lets move it up when we have some more data (seems like we are having trouble right now to get data for all our WebPageTest runs).

In terms of mobile beta, does it make sense to have separate graphs for 2G / 3G with beta or just have one global one plotting the current first paint time from real users using navigating timing?

event_mobileMode is setup in NavigationTiming API so it should be possible to do all these graphs.

Is there any documentation on how to setup dashboards? I've completely forgotten and can't find any wiki pages about that...

Jdlrobson closed this task as Resolved.Jan 26 2016, 1:06 AM

Looks like the needle is at 12s - which sounds about right. Let's work to move it down :)