FYI: Today when we get an alert email, we look into the issue, but we haven't documented the process and how to "think", lets do that in T225987.

Hi @ayounsi I've tracked this in T225403. The reason is that when we updated to Chrome 75, we could see that it's slower for both WebPageTest and Browsertime/WebPageReplay when we run it in emulated mobile mode. So we gonna keep that task open until we have enough users on mobile that uses 75 and see if we can get the same pattern there, then we will create an upstream issue in Chrome (last week 75 wasn't released yet on Android when I tried it out).

In Chrome 77 it will be possible to "try" it out again: https://cs.chromium.org/chromium/src/components/previews/README

If we could upgrade to 6.2.x that would be great. I've been using it for my projects for a while and the lazy loading of panels out of view really makes large dashboards working better (and faster). We have a couple of dashboards in the performance team that are really slow.

Yep got a small change on Browsertime/WebPageReplay too (one tick):

Re-open since we later this year probably can support iOS devices!

One last thing: It's good to turn on screenshots, that makes easier to understand what happens (--chrome.enableTraceScreenshots). Also I tried to add annotations to the timeline console.timeStamp("Test"); but I cannot get them to be visible in the gui (I see them in the trace log though).

Signing over to you Nick to answer the questions.

At least in theory you can tell the webdriver that you wanna have the control back asap the navigation has started (pageLoadStrategy on the webdriver world).

I've had a go today to first try to automate it, so we can find these kind of things in the future when we have a performance device lab. First @nray for the original, that has been solved right?

Oh @nray sorry I missed this, let me have a look first thing tomorrow.

I've pushed Chrome 75 now, let it run for the day and have another look.

No change in WebPageReplay and I could see that it happened in the same time of Chrome 75 for WebPageTest. On Wednesday I'll roll out 75 for WebPageReplay and I can create an upstream task if we get it there too.

Running both for WPT and WPR

This is probably not practical: When we will create alerts for all Wikis we will do that on one dashboard per Wiki. Having everything (or something smart) on one dashboard is probably too complicated.

https://wikitech.wikimedia.org/wiki/Performance/Runbook/WebPageReplay

Done in: https://wikitech.wikimedia.org/wiki/Performance/Synthetic_Measurement_Experiment_2019

The list is done.

This is one of the ones that we blogged about and Chrome acknowledged.

Yes it is fixed ... but I tested it yesterday and the timeline viewer doesn't support the new timings (fixed that yesterday). So I'm thinking as a long term solution we should remove what I did and ask people to use their local version of Chrome.

We blogged about and get some feedback from the Chrome team that we found something.

Yes we do says @Gilles so closing

When we move to the new setup this will be automatically fixed and will be in Git.

This is fixed now

This works better now when we moved to 30 fps for the video.

T190020 can make it easier to get better metrics.

Let us fix that when we have the new setup.

We will automatically have this when we have the mobile device lab.

I think we should do this differently: On alerts we should pre-populate a link with runs before/after a alert, so you as a user just have a link to click on that automatically fills in the right values into the compare page.

Yeah I've modified our instance but that's is no long term solution. Some time ago I added it back, so we more easily can keep our server update.

Moving to the new setup after the summer we will automatically have it.

When we move this to the new setup all configuration will be in JSON and will be much easier than the current setup.

The simplest way to get the temperature is:

adb -s ZY3222N2CZ shell dumpsys battery | grep temperature | grep -Eo '[0-9]{1,3}'

I've changed so default is 30 instead of null as a hack. If it works good I'll make a PR for the API.

I looked into the code and there is a hidden query fps parameter that you can use. There's two ways to do that:

1. Add support to the WPT API (I can do a PR for that)
2. Hack and set it to 30 fps per default (the same way we hacked setting Visual Elements on all runs.

We got feedback in the upstream issue that there's no way to test today but they will add a flag: https://bugs.chromium.org/p/chromium/issues/detail?id=956685#c13

Looks like you only can set the fps for the video per agent (when you start the agent): https://github.com/WPO-Foundation/wptagent/blob/829a9f8111ae8bcf52c99ff574bcec3c924ebd9e/README.md#video-capturedisplay-settings

I think we can close it and the next time someone tries it out and it doesn't work, let us update the docs!

I've updated Browsertime to use Tracium and a setup that catch more log events, that will work perfect for us. It will work the next time we update.

Hi @nray can we do this in two steps? Since we missed it, I think we should try out first to instrument it. We could use https://developer.mozilla.org/en-US/docs/Web/API/User_Timing_API (we already use that for the startup module)? If we can make sure that we measure it, and push it first, then we get some data from real users, before you push the fix. That way we can make sure it is a real issue today, fix it and enjoy the win of the fix and maybe blog about it :)

It become better with 74 but we still have some bad values in median.

Since this is Chrome running in emulated mode, it's hard to say if it will affect users or not. At least know we know the reason for the increase.

We had the same thing on WebPageReplay too. I actually didn't pay attention to it since we had so unstable metrics on Desktop with 73 T220796

I've been listening to the Chrome teams setup and they wait for the battery heat to come down before they make the next run (to get more stable metrics). It wasn't something I saw as an issue when I run the tests before but we should prepare for getting those numbers (it should be pretty straight forward).

Upstream: https://bugs.chromium.org/p/chromium/issues/detail?id=956685

Let me create an upstream issue asking the correct way to test/turn it on + ask more on how it actually works. I'll do that later tonight.

Should we really spend time on it, I think we should just add the header. Its something we should have had for a long time + until we know exactly how it works, I'm a skeptical.

Hmm, I enabled it again today to test it out and now the "Lite" bubble is removed and there's nothing signal that I'm using a Lite page. I could see that I'm on a Lite page though since no images is loading. Have we seen any drop in traffic since it was released?

I've updated our documentation.

I continued to test and added the --init to get the rid of the zombie processes. That worked, but after 24 h (or so) Firefox started to fail again. I have reverted back to the old agent and hope to get some attention upstream.

It looks that 74 gives less variation (but probably not as 72):

Ok Firefox worked when I started the container but something happens and now it looks like this again:

Also enabled 74 for enwiki. I think it looks better on variation but maybe some URLs are slower. Need to run it through the weekend and see what it looks like.

No it was not across all wikis. Testing 74 now on German, Spanish, Japanese, Chinese and Russian Wikipedia

No change for Firefox at the same moment, that indicates that there's no change in the Browsertime side, rather a Chrome problem. I'll try out a beta of 74 and see if something changes.

I went through all the tests I ran before on the Dockerized version and then the Firefox tests worked: http://wpt.wmftest.org/result/190403_4N_CA/

I like logs but the logs for WPT is really heard to follow. Here's a snippet from the container:

Created an upstream issue asking for help: https://github.com/WPO-Foundation/wptagent/issues/253