Page MenuHomePhabricator

Peter (Peter Hedenskog)
Software Engineer, Wikimedia Foundation

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Thursday

  • Clear sailing ahead.

User Details

User Since
Aug 17 2015, 6:48 PM (271 w, 10 h)
Availability
Available
IRC Nick
phedenskog
LDAP User
Unknown
MediaWiki User
PHedenskog (WMF) [ Global Accounts ]

Recent Activity

Yesterday

Peter moved T266489: Investigate Firefox alerts from Inbox to Doing on the Performance-Team board.
Mon, Oct 26, 5:51 PM · WebPageReplay, Performance-Team
Peter created T266489: Investigate Firefox alerts.
Mon, Oct 26, 5:34 PM · WebPageReplay, Performance-Team

Fri, Oct 23

Peter closed T266059: Learn from how Mozilla runs their tests on Android as Resolved.

Implemented start/clearing Firefox/Chrome before we start our tests.

Fri, Oct 23, 8:14 AM · Performance-Team, Performance-Device-Lab

Thu, Oct 22

Peter added a comment to T263224: Enable Firefox tests for performance device lab.

A positive thing is that yesterdays fix handles when the browser hangs. The root issue is still there, but we handle that at least.

Thu, Oct 22, 7:07 AM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

Kobiton is gonna change the setup for ZY2242HJDX and wire it in the Mac without the hub so we can exclude that the hub is the reason. If the problem still persist I'm gonna do a longer upstream issue.

Thu, Oct 22, 6:38 AM · Patch-For-Review, Performance-Team, Performance-Device-Lab

Wed, Oct 21

Peter added a comment to T263224: Enable Firefox tests for performance device lab.

At least there could be better error messages upstream. Also asked Kobiton about the setup between the Mac Mini and the phones, if there could be something special there.

Wed, Oct 21, 12:11 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

Adding more logs for reference:

1603281834291	geckodriver::android	DEBUG	Launching org.mozilla.firefox/org.mozilla.fenix.IntentReceiverActivity
1603281834392	mozdevice	DEBUG	execute_host_command: >> "host:transport:ZY2242HJDX"
1603281834492	mozdevice	DEBUG	execute_host_command: << []
1603281834492	mozdevice	DEBUG	execute_host_command: >> "shell:am start -W -n org.mozilla.firefox/org.mozilla.fenix.IntentReceiverActivity -a android.intent.action.VIEW -d about:blank --es args --marionette\\ --
profile\\ /mnt/sdcard/org.mozilla.firefox-geckodriver-profile"
1603281838030	mozdevice	DEBUG	execute_host_command: << "Starting: Intent { act=android.intent.action.VIEW dat=about:blank cmp=org.mozilla.firefox/org.mozilla.fenix.IntentReceiverActivity (has extras) }\nStatus: ok\nActivity: org.mozilla.firefox/org.mozilla.fenix.HomeActivity\nThisTime: 1360\nTotalTime: 2284\nWaitTime: 2297\nComplete\n"
1603281838030	geckodriver::marionette	DEBUG	Waiting 60s to connect to browser on 127.0.0.1:64772
1603281838211	geckodriver::android	DEBUG	Force stopping the Android package: org.mozilla.firefox
1603281838211	mozdevice	DEBUG	Force stopping Android package: org.mozilla.firefox
1603281838311	mozdevice	DEBUG	execute_host_command: >> "host:transport:ZY2242HJDX"
1603281838411	mozdevice	DEBUG	execute_host_command: << []
1603281838411	mozdevice	DEBUG	execute_host_command: >> "shell:am force-stop org.mozilla.firefox"
1603281839918	mozdevice	DEBUG	execute_host_command: << ""
1603281839918	webdriver::server	DEBUG	<- 500 Internal Server Error {"value":{"error":"timeout","message":"Socket timeout reading Marionette handshake data: Resource temporarily unavailable (os error 35)","stacktrace":""}}
Wed, Oct 21, 12:06 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

I've pushed a changed where I added a timeout of two minutes when we try close the browser to avoid making tests hang. When I looked at the code, that was the only thing I could find. Best case that would at least make sure our tests continuous to run. The root cause will still be there (the browser freezes) but maybe we can live with that.

Wed, Oct 21, 11:48 AM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T194639: Pick up Navigation Timing metrics with WebPageTest.

I think that's how WebPageTest choose to iterate over the object. Its alphabetically sorted where I created it.

Wed, Oct 21, 11:01 AM · Performance-Team, WebPageTest
Peter added a comment to T194639: Pick up Navigation Timing metrics with WebPageTest.

I've added :/var/www/webpagetest/www/settings/custom_metrics/navigationTimings.js on the server.

Wed, Oct 21, 8:31 AM · Performance-Team, WebPageTest
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

Been running the tests though the night and suddenly the tests stopped. Look at the time difference between the last log line and the rotation of the log:

[2020-10-20 19:36:25] INFO: URL failed to load, trying 0 more time(s): Failed waiting on page https://en.m.wikipedia.org/wiki/Main_Page  to finished loading, timed out after 300000 ms 
[2020-10-20 19:36:25] ERROR: Could not load URL https://en.m.wikipedia.org/wiki/Main_Page UrlLoadError: Failed waiting on page https://en.m.wikipedia.org/wiki/Main_Page  to finished loading, timed out after 300000 ms 
    at SeleniumRunner.wait (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:188:13)
[2020-10-20 19:36:25] ERROR: UrlLoadError: Failed to load https://en.m.wikipedia.org/wiki/Main_Page
    at SeleniumRunner.loadAndWait (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:313:17)
Oct 21 00:30:04 Kobitons-Mac-mini-156 newsyslog[27602]: logfile turned over
Wed, Oct 21, 6:26 AM · Patch-For-Review, Performance-Team, Performance-Device-Lab

Tue, Oct 20

Peter added a comment to T263224: Enable Firefox tests for performance device lab.

My guess is that " TypeError: win is null" comes from RUMSpeedIndex, that's the only JS that we try to run that has "win" defined.

Tue, Oct 20, 9:02 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

The new phone also gets errors:

[2020-10-20 16:33:29] INFO: firefox failed to start, trying 2 more time(s): Failed to start firefox in 60 seconds.
Tue, Oct 20, 8:58 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T219496: [Meta] Next level of synthetic testing.

see Performance-Device-Lab

Tue, Oct 20, 7:57 PM · Performance-Team
Peter added a comment to T194639: Pick up Navigation Timing metrics with WebPageTest.

Let me pick this up again in a week or two.

Tue, Oct 20, 7:47 PM · Performance-Team, WebPageTest
Peter committed rPMSMf2c37d374e2d: Prepare settings to run Firefox again. (authored by Peter).
Prepare settings to run Firefox again.
Tue, Oct 20, 6:53 PM
Peter moved T265514: Checkout Kobiton throttling setup from Inbox to Backlog: Small & Maintenance on the Performance-Team board.
Tue, Oct 20, 6:45 PM · Performance-Team, Performance-Device-Lab
Peter moved T265519: Verify bulk test documentation from Inbox to Backlog: Small & Maintenance on the Performance-Team board.
Tue, Oct 20, 6:45 PM · Documentation, WebPageTest, Performance-Team
Peter closed T265532: Investigate why the temperature don't go down on ZY3222N2CZ as Resolved.

This haven't happened again so close it.

Tue, Oct 20, 6:44 PM · Performance-Team, Performance-Device-Lab
Peter closed T265855: Add better error messages for failing Chrome and investigate connection refused as Resolved.
Tue, Oct 20, 6:43 PM · Performance-Team, Performance-Device-Lab
Peter closed T265856: Investigate Firefox failures on Kobiton as Resolved.

Investigations happened in T263224

Tue, Oct 20, 6:42 PM · Performance-Team, Performance-Device-Lab
Peter moved T266059: Learn from how Mozilla runs their tests on Android from Inbox to Doing on the Performance-Team board.
Tue, Oct 20, 6:42 PM · Performance-Team, Performance-Device-Lab
Peter added a comment to T265514: Checkout Kobiton throttling setup.

I've asked for access but they haven't got back yet.

Tue, Oct 20, 6:41 PM · Performance-Team, Performance-Device-Lab
Peter created T266059: Learn from how Mozilla runs their tests on Android.
Tue, Oct 20, 6:39 PM · Performance-Team, Performance-Device-Lab
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

Hit the right spot! Chrome looks like this:

Tue, Oct 20, 6:11 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter committed rPMSM309f52eb594f: Try Chrome on the same phone. (authored by Peter).
Try Chrome on the same phone.
Tue, Oct 20, 6:04 PM
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

I will try running Chrome on the same phone for a couple of days and see if we get the same problem there.

Tue, Oct 20, 5:59 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

Latest from Mozilla is that its "some weird networking/port forwarding issue". We setup the wifi with proxy, I wonder if that somehow can have something to do with it? One way could be to just test with WebPageReplay and see what if we get the same problem.

Tue, Oct 20, 5:47 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

Let me do summary of the status: I've enabled continuous testing on Kobiton for one phone last week. Test works for a while (a couple of URLs for X runs), then it starts to fail because the browser cannot be open. Then after some more time it starts to work again. I cannot reproduce this behavior when I test on my local phones.

Tue, Oct 20, 5:28 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter closed T198394: Change viewport for desktop tests (Browsertime and WebPageTest) as Resolved.

Let look at the metrics. Focus on First Visual Change. For WebPageTest I can see that some pages are affected (Facebook/Obama):


Tue, Oct 20, 10:02 AM · Patch-For-Review, WebPageReplay, WebPageTest, Performance-Team

Mon, Oct 19

Peter added a comment to T263224: Enable Firefox tests for performance device lab.

Looking at the logs when Firefox can't start (INFO: Browser failed to start, trying 2 more time(s): Failed to start browser in 60 seconds.) using:

adb -s ZY322HX8RC logcat

Looks like this can be related:

"adbd    : failed to connect to socket 'tcp:2829': Connection refused "

Looking at only Marionette logs only gives INFO logs with no help. I'm gonna turn on verbose logging and see if I can see something there.

Mon, Oct 19, 11:54 AM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T198394: Change viewport for desktop tests (Browsertime and WebPageTest).

For Browsertime it looks like this:

Mon, Oct 19, 11:23 AM · Patch-For-Review, WebPageReplay, WebPageTest, Performance-Team
Peter added a comment to T198394: Change viewport for desktop tests (Browsertime and WebPageTest).

Here's what it looked like on WebPageTest before the change:

Mon, Oct 19, 11:15 AM · Patch-For-Review, WebPageReplay, WebPageTest, Performance-Team
Peter added a comment to T265856: Investigate Firefox failures on Kobiton.

Hmm I was wrong, restarting the phone got the test to start running again (without restarting the tests), but I couldn't see anything that was stuck in the logs or using ps. Strange.

Mon, Oct 19, 7:10 AM · Performance-Team, Performance-Device-Lab
Peter added a comment to T265856: Investigate Firefox failures on Kobiton.

The last thing that happened on the phone was:

[2020-10-17 10:31:16] ERROR: Failed waiting on page https://en.m.wikipedia.org/wiki/Barack_Obama to finished loading, timed out after 300000 ms BrowserError: Running page complete check 
return (function(waitTime) {
    try { 
            var end = window.performance.timing.loadEventEnd;
            var start= window.performance.timing.navigationStart;
            return (end > 0) && (performance.now() > end - start + waitTime);
    } 
    catch(e) {
        return true;
    }
})(arguments[arguments.length - 1]);
 took too long
return (function(waitTime) {
    try { 
            var end = window.performance.timing.loadEventEnd;
            var start= window.performance.timing.navigationStart;
            return (end > 0) && (performance.now() > end - start + waitTime);
    } 
    catch(e) {
        return true;
    }
})(arguments[arguments.length - 1]);
 took too long
    at /usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:39:38
    at new Promise (<anonymous>)
    at timeout (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:38:5)
    at SeleniumRunner.wait (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:169:13)
    at SeleniumRunner.loadAndWait (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:259:20)
    at async Measure.start (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/command/measure.js:201:9)
    at async /usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/run.js:9:9
    at async Iteration.run (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/iteration.js:191:9)
    at async Engine.runByScript (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/index.js:215:20)
    at async Object.analyzeUrl (/usr/local/lib/node_modules/sitespeed.io/lib/plugins/browsertime/analyzer.js:124:21)
    at async Object.processMessage (/usr/local/lib/node_modules/sitespeed.io/lib/plugins/browsertime/index.js:134:24)
[2020-10-17 10:31:16] INFO: URL failed to load, trying 1 more time(s): Failed waiting on page https://en.m.wikipedia.org/wiki/Barack_Obama  to finished loading, timed out after 300000 ms 
[2020-10-17 10:36:17] ERROR: Failed waiting on page https://en.m.wikipedia.org/wiki/Barack_Obama to finished loading, timed out after 300000 ms BrowserError: Running page complete check 
return (function(waitTime) {
    try { 
            var end = window.performance.timing.loadEventEnd;
            var start= window.performance.timing.navigationStart;
            return (end > 0) && (performance.now() > end - start + waitTime);
    } 
    catch(e) {
        return true;
    }
})(arguments[arguments.length - 1]);
 took too long
return (function(waitTime) {
    try { 
            var end = window.performance.timing.loadEventEnd;
            var start= window.performance.timing.navigationStart;
            return (end > 0) && (performance.now() > end - start + waitTime);
    } 
    catch(e) {
        return true;
    }
})(arguments[arguments.length - 1]);
 took too long
    at /usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:39:38
    at new Promise (<anonymous>)
    at timeout (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:38:5)
    at SeleniumRunner.wait (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:169:13)
    at SeleniumRunner.loadAndWait (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:259:20)
    at async Measure.start (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/command/measure.js:201:9)
    at async /usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/run.js:9:9
    at async Iteration.run (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/iteration.js:191:9)
    at async Engine.runByScript (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/engine/index.js:215:20)
    at async Object.analyzeUrl (/usr/local/lib/node_modules/sitespeed.io/lib/plugins/browsertime/analyzer.js:124:21)
    at async Object.processMessage (/usr/local/lib/node_modules/sitespeed.io/lib/plugins/browsertime/index.js:134:24)
[2020-10-17 10:36:17] INFO: URL failed to load, trying 0 more time(s): Failed waiting on page https://en.m.wikipedia.org/wiki/Barack_Obama  to finished loading, timed out after 300000 ms 
[2020-10-17 10:36:17] ERROR: Could not load URL https://en.m.wikipedia.org/wiki/Barack_Obama UrlLoadError: Failed waiting on page https://en.m.wikipedia.org/wiki/Barack_Obama  to finished loading, timed out after 300000 ms 
    at SeleniumRunner.wait (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:186:13)
[2020-10-17 10:36:17] ERROR: UrlLoadError: Failed to load https://en.m.wikipedia.org/wiki/Barack_Obama
    at SeleniumRunner.loadAndWait (/usr/local/lib/node_modules/sitespeed.io/node_modules/browsertime/lib/core/seleniumRunner.js:306:17)
Oct 18 00:30:04 Kobitons-Mac-mini-156 newsyslog[22220]: logfile turned over
Mon, Oct 19, 6:51 AM · Performance-Team, Performance-Device-Lab
Peter added a comment to T198394: Change viewport for desktop tests (Browsertime and WebPageTest).

I've restarted WebPageTest to also use 1920x1080.

Mon, Oct 19, 6:39 AM · Patch-For-Review, WebPageReplay, WebPageTest, Performance-Team
Peter created T265856: Investigate Firefox failures on Kobiton.
Mon, Oct 19, 6:36 AM · Performance-Team, Performance-Device-Lab
Peter created T265855: Add better error messages for failing Chrome and investigate connection refused.
Mon, Oct 19, 6:34 AM · Performance-Team, Performance-Device-Lab

Fri, Oct 16

Peter added a comment to T263224: Enable Firefox tests for performance device lab.

I've been trying to get Firefox to work better or rather understand what's wrong. I think there's multiple issues. One have been that there's already been running an instance. Since we running on a non root and shell am force-stop org.mozilla.firefox couldn't stop the device since we have multiple instances running(?) I've been restarting the instance to kill Firefox.

Fri, Oct 16, 4:15 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T198394: Change viewport for desktop tests (Browsertime and WebPageTest).

I configured for one of the test for WebPageReplay, push it on Monday to and let us see what changes.

Fri, Oct 16, 11:50 AM · Patch-For-Review, WebPageReplay, WebPageTest, Performance-Team
Peter added a comment to T198394: Change viewport for desktop tests (Browsertime and WebPageTest).

I couldn't find that the WebPageTest API supports setting size, but you can configure it in the settings in /var/www/webpagetest/www/settings/settings.ini. I prepared for the change from 1024x768 to 1920x1080 in the settings, but I prefer pushing on Monday!

Fri, Oct 16, 11:46 AM · Patch-For-Review, WebPageReplay, WebPageTest, Performance-Team

Thu, Oct 15

Peter committed rPMSMa2c49f184d4f: Reboot the phone if the temperature is too high for long time. (authored by Peter).
Reboot the phone if the temperature is too high for long time.
Thu, Oct 15, 12:30 PM

Wed, Oct 14

Peter closed T192050: Use Docker version of WebPageTest agents as Declined.

When we move the agent in-house we will use our own git copy.

Wed, Oct 14, 9:23 PM · Upstream, WebPageTest, Performance-Team
Peter closed T164714: Calculate parsing speed for JavaScript on mobile, a subtask of T136850: Test performance on real mobile devices, as Resolved.
Wed, Oct 14, 9:13 PM · Performance-Team, Epic
Peter closed T164714: Calculate parsing speed for JavaScript on mobile as Resolved.

Woho we do that now.

Wed, Oct 14, 9:13 PM · Mobile, Performance-Team
Peter closed T217771: Remove (or fix) noisy Speed Index alerts as Declined.

We will not fix this since with CLS its important that keep track of Speed Index alerts that fires.

Wed, Oct 14, 9:13 PM · WebPageTest, WebPageReplay, Performance-Team
Peter created T265532: Investigate why the temperature don't go down on ZY3222N2CZ.
Wed, Oct 14, 7:43 PM · Performance-Team, Performance-Device-Lab
Peter created T265519: Verify bulk test documentation .
Wed, Oct 14, 6:17 PM · Documentation, WebPageTest, Performance-Team
Peter added a comment to T123582: Use "preconnect" resource hint for thumbnail host.

For one off tests I think https://wikitech.wikimedia.org/wiki/Performance/WebPageTest/WPTBulkTester and Andy Davies spreadsheet is perfect. Adding it here for the next time we gonna do it.

Wed, Oct 14, 6:15 PM · Patch-For-Review, Performance-Team, MediaWiki-General, Multimedia
Peter created T265514: Checkout Kobiton throttling setup.
Wed, Oct 14, 5:47 PM · Performance-Team, Performance-Device-Lab

Tue, Oct 13

Peter moved T264734: Investigate first visual change increase for NL using Firefox from Inbox to Backlog: Small & Maintenance on the Performance-Team board.
Tue, Oct 13, 6:14 PM · WebPageReplay, Performance-Team
Peter moved T265273: Update documentation to what we learned the last months from Next In This Quarter / Oct-Dec 2020 to Doing on the Performance-Team board.
Tue, Oct 13, 6:14 PM · Documentation, Performance-Team, Performance-Device-Lab
Peter moved T265273: Update documentation to what we learned the last months from Inbox to Next In This Quarter / Oct-Dec 2020 on the Performance-Team board.
Tue, Oct 13, 6:14 PM · Documentation, Performance-Team, Performance-Device-Lab
Peter closed T263226: Simplify start/stop of mobile devices as Resolved.
Tue, Oct 13, 12:45 PM · Performance-Team, Performance-Device-Lab
Peter closed T263907: Fine tune the start script for mobile testing as Resolved.
Tue, Oct 13, 12:44 PM · Performance-Team, Performance-Device-Lab
Peter committed rPMSM27fb9e1bf9f8: Adjust the stop script to follow the latest change. (authored by Peter).
Adjust the stop script to follow the latest change.
Tue, Oct 13, 11:21 AM
Peter committed rPMSMe43ea99d3cbc: Fix stop permissions. (authored by Peter).
Fix stop permissions.
Tue, Oct 13, 11:21 AM
Peter committed rPMSM4a0e9f7075f3: Cleanup start script: start one or all phones. (authored by Peter).
Cleanup start script: start one or all phones.
Tue, Oct 13, 10:59 AM
Peter committed rPMSM60c2a89fe720: Stop script to stop all tests or test on one phone. (authored by Peter).
Stop script to stop all tests or test on one phone.
Tue, Oct 13, 10:37 AM
Peter committed rPMSM6cc1a8747e91: Update the last phone to Chromedriver 86. (authored by Peter).
Update the last phone to Chromedriver 86.
Tue, Oct 13, 6:26 AM
Peter closed T265211: Phone lost wifi connection as Resolved.

The problem was the box that shared the wifi. Kobiton told me they couldn't connect to that wifi with desktop either. They restarted the box and it worked. They plan to replace it soon.

Tue, Oct 13, 5:54 AM · Performance-Device-Lab, Performance-Team

Mon, Oct 12

Peter added a comment to T263224: Enable Firefox tests for performance device lab.

I updated Firefox on all phones today to run 81.1.2 but no luck when running tests. Now it always fail. Turned on the trace log for gecko driver:

Mon, Oct 12, 11:10 AM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter created T265273: Update documentation to what we learned the last months.
Mon, Oct 12, 11:02 AM · Documentation, Performance-Team, Performance-Device-Lab
Peter closed T265116: Upgrade phones to use Chrome 86 as Resolved.
Mon, Oct 12, 9:42 AM · Performance-Team, Performance-Device-Lab
Peter closed T264902: Add documentation how to debug WebPageReplay failures as Resolved.

Added https://wikitech.wikimedia.org/wiki/Performance/WebPageReplay#Debug_missing_metrics

Mon, Oct 12, 7:36 AM · Performance-Team, Documentation, WebPageReplay
Peter closed T264585: Investigate battery temperature measurements as Resolved.

Closing this since it was a Kobiton issue, fixed on Tuesday the coming week.

Mon, Oct 12, 6:51 AM · Performance-Team, Performance-Device-Lab
Peter closed T264901: Add documentation how to debug WebPageTest failures as Resolved.

https://wikitech.wikimedia.org/wiki/Performance/WebPageTest#Debug_missing_metrics

Mon, Oct 12, 6:37 AM · Performance-Team, Documentation, WebPageTest

Sun, Oct 11

Peter created T265211: Phone lost wifi connection.
Sun, Oct 11, 1:53 PM · Performance-Device-Lab, Performance-Team

Fri, Oct 9

Peter added a comment to T262962: Move the WebPageTest server in-house.

I think we should sync this, I see some moving parts (like WebPageTest auto update of code, update browsers etc) that we need to discuss, I'm not sure how it would work. I'll invite you for a start meeting on Monday @dpifke and then we can start by me going through the current setup and how it works. WebPageTest misses a lot of documentation and when I wrote our part it was mostly focused on making sure I would remember how set it up a second time.

Fri, Oct 9, 10:55 AM · Patch-For-Review, Performance-Team, WebPageTest
Peter created T265116: Upgrade phones to use Chrome 86.
Fri, Oct 9, 7:54 AM · Performance-Team, Performance-Device-Lab

Thu, Oct 8

Peter added a comment to T263224: Enable Firefox tests for performance device lab.

Or actually I should try install Firefox beta on the phones and see if it works.

Thu, Oct 8, 12:42 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter added a comment to T263224: Enable Firefox tests for performance device lab.

Haven't got this to work on the phones on Kobiton. Works on my machine though :) I think the version of Firefox will be released soon (in the middle of October) so lets wait for that + see if I can get a throttled wifi at home.

Thu, Oct 8, 12:36 PM · Patch-For-Review, Performance-Team, Performance-Device-Lab
Peter closed T264916: Upgrade to Chrome 86 as Resolved.

Thu, Oct 8, 11:31 AM · Performance-Team, WebPageReplay
Peter added a comment to T262962: Move the WebPageTest server in-house.

Yep I think we can try the wptserver-install, the image on AWS was created a couple of years before the official instructions so I've never looked into the dependencies for the server :)

Thu, Oct 8, 6:41 AM · Patch-For-Review, Performance-Team, WebPageTest

Wed, Oct 7

Peter closed T263859: Consider purging articles before synthetic test runs as Resolved.
Wed, Oct 7, 6:11 PM · Performance-Team
Peter added a comment to T263859: Consider purging articles before synthetic test runs.

Yep I added https://en.m.wikipedia.beta.wmflabs.org/wiki/Barack_Obama I think we only test one article there.

Wed, Oct 7, 6:10 PM · Performance-Team
Peter closed T263906: Investigate WebPageReplay tests as Resolved.

This has been fixed now. There reason was the configured proxy on the phone.

Wed, Oct 7, 6:04 PM · Performance-Team, Performance-Device-Lab
Peter moved T264916: Upgrade to Chrome 86 from Inbox to Doing on the Performance-Team board.
Wed, Oct 7, 5:59 PM · Performance-Team, WebPageReplay
Peter created T264916: Upgrade to Chrome 86.
Wed, Oct 7, 5:59 PM · Performance-Team, WebPageReplay
Peter added a comment to T264585: Investigate battery temperature measurements.

I talked to Kobiton: I back tracked when the temperature increased and they could see that they installed more phones at that time in the same cabinet. They gonna move the phones and let me know when they do it.

Wed, Oct 7, 5:05 PM · Performance-Team, Performance-Device-Lab
Peter added a comment to T264585: Investigate battery temperature measurements.

I asked Mozilla about there setups but the performance team doesn't do anything special, the phone host has fixed everything. I did some check for processes to try to find if something strange was running but couldn't find anything. Next step will be to restart the phone and let it rest for a couple of hours and check what kind of temperature we get then.

Wed, Oct 7, 4:34 PM · Performance-Team, Performance-Device-Lab
Peter closed T264304: Missing data from WebPageTest since 29/9 as Resolved.

Created T264901 for documentation.

Wed, Oct 7, 3:49 PM · Upstream, Performance-Team, WebPageTest
Peter moved T264901: Add documentation how to debug WebPageTest failures from Inbox to Doing on the Performance-Team board.
Wed, Oct 7, 3:48 PM · Performance-Team, Documentation, WebPageTest
Peter moved T264903: Add documentation how to debug performance device lab failures from Inbox to Backlog: Small & Maintenance on the Performance-Team board.
Wed, Oct 7, 3:48 PM · Documentation, Performance-Team, Performance-Device-Lab
Peter moved T264902: Add documentation how to debug WebPageReplay failures from Inbox to Backlog: Small & Maintenance on the Performance-Team board.
Wed, Oct 7, 3:48 PM · Performance-Team, Documentation, WebPageReplay
Peter created T264903: Add documentation how to debug performance device lab failures.
Wed, Oct 7, 3:46 PM · Documentation, Performance-Team, Performance-Device-Lab
Peter created T264902: Add documentation how to debug WebPageReplay failures.
Wed, Oct 7, 3:45 PM · Performance-Team, Documentation, WebPageReplay
Peter created T264901: Add documentation how to debug WebPageTest failures.
Wed, Oct 7, 3:43 PM · Performance-Team, Documentation, WebPageTest
Peter added a comment to T264304: Missing data from WebPageTest since 29/9.

Metrics is slowly coming in:

Wed, Oct 7, 11:46 AM · Upstream, Performance-Team, WebPageTest
Peter added a comment to T264304: Missing data from WebPageTest since 29/9.

All old jobs removed. Let me keep this task open until we see new data arriving and that I've added docs for error searching.

Wed, Oct 7, 10:45 AM · Upstream, Performance-Team, WebPageTest
Peter added a comment to T264304: Missing data from WebPageTest since 29/9.

Yep the data storage is back to normal (6% usage on /data/). The problem is that all the tests that has been submitted during the downtime is being processed. To fix that I remove all jobs in /var/www/webpagetest/www/work/jobs/us-east and then we just get the new jobs.

Wed, Oct 7, 10:42 AM · Upstream, Performance-Team, WebPageTest
Peter claimed T264304: Missing data from WebPageTest since 29/9.

It seems though that the tests don't run (I cannot see any new data in Graphite). Let me have look and if I cannot find it I'll ping you tonight @dpifke so we can work together on it so we get it going.

Wed, Oct 7, 10:36 AM · Upstream, Performance-Team, WebPageTest

Tue, Oct 6

Peter added a comment to T247727: Simulate slow connections on real devices.

I'll add some new background: At the moment I'm not able to replicate our setup at Kobiton. I throttle the connection using gnirehtet to let the phone use the connection of my desktop. When testing out Firefox on Kobiton I've got problems where the first test works and the following tests do not work. I tested on multiple phones there and get the same result. But I can't reproduce on my own machine use gnirehtet, I get other problems/errors and I think the key issue here is that on my local all traffic + driving the device go though USB but at Kobiton we drive the device through USB and traffic through wifi.

Tue, Oct 6, 1:53 PM · Performance-Device-Lab, Patch-For-Review, Performance-Team
Peter added a project to T247727: Simulate slow connections on real devices: Performance-Device-Lab.
Tue, Oct 6, 1:48 PM · Performance-Device-Lab, Patch-For-Review, Performance-Team
Peter added a comment to T264585: Investigate battery temperature measurements.

I stopped the test for almost 4 hours and when starting again the temperature was 35 for ZY3222N2CZ and 33 for ZY2242HHQ8. So ZY3222N2CZ didn't come down at all. The temperature in the data center is 20.

Tue, Oct 6, 1:21 PM · Performance-Team, Performance-Device-Lab
Peter created T264734: Investigate first visual change increase for NL using Firefox.
Tue, Oct 6, 11:09 AM · WebPageReplay, Performance-Team
Peter added a comment to T264032: Record long tasks in navtiming.

FYI: Sometime ago I implemented the buffered flag in Browsertime but I needed to rollback to the old solution (inject the observer when the page start to load) because I could see that the buffered flag missed long tasks. Also see https://twitter.com/Joseph_Wynn/status/1310755139704074240 and https://bugs.chromium.org/p/chromium/issues/detail?id=1131385&q=long%20task%20buffered&can=2

Tue, Oct 6, 7:16 AM · Patch-For-Review, MediaWiki-extensions-NavigationTiming, Performance-Team
Peter reassigned T264304: Missing data from WebPageTest since 29/9 from Peter to dpifke.

Checking the nginx error logs in /var/log/nginx it looks like this:

Tue, Oct 6, 6:57 AM · Upstream, Performance-Team, WebPageTest

Mon, Oct 5

Peter moved T264304: Missing data from WebPageTest since 29/9 from Inbox to Doing on the Performance-Team board.
Mon, Oct 5, 6:31 PM · Upstream, Performance-Team, WebPageTest
Peter moved T264317: Review performance tests outcome from Inbox to Backlog: Small & Maintenance on the Performance-Team board.
Mon, Oct 5, 6:30 PM · Performance-Team, Performance-Device-Lab