Page MenuHomePhabricator

Peter (Peter Hedenskog)
Software Engineer, Wikimedia Foundation

Today

  • Clear sailing ahead.

Tomorrow

  • Clear sailing ahead.

Wednesday

  • Clear sailing ahead.

User Details

User Since
Aug 17 2015, 6:48 PM (444 w, 6 d)
Availability
Available
IRC Nick
phedenskog
LDAP User
Unknown
MediaWiki User
PHedenskog (WMF) [ Global Accounts ]

Recent Activity

Thu, Feb 22

Peter updated subscribers of T358051: Many errors from Grafana with alertname=DatasourceError team=qte 2024-02-20.

This started to happen again today. I wonder @fgiunchedi can you help me point in the right direction how long timeout we have today for the alerts that goes to the synthetic graphite instance? I looked at our instance but couldn't see anything.

Thu, Feb 22, 5:37 PM · Quality-and-Test-Engineering-Team
Peter reopened T358051: Many errors from Grafana with alertname=DatasourceError team=qte 2024-02-20 as "Open".
Thu, Feb 22, 5:32 PM · Quality-and-Test-Engineering-Team
Peter added a comment to T357530: Move direct tests to bare metal servers.

The cleanup is done so lets do this the 4th of March.

Thu, Feb 22, 3:41 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T357628: Remove synthetic direct tests as Resolved.

Cleanup done.

Thu, Feb 22, 3:40 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T357628: Remove synthetic direct tests, a subtask of T357530: Move direct tests to bare metal servers, as Resolved.
Thu, Feb 22, 3:40 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T358051: Many errors from Grafana with alertname=DatasourceError team=qte 2024-02-20 as Resolved.

It seems like this was a temporary glitch.

Thu, Feb 22, 3:03 PM · Quality-and-Test-Engineering-Team
Peter closed T358201: Change alert rules for Mann Whitney U tests, a subtask of T351929: Deploy Mann Whitney U tests for WebPageReplay tests, as Resolved.
Thu, Feb 22, 2:20 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T358201: Change alert rules for Mann Whitney U tests as Resolved.

This must be some glitch in Grafana. After many tries and deleting the rules and re-create them, they work as expected.

Thu, Feb 22, 2:20 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T358201: Change alert rules for Mann Whitney U tests.

So this annoying. When add graph in Grafana to graph number of pages that have regression:

alerts.jpg (1×2 px, 281 KB)

Thu, Feb 22, 12:59 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter renamed T358201: Change alert rules for Mann Whitney U tests from Change alert rules to Change alert rules for Mann Whitney U tests.
Thu, Feb 22, 11:15 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter created T358201: Change alert rules for Mann Whitney U tests.
Thu, Feb 22, 11:15 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Wed, Feb 21

Peter created T358129: Add support for logging in the webdriver.io tests.
Wed, Feb 21, 5:54 PM · Browser-Tests, MediaWiki-Core-Tests, Quality-and-Test-Engineering-Team
Peter renamed T358092: Fix async for wdio in core from Add missing async for wdio in core to Fix async for wdio in core.
Wed, Feb 21, 11:45 AM · Browser-Tests, MediaWiki-Core-Tests, Patch-For-Review, Quality-and-Test-Engineering-Team
Peter created T358094: Add missing await on wdio tests.
Wed, Feb 21, 11:35 AM · Patch-For-Review, Quality-and-Test-Engineering-Team, Browser-Tests, MediaWiki-Core-Tests
Peter added projects to T358092: Fix async for wdio in core: MediaWiki-Core-Tests, Browser-Tests.
Wed, Feb 21, 11:33 AM · Browser-Tests, MediaWiki-Core-Tests, Patch-For-Review, Quality-and-Test-Engineering-Team
Peter created T358092: Fix async for wdio in core.
Wed, Feb 21, 11:25 AM · Browser-Tests, MediaWiki-Core-Tests, Patch-For-Review, Quality-and-Test-Engineering-Team

Tue, Feb 20

Peter created T358051: Many errors from Grafana with alertname=DatasourceError team=qte 2024-02-20.
Tue, Feb 20, 8:53 PM · Quality-and-Test-Engineering-Team

Mon, Feb 19

Peter closed T347261: Update performance team documentation as Resolved.
Mon, Feb 19, 7:08 AM · Quality-and-Test-Engineering-Team, Documentation
Peter closed T356839: Restructure test and configuration names for performance tests as Resolved.
Mon, Feb 19, 5:48 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Fri, Feb 16

Peter added a comment to T356839: Restructure test and configuration names for performance tests.

This has been changed. To close I need to remove the old data in Graphite.

Fri, Feb 16, 9:27 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T356839: Restructure test and configuration names for performance tests.

When we do this change I want to do two more changes:

  • remove the latency for the "old" test so we run all the tests the same way. That will makes our tests run faster.
  • change group 1 tests to use it.wikipedia.org. Pages there have more content and that will make it easier to spot regressions.
Fri, Feb 16, 8:19 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Thu, Feb 15

Peter created T357628: Remove synthetic direct tests.
Thu, Feb 15, 1:09 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Wed, Feb 14

Peter created T357530: Move direct tests to bare metal servers.
Wed, Feb 14, 1:32 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Tue, Feb 13

Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

What I've done now is reconfigured a personal bare metal server and then I'm gonna keep that running for a couple of days and se how stable metrics we can with direct tests.

Tue, Feb 13, 8:43 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

I think this is showing when things started go wrong. It's the same for all Chrome URLs that we test. What is strange though is that we don't see the same on Firefox (on the same server). I verified that it's the exact same Chrome version.

firstvisual.jpg (952×2 px, 238 KB)

Tue, Feb 13, 7:15 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Fri, Feb 9

zeljkofilipin awarded Blog Post: Web Performance at FOSDEM 4th of February a Like token.
Fri, Feb 9, 1:11 PM
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

This happened again last night. I've been comparing processes and in the high states there is kworker (kernel worker threads) but I having a hard time digging deeper.

Fri, Feb 9, 6:54 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Thu, Feb 8

Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

Looking at the server stats I can see that everytime this happens (increase in first visual change, total blocking time) we have an decrease in CPU idle time and decrease ps state running.

Thu, Feb 8, 1:41 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

I cannot see any correlation for the CPU benchmark metric and the TBT.

Thu, Feb 8, 12:09 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

All WebPageReplay tests has been updated to use Mann Whitney. I've updated the documentation too, now I need to finish the blog post to close this task.

Thu, Feb 8, 11:59 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

The numbers have gone back. We can see that long tasks changed, and that made first visual change better.

Thu, Feb 8, 11:46 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Wed, Feb 7

Peter added a comment to T356875: Run a linter in CI to check if all Selenium tests have correct async/await.

I did some checks and for linting I think it will only help us if we used typescript with rules like @typescript-eslint/await-thenable, @typescript-eslint/require-await and @typescript-eslint/no-floating-promises.

Wed, Feb 7, 6:00 PM · MediaWiki-Core-Tests, User-zeljkofilipin
Peter created T356839: Restructure test and configuration names for performance tests.
Wed, Feb 7, 7:47 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

I'm updating the group 0 and group 1 alerts today for desktop. What's missing to close this is then to do the blog post and do emulated mobile tests for with significant change.

Wed, Feb 7, 7:29 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Tue, Feb 6

Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

I removed some old test (beta cluster) and increased the number or runs for group 0 and 1 and then turned on Mann Whitney tests for them too.

Tue, Feb 6, 2:22 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Mon, Feb 5

Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

I got some feedback from Greg:

  • First let's run the exact same test against the exact same content throughout a day or two and see the variance we have. I need to do some changes in our implementation because today we alway re-record against a fresh version of the page. With the fix, we will run against the exact same version (the way we tried with the static Banksy page). Maybe we can use the Bansky page first.
  • When we have a significant change, alert of the change is larger than 2%. Today we alert on all significant changes.
Mon, Feb 5, 3:47 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T356367: Run baseline tests over a day and verify stability, a subtask of T355790: Verify functionality for device lab, as Resolved.
Mon, Feb 5, 1:49 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter closed T356367: Run baseline tests over a day and verify stability as Resolved.
Mon, Feb 5, 1:49 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a comment to T356367: Run baseline tests over a day and verify stability.
{
  "android": true,
  "replay": true,
    "browsertime": {
	"android": {
	"rooted": true
	},
	"connectivity": {
      "engine": "throttle",
      "throttle": {
        "localhost": true
      },
      "profile": "custom",
      "alias:": "120rtt",
      "rtt": 120
    },
      "chrome": {
        "args": [
          "host-resolver-rules=MAP *:80 127.0.0.1:8085,MAP *:443 127.0.0.1:8086,EXCLUDE localhost",
          "ignore-certificate-errors-spki-list=PhrPvGIaAMmd29hj8BCZOq096yj7uMpRNHpn5PDxI6I=",
          "user-data-dir=/data/local/tmp/chrome/"
        ]
      },
      "firefox": {
        "preference": [
          "network.dns.forceResolve:127.0.0.1",
          "security.OCSP.enabled:0",
          "network.socket.forcePort:80=8085;443=8086"
Mon, Feb 5, 1:49 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

I just re-deployed the server direct1 to see if we get any difference in metrics. There's an annotation when I deployed so lets wait a couple hours and check.

Mon, Feb 5, 12:50 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Sun, Feb 4

Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

Looking at the Chrome trace log there's a layout phase before FCP that causes it that is something like 300 ms longer. It the same amount of elements though.

Sun, Feb 4, 6:45 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

A next step tomorrow could be to just destroy that instance and then deploy a new and run the tests there to see if it's dependent on the instance.

Sun, Feb 4, 6:34 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

I've looked at the instances at https://grafana.wikimedia.org/d/Vikt09zIk/performance-synthetic-server-metrics and the instance that run these tests seems to use more memory than the rest of the instances. I've rebooted the instance and will restart the tests. I'm just worried that these instabilities comes from the Hetzner cloud instance.

Sun, Feb 4, 5:34 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

Adding some more data over time. This is for Largest Contentful Paint:

Screenshot 2024-02-04 at 18.21.33.png (1×2 px, 966 KB)

Sun, Feb 4, 5:23 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter updated the task description for T356057: Verify running Firefox with and without WebPageReplay.
Sun, Feb 4, 5:17 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter closed T356085: Prepare talk for FOSDEM as Resolved.
Sun, Feb 4, 5:15 PM · Quality-and-Test-Engineering-Team
Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

Ok, this has been running for over a month now. I'm having a meeting with Gregory Mierzwinski tomorrow for some feedback. Since I've upgraded to latest WebPageReplay we have had no instability in metrics for Chrome. For Firefox we had one page that goes back and forth. Feedback from Greg is that if we only have one URL that fails, maybe we should skip that URL.

Sun, Feb 4, 5:13 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Fri, Feb 2

Jdlrobson awarded Blog Post: Web Performance at FOSDEM 4th of February a Like token.
Fri, Feb 2, 6:24 PM
Peter added a comment to T355238: Performance Team Alerts: DatasourceNoData unsilencing forthcoming.

I removed the alerts and will look at the root cause when I'm back from FOSDEM on Monday, I'll keep the task open so I don't forget.

Fri, Feb 2, 8:39 AM · Synthetic-Performance-Testing
Peter claimed T355238: Performance Team Alerts: DatasourceNoData unsilencing forthcoming.
Fri, Feb 2, 8:30 AM · Synthetic-Performance-Testing

Thu, Feb 1

Peter added a comment to T356367: Run baseline tests over a day and verify stability.

I verified using Chrome on my rooted Samsung A51, the metrics are stable. I'll add my configuration here before I close the issue.

Thu, Feb 1, 2:02 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

This has come back again for all articles tested using Chrome direct tests:

Thu, Feb 1, 5:46 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter created T356367: Run baseline tests over a day and verify stability.
Thu, Feb 1, 5:40 AM · Quality-and-Test-Engineering-Team, Performance-Device-Lab

Wed, Jan 31

Peter created Blog Post: Web Performance at FOSDEM 4th of February.
Wed, Jan 31, 6:11 PM

Mon, Jan 29

Peter closed T355793: Install and prepare a Raspberry Pi 5 for testing as Resolved.

This is done, it's much faster than the old 4.

Mon, Jan 29, 5:35 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter closed T355793: Install and prepare a Raspberry Pi 5 for testing, a subtask of T355790: Verify functionality for device lab, as Resolved.
Mon, Jan 29, 5:34 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter created T356085: Prepare talk for FOSDEM.
Mon, Jan 29, 5:32 PM · Quality-and-Test-Engineering-Team
Peter added a comment to T356057: Verify running Firefox with and without WebPageReplay.

The full trace log looks like this:

Mon, Jan 29, 2:57 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a comment to T356057: Verify running Firefox with and without WebPageReplay.

Running from the Raspberry Pi, running 11 runs, one or two of the runs get ERROR: WebDriverError: Failed to decode response from marionette. To get some more useful out of that I need to turn on the Geckodriver log.

Mon, Jan 29, 2:52 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter created T356057: Verify running Firefox with and without WebPageReplay.
Mon, Jan 29, 2:50 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab

Jan 26 2024

Peter closed T355792: Test if gnirehtet can be used to update browsers from the Play store as Resolved.

This actually works. The first time I tried it failed but after that it just works.

Jan 26 2024, 1:24 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter closed T355792: Test if gnirehtet can be used to update browsers from the Play store, a subtask of T355791: Build and verify gnirehtet, as Resolved.
Jan 26 2024, 1:24 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter created T355925: Verify that scrcpy works .
Jan 26 2024, 8:22 AM · Quality-and-Test-Engineering-Team, Performance-Device-Lab

Jan 25 2024

Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

I've reached out to Greg at Mozilla today to ask about a feedback on a couple of things. For some URLs we get changes going up and down for example (the ChatGPT page using Firefox):

Screenshot 2024-01-25 at 13.57.33.png (1×2 px, 405 KB)

Jan 25 2024, 12:58 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T355793: Install and prepare a Raspberry Pi 5 for testing.

I ordered a Raspberry Pi 5, 8gb memory, 16 gb card and will use that to verify the setup.

Jan 25 2024, 12:24 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab

Jan 24 2024

Peter closed T355791: Build and verify gnirehtet, a subtask of T355790: Verify functionality for device lab, as Resolved.
Jan 24 2024, 2:52 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter closed T355791: Build and verify gnirehtet as Resolved.

I have a new version built and I tested it and it works.

Jan 24 2024, 2:52 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a comment to T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.

It seems this was magically fixed a day ago. I did a quick check this morning and could not see anything obvious.

Jan 24 2024, 1:50 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter created T355793: Install and prepare a Raspberry Pi 5 for testing.
Jan 24 2024, 1:45 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter created T355792: Test if gnirehtet can be used to update browsers from the Play store.
Jan 24 2024, 1:45 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter created T355791: Build and verify gnirehtet.
Jan 24 2024, 1:43 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter closed T355710: Build a new version of WebPageReplay for ARM as Resolved.

The new version is built and I verified that it works.

Jan 24 2024, 1:41 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing, Performance-Device-Lab
Peter closed T355710: Build a new version of WebPageReplay for ARM, a subtask of T355790: Verify functionality for device lab, as Resolved.
Jan 24 2024, 1:41 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a subtask for T355790: Verify functionality for device lab: T355710: Build a new version of WebPageReplay for ARM.
Jan 24 2024, 1:40 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a parent task for T355710: Build a new version of WebPageReplay for ARM: T355790: Verify functionality for device lab.
Jan 24 2024, 1:40 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing, Performance-Device-Lab
Peter created T355790: Verify functionality for device lab.
Jan 24 2024, 1:40 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a comment to T355710: Build a new version of WebPageReplay for ARM.

There was a commit in December for WebPageReplay that broke building for ARM (and also for Mac). They have added brotli support for injecting JavaScript. I've override that change and built the new versions, but I need to find the correct way to report that upstream.

Jan 24 2024, 6:41 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing, Performance-Device-Lab

Jan 23 2024

Peter created T355710: Build a new version of WebPageReplay for ARM.
Jan 23 2024, 6:45 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing, Performance-Device-Lab
Peter created T355709: Investigate change in First Visual Change 2024-01-15 for Chrome.
Jan 23 2024, 6:43 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Jan 22 2024

Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

I've added the missing alerts today and updated the documentation. Would like to summarise the work in a blog post and then close this task as finished.

Jan 22 2024, 8:53 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T352187: Try out changing CPU throttling and connectivity change for WebPageReplay as Resolved.

For the new tests we run full CPU speed and no network throttling.

Jan 22 2024, 8:52 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T352187: Try out changing CPU throttling and connectivity change for WebPageReplay, a subtask of T351929: Deploy Mann Whitney U tests for WebPageReplay tests, as Resolved.
Jan 22 2024, 8:52 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T355423: Run Baseline tests once a week, a subtask of T351929: Deploy Mann Whitney U tests for WebPageReplay tests, as Resolved.
Jan 22 2024, 8:51 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T355423: Run Baseline tests once a week as Resolved.
Jan 22 2024, 8:51 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Jan 19 2024

Peter created T355423: Run Baseline tests once a week.
Jan 19 2024, 1:37 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

I've rearranged alerting and will turn them on Monday and then update documentation. The last thing to do for now would then make sure that we baseline automatically once a week (Sundays?) so that we through the week runs against a baseline that was created that Sunday. At least that is better than now where I manually makes sure we run a baseline test.

Jan 19 2024, 1:35 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T355202: Make it easier to identify why we have a performance regression. as Resolved.

This has been deployed.

Jan 19 2024, 9:47 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T355202: Make it easier to identify why we have a performance regression., a subtask of T351929: Deploy Mann Whitney U tests for WebPageReplay tests, as Resolved.
Jan 19 2024, 9:47 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T355201: Setup new server to run the Mann Whitney U tests for Firefox as Resolved.

Tests are up and running

Jan 19 2024, 9:46 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T355201: Setup new server to run the Mann Whitney U tests for Firefox, a subtask of T351929: Deploy Mann Whitney U tests for WebPageReplay tests, as Resolved.
Jan 19 2024, 9:46 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Jan 18 2024

Peter added a comment to T355201: Setup new server to run the Mann Whitney U tests for Firefox.

I installed the server today and lets start to run the tests first thing tomorrow.

Jan 18 2024, 3:02 PM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter closed T354628: New home for the performance device lab as Resolved.

I've been talking to three companies: ScaleUp, Glesys and LambdaTest. With LambdaTest we agreed it would be squeezing in our solution in there GUI-testing solution and would be the same result as with the others we tried. Glesys couldn't host our setup. ScaleUp can however. I'll be working on summarising the cost for that.

Jan 18 2024, 9:41 AM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

The next step to finish this is to deploy tests for Firefox and then also deploy a new version that more regression data in the HTML result. I plan to push the Firefox tests on Friday and then on Monday roll out the new HTML change.

Jan 18 2024, 9:34 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter added a comment to T351929: Deploy Mann Whitney U tests for WebPageReplay tests.

I've turned on the first alarm. It will fire if First Visual change has a significant change for more than 3 URLS (we test 8).

Jan 18 2024, 9:31 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Jan 17 2024

Peter added a comment to T354710: Basic.js test causing rendering of chrome page to crash on betacluster.

Sometime you can find useful information in the Chrome crash report: https://www.chromium.org/developers/crash-reports/ or it can be something we can use if we need to report a bug to Chrome.

Jan 17 2024, 2:36 PM · User-vaughnwalters, User-zeljkofilipin, WikiLambda, Abstract Wikipedia team
Peter created T355202: Make it easier to identify why we have a performance regression..
Jan 17 2024, 8:21 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing
Peter created T355201: Setup new server to run the Mann Whitney U tests for Firefox.
Jan 17 2024, 7:42 AM · Quality-and-Test-Engineering-Team, Synthetic-Performance-Testing

Jan 12 2024

Peter added a comment to T354771: Reverse tethering mobile devices.

I spent time on getting different guides to work but no luck. Next step is to try out https://github.com/catchpoint/WebPageTest.vpn-reverse-tether (or rather a version that is Open Source). But I need to build the c-code from scratch so it works on an ARM64 (the old is to 32 bit and do not work).

Jan 12 2024, 12:49 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab

Jan 10 2024

Peter created T354771: Reverse tethering mobile devices.
Jan 10 2024, 3:15 PM · Quality-and-Test-Engineering-Team, Performance-Device-Lab
Peter added a comment to T354710: Basic.js test causing rendering of chrome page to crash on betacluster.

@vaughnwalters do you get anything useful out of the log?

Jan 10 2024, 2:50 PM · User-vaughnwalters, User-zeljkofilipin, WikiLambda, Abstract Wikipedia team