Page MenuHomePhabricator
Create Task
Maniphest T318415

Add Firefox tests
Closed, ResolvedPublic
Actions

Description

I haven't pushed any Firefox tests to the continuous tests since there where some problem at the beginning. We should at least have a couple of tests, so let me start by adding one for WebPageReplay and one for first view.

Details

Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedPeterT311986 Setup test on BitBar
 ResolvedPeterT318415 Add Firefox tests

Event Timeline

Peter created this task.Sep 23 2022, 1:10 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 23 2022, 1:10 PM
Peter moved this task from Inbox, needs triage to To-do: Goals prioritized current Quarter on the Performance-Team board.Sep 26 2022, 11:20 AM
gerritbot added a comment.Sep 26 2022, 1:10 PM

Change 835176 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Add Firefox tests to our first view tests.

https://gerrit.wikimedia.org/r/835176

gerritbot added a project: Patch-For-Review.Sep 26 2022, 1:10 PM
gerritbot added a comment.Sep 26 2022, 1:21 PM

Change 835176 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Add Firefox tests to our first view tests.

https://gerrit.wikimedia.org/r/835176

Peter mentioned this in rPMSM77dd08610dc5: Add Firefox tests to our first view tests..Sep 26 2022, 1:22 PM
Maintenance_bot removed a project: Patch-For-Review.Sep 26 2022, 1:30 PM
Peter added a comment.Sep 26 2022, 5:23 PM

Oops the tests failed hard so let me remove them and try it out on another device first.

Peter moved this task from To-do: Goals prioritized current Quarter to Doing: Goals on the Performance-Team board.Oct 4 2022, 8:47 PM
Peter added a comment.Oct 4 2022, 9:04 PM

Hmm we still get these kind of things where Firefox do not start at first try:

[2022-10-04 20:53:43] INFO: Skip install HAR trigger on Android
[2022-10-04 20:54:43] INFO: firefox failed to start, trying 2 more time(s): Failed to start firefox in 60 seconds.
[2022-10-04 20:54:43] INFO: Skip install HAR trigger on Android
gerritbot added a comment.Oct 8 2022, 1:08 PM

Change 840280 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Run one URL test with WebPageReplay for Firefox.

https://gerrit.wikimedia.org/r/840280

gerritbot added a project: Patch-For-Review.Oct 8 2022, 1:08 PM
gerritbot added a comment.Oct 8 2022, 1:13 PM

Change 840280 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Run one URL test with WebPageReplay for Firefox.

https://gerrit.wikimedia.org/r/840280

Peter mentioned this in rPMSM575e9bc9996d: Run one URL test with WebPageReplay for Firefox..Oct 8 2022, 1:13 PM
gerritbot added a comment.Oct 8 2022, 1:17 PM

Change 840282 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Add one test to run on Firefox for first view tests.

https://gerrit.wikimedia.org/r/840282

gerritbot added a comment.Oct 8 2022, 1:21 PM

Change 840282 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Add one test to run on Firefox for first view tests.

https://gerrit.wikimedia.org/r/840282

Peter mentioned this in rPMSM66cf13903467: Add one test to run on Firefox for first view tests..Oct 8 2022, 1:21 PM
Maintenance_bot removed a project: Patch-For-Review.Oct 8 2022, 1:30 PM
Peter added a comment.Oct 8 2022, 1:37 PM

By default we install a browser extension that helps us adding things like requests headers on Firefox. That works fine locally but on Bitbar that seems to cause timeouts (there's some extra kb that goes over the wire to the phones). I've disabled that for our tests now and added one single Firefox tests for first view and one for WebPageReplay for now.

gerritbot added a comment.Oct 8 2022, 6:58 PM

Change 840333 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Fix correct path to the Firefox download.

https://gerrit.wikimedia.org/r/840333

gerritbot added a project: Patch-For-Review.Oct 8 2022, 6:58 PM
gerritbot added a comment.Oct 8 2022, 7:01 PM

Change 840333 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Fix correct path to the Firefox download.

https://gerrit.wikimedia.org/r/840333

Peter mentioned this in rPMSMae00b5c0182c: Fix correct path to the Firefox download..Oct 8 2022, 7:02 PM
Maintenance_bot removed a project: Patch-For-Review.Oct 8 2022, 7:30 PM
gerritbot added a comment.Oct 8 2022, 10:44 PM

Change 840349 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Do not run prewarm server for Firefox.

https://gerrit.wikimedia.org/r/840349

gerritbot added a project: Patch-For-Review.Oct 8 2022, 10:44 PM
Peter added a comment.Oct 8 2022, 10:46 PM

The good news is that the WebPageReplay tests works. There's something going on with the first view tests ...

gerritbot added a comment.Oct 8 2022, 10:46 PM

Change 840349 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Do not run prewarm server for Firefox.

https://gerrit.wikimedia.org/r/840349

Peter mentioned this in rPMSM23d4d1a60228: Do not run prewarm server for Firefox..Oct 8 2022, 10:46 PM
gerritbot added a comment.Oct 8 2022, 10:56 PM

Change 840353 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Add one more tests for Firefox.

https://gerrit.wikimedia.org/r/840353

gerritbot added a comment.Oct 8 2022, 11:01 PM

Change 840353 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Add one more tests for Firefox.

https://gerrit.wikimedia.org/r/840353

Peter mentioned this in rPMSM7233567d40b3: Add one more tests for Firefox..Oct 8 2022, 11:01 PM
Peter added a comment.EditedOct 9 2022, 6:27 AM

There's still something strange going on with the first view tests for Firefox:

[2022-10-09 04:48:31] INFO: firefox failed to start, trying 2 more time(s): Failed to decode response from marionette

Lets try to setup the configuration for Firefox from scratch.

gerritbot added a comment.Oct 9 2022, 6:28 AM

Change 840508 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Try out as clean configuration for Firefox as possible.

https://gerrit.wikimedia.org/r/840508

gerritbot added a comment.Oct 10 2022, 12:31 PM

Change 840508 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Try out as clean configuration for Firefox as possible.

https://gerrit.wikimedia.org/r/840508

Peter mentioned this in rPMSMdb526098310e: Try out as clean configuration for Firefox as possible..Oct 10 2022, 12:33 PM
gerritbot added a comment.Oct 10 2022, 4:46 PM

Change 841178 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Fix relative path to configuration file.

https://gerrit.wikimedia.org/r/841178

gerritbot added a comment.Oct 10 2022, 5:01 PM

Change 841178 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Fix relative path to configuration file.

https://gerrit.wikimedia.org/r/841178

Peter mentioned this in rPMSM2851bca2a7ae: Fix relative path to configuration file..Oct 10 2022, 5:02 PM
gerritbot added a comment.Oct 11 2022, 7:13 AM

Change 841393 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Let Firefox firstview tests report under the right keys.

https://gerrit.wikimedia.org/r/841393

gerritbot added a comment.Oct 11 2022, 7:18 AM

Change 841393 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Let Firefox firstview tests report under the right keys.

https://gerrit.wikimedia.org/r/841393

Peter mentioned this in rPMSM2826093854b9: Let Firefox firstview tests report under the right keys..Oct 11 2022, 7:19 AM
gerritbot added a comment.Oct 11 2022, 7:57 AM

Change 841447 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Add last URLs for Firefox WebPageReplay tests.

https://gerrit.wikimedia.org/r/841447

gerritbot added a comment.Oct 11 2022, 7:59 AM

Change 841447 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Add last URLs for Firefox WebPageReplay tests.

https://gerrit.wikimedia.org/r/841447

Peter mentioned this in rPMSM181b617b90ea: Add last URLs for Firefox WebPageReplay tests..Oct 11 2022, 8:00 AM
gerritbot added a comment.Oct 11 2022, 12:06 PM

Change 841482 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Disable Firefox tests for first view.

https://gerrit.wikimedia.org/r/841482

gerritbot added a comment.Oct 11 2022, 2:39 PM

Change 841482 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Disable Firefox tests for first view.

https://gerrit.wikimedia.org/r/841482

Peter mentioned this in rPMSM9ff7f08968c3: Disable Firefox tests for first view..Oct 11 2022, 2:40 PM
Peter added a comment.Oct 12 2022, 9:29 AM

For the test that do not run with WebPageReplay (on Moto G5) we get a lot of errors that looks like:

adbd    : failed to connect to socket 'tcp:c: Connection refused
gerritbot added a comment.Oct 13 2022, 2:37 PM

Change 842462 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Enable Firefox tests again.

https://gerrit.wikimedia.org/r/842462

gerritbot added a comment.Oct 13 2022, 2:38 PM

Change 842462 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Enable Firefox tests again.

https://gerrit.wikimedia.org/r/842462

Peter mentioned this in rPMSM897b88f85cb0: Enable Firefox tests again..Oct 13 2022, 2:41 PM
gerritbot added a comment.Oct 14 2022, 11:01 AM

Change 842747 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Increase number of Firefox runs and test another URL.

https://gerrit.wikimedia.org/r/842747

gerritbot added a comment.Oct 14 2022, 11:17 AM

Change 842747 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Increase number of Firefox runs and test another URL.

https://gerrit.wikimedia.org/r/842747

Peter mentioned this in rPMSM8b0b4bf84197: Increase number of Firefox runs and test another URL..Oct 14 2022, 11:17 AM
gerritbot added a comment.Oct 15 2022, 4:55 AM

Change 842885 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Fix typeo for Firefox tests.

https://gerrit.wikimedia.org/r/842885

gerritbot added a comment.Oct 15 2022, 4:58 AM

Change 842885 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Fix typeo for Firefox tests.

https://gerrit.wikimedia.org/r/842885

Peter mentioned this in rPMSMc33f27980250: Fix typeo for Firefox tests..Oct 15 2022, 4:58 AM
gerritbot added a comment.Oct 16 2022, 5:28 PM

Change 842965 had a related patch set uploaded (by Phedenskog; author: Phedenskog):

[performance/mobile-synthetic-monitoring-tests@master] Test four URLs with Firefox.

https://gerrit.wikimedia.org/r/842965

gerritbot added a comment.Oct 16 2022, 5:29 PM

Change 842965 merged by jenkins-bot:

[performance/mobile-synthetic-monitoring-tests@master] Test four URLs with Firefox.

https://gerrit.wikimedia.org/r/842965

Peter mentioned this in rPMSM589e43898036: Test four URLs with Firefox..Oct 16 2022, 5:29 PM
Peter closed this task as Resolved.Oct 17 2022, 7:32 AM

I think all the Firefox tests are there and working now.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL