Page MenuHomePhabricator
Create Task
Maniphest T402927

Investigate why increasing maxInstances for WebdriverIO causes tests to fail
Closed, InvalidPublic5 Estimated Story PointsSpike
Actions

Description

In T402397: Increase maxInstances for WebdriverIO test in core I have noticed that increasing maxInstances for WebdriverIO causes tests to fail.

AC

  • find out if the same tests fail all the time, or if there is no pattern
  • if there is a pattern, find out what is causing the failures
  • if the problem is found, create a follow up task where we try to fix the problem
  • test in core
  • test in at least one extension (examples)
  • check if updating webdriverio to the latest v9 helps
  • The questions above are answered in the comments to this ticket
  • If issues are identified, follow up tickets are created in the backlog

Related Objects
Search...

Event Timeline

zeljkofilipin created this task.Aug 26 2025, 1:53 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 26 2025, 1:53 PM
zeljkofilipin mentioned this in T402397: Increase maxInstances for WebdriverIO test in core.Aug 26 2025, 1:54 PM
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin added a parent task: T226869: Run browser tests in parallel.Aug 26 2025, 5:15 PM
Peter subscribed.Sep 1 2025, 5:13 AM
zeljkofilipin moved this task from Backlog to Ready to Refine on the Test Platform board.Sep 9 2025, 4:36 PM
Peter updated the task description. (Show Details)Sep 22 2025, 3:40 PM
Mhurd subscribed.EditedSep 22 2025, 3:40 PM

My ideas:

  • try with one instance but throttle cpu artificially by some method to see if flakiness is consistent
  • play chrome flags like the following:
Memory and Resource Management

--disable-dev-shm-usage
--memory-pressure-off
--max_old_space_size=4096
--disable-background-timer-throttling
--disable-renderer-backgrounding
--disable-features=TranslateUI
--disable-ipc-flooding-protection

GPU and Rendering

--disable-gpu
--disable-gpu-sandbox
--no-sandbox
--disable-setuid-sandbox
--disable-accelerated-2d-canvas
--disable-web-security
--disable-features=VizDisplayCompositor

Process Isolation

--disable-site-isolation-trials
--disable-features=IsolateOrigins,site-per-process
--process-per-site
--single-process

Performance and Stability

--disable-blink-features=AutomationControlled
--disable-extensions
--disable-plugins
--disable-images
--disable-javascript
--mute-audio
--no-first-run
--no-default-browser-check
--disable-background-networking
--disable-sync
--disable-translate
--metrics-recording-only
--safebrowsing-disable-auto-update
--disable-component-update

Crash Prevention

--disable-hang-monitor
--disable-prompt-on-repost
--disable-domain-reliability
--disable-features=AudioServiceOutOfProcess
--disable-print-preview
--disable-crash-reporter
--disable-breakpad
--disable-logging
Peter set the point value for this task to 5.Sep 22 2025, 3:41 PM
Peter moved this task from Ready to Refine to Refined (Discussed and Estimated) Backlog on the Test Platform board.
zeljkofilipin updated the task description. (Show Details)Sep 23 2025, 12:22 PM
Peter added a comment.Sep 24 2025, 8:11 AM

Let me know if you need help to throttle the CPU. We do that today on the performance synthetic machines. Those machines are Intel based and runs Ubuntu. I think it would be a nice feature to run all our tests on a "slow" device to be able to see tests that is dependent of a fast CPU.

Peter added a project: Browser Test Platform.Nov 18 2025, 9:44 AM
Peter moved this task from Backlog to Prioritized on the Browser Test Platform board.Nov 18 2025, 11:22 AM
zeljkofilipin renamed this task from Investigate why increasing maxInstances for WebdriverIO test in core causes tests to fail to Investigate why increasing maxInstances for WebdriverIO tests causes tests to fail.Nov 20 2025, 2:30 PM
zeljkofilipin renamed this task from Investigate why increasing maxInstances for WebdriverIO tests causes tests to fail to Investigate why increasing maxInstances for WebdriverIO causes tests to fail.Nov 20 2025, 2:32 PM
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)Nov 20 2025, 2:34 PM
AMarkossyan-WMF added a project: Spike.Nov 24 2025, 4:48 PM
AMarkossyan-WMF updated the task description. (Show Details)
Restricted Application changed the subtype of this task from "Task" to "Spike". · View Herald TranscriptNov 24 2025, 4:48 PM
AMarkossyan-WMF assigned this task to zeljkofilipin.Nov 24 2025, 4:50 PM
AMarkossyan-WMF edited projects, added Test Platform (Bashkent One 19); removed Test Platform.
Mhurd added a comment.Nov 24 2025, 4:50 PM

Generally my recollection is keeping the number of browser instances less than or equal to the number of cpu cores is best

RAM pressure is also good to keep an eye on - not exceeding available ram, as usual, is desirable

zeljkofilipin added a project: User-zeljkofilipin.Nov 25 2025, 2:00 PM
zeljkofilipin moved this task from Backlog 🪒 to Sprint 🏃‍➡️ on the User-zeljkofilipin board.
SDunlap edited projects, added Test Platform (Bashkent Two 20); removed Test Platform (Bashkent One 19).Dec 1 2025, 4:44 PM
zeljkofilipin moved this task from Prioritized to Doing on the Browser Test Platform board.Dec 2 2025, 1:29 PM
SDunlap edited projects, added Test Platform (Bashkent Jubilee 21); removed Test Platform (Bashkent Two 20).Dec 17 2025, 9:35 PM
zeljkofilipin removed zeljkofilipin as the assignee of this task.Dec 18 2025, 4:12 PM
zeljkofilipin edited projects, added Test Platform (Tkaronto Marathon 22); removed Test Platform (Bashkent Jubilee 21).
zeljkofilipin removed a project: User-zeljkofilipin.
zeljkofilipin moved this task from Doing to Prioritized on the Browser Test Platform board.
zeljkofilipin claimed this task.Jan 5 2026, 2:55 PM
zeljkofilipin added a project: User-zeljkofilipin.
zeljkofilipin removed zeljkofilipin as the assignee of this task.Jan 29 2026, 6:05 PM
zeljkofilipin edited projects, added Test Platform (Wernigerode 23); removed Test Platform (Tkaronto Marathon 22).Feb 10 2026, 3:46 PM
AMarkossyan-WMF assigned this task to zeljkofilipin.Feb 10 2026, 5:11 PM
hashar mentioned this in T417130: Implement changes for reduced CI overhead/increased speed for webdriverIO tests.Feb 11 2026, 10:33 AM
zeljkofilipin moved this task from Prioritized to Doing on the Browser Test Platform board.Feb 13 2026, 12:59 PM
zeljkofilipin removed zeljkofilipin as the assignee of this task.Feb 13 2026, 3:16 PM
zeljkofilipin edited parent tasks, added: T408361: Speed up quibble-with-gated-extensions-selenium-php83 - selenium/webdriverio; removed: T226869: Run browser tests in parallel.
zeljkofilipin removed a project: User-zeljkofilipin.
zeljkofilipin moved this task from Backlog to Done on the Test Platform (Wernigerode 23) board.Tue, Feb 17, 2:07 PM
zeljkofilipin closed this task as Invalid.Tue, Feb 17, 2:09 PM
zeljkofilipin assigned this task to Peter.

Looks like this is no longer a problem. See T414904: Increase maxInstances for Core to run webdriverIO tests in parallel.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits