Page MenuHomePhabricator
Create Task
Maniphest T430415

More flaky browser tests in MW 1.43
Open, MediumPublic
Actions

Description

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/timeline/+/1306054
https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php83-selenium/5452/console

Probably going to be similar to T429858: Flaky browser tests in MW 1.43...

17:49:44 [0-6] Error in "Page.should be re-creatable"
17:49:44 net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.7192496938994285-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:49:44 Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.7192496938994285-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:49:44     at navigate (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:155:23)
17:49:44     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
17:49:44     at async FrameManager.navigateFrame (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:130:21)
17:49:44     at async Frame.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:501:16)
17:49:44     at async Page.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:1167:16)
17:49:44     at async DevToolsDriver.navigateTo (/workspace/src/node_modules/devtools/build/commands/navigateTo.js:18:5)
17:49:44     at async Browser.wrappedCommand (/workspace/src/node_modules/devtools/build/devtoolsdriver.js:128:26)
17:49:44     at async EditPage.openTitle (/workspace/src/tests/selenium/wdio-mediawiki/Page.js:22:3)
17:49:44     at async EditPage.openForEditing (/workspace/src/tests/selenium/pageobjects/edit.page.js:36:3)
17:49:44     at async EditPage.edit (/workspace/src/tests/selenium/pageobjects/edit.page.js:58:3)
17:49:44     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:68:3)
17:49:49 [0-8] PASSED in chrome - /tests/selenium/specs/user.js
17:49:52 [0-6] Error in "Page.should be restorable"
17:49:52 net::ERR_ABORTED at http://127.0.0.1:9413//index.php?title=Special%3AUndelete%2FBeforeEach-name-0.5382024852754899-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:49:52 Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?title=Special%3AUndelete%2FBeforeEach-name-0.5382024852754899-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:49:52     at navigate (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:155:23)
17:49:52     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
17:49:52     at async FrameManager.navigateFrame (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:130:21)
17:49:52     at async Frame.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:501:16)
17:49:52     at async Page.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:1167:16)
17:49:52     at async DevToolsDriver.navigateTo (/workspace/src/node_modules/devtools/build/commands/navigateTo.js:18:5)
17:49:52     at async Browser.wrappedCommand (/workspace/src/node_modules/devtools/build/devtoolsdriver.js:128:26)
17:49:52     at async RestorePage.openTitle (/workspace/src/tests/selenium/wdio-mediawiki/Page.js:22:3)
17:49:52     at async RestorePage.restore (/workspace/src/tests/selenium/pageobjects/restore.page.js:23:3)
17:49:52     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:119:3)
17:49:57 [0-6] RETRYING in chrome - /tests/selenium/specs/page.js
17:49:58 [0-6] RUNNING in chrome - /tests/selenium/specs/page.js
17:50:13 [0-6] Error in "Page.should be deletable"
17:50:13 net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=delete&title=BeforeEach-name-0.015470048371037493-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:13 Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=delete&title=BeforeEach-name-0.015470048371037493-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:13     at navigate (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:155:23)
17:50:13     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
17:50:13     at async FrameManager.navigateFrame (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:130:21)
17:50:13     at async Frame.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:501:16)
17:50:13     at async Page.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:1167:16)
17:50:13     at async DevToolsDriver.navigateTo (/workspace/src/node_modules/devtools/build/commands/navigateTo.js:18:5)
17:50:13     at async Browser.wrappedCommand (/workspace/src/node_modules/devtools/build/devtoolsdriver.js:128:26)
17:50:13     at async DeletePage.openTitle (/workspace/src/tests/selenium/wdio-mediawiki/Page.js:22:3)
17:50:13     at async DeletePage.delete (/workspace/src/tests/selenium/pageobjects/delete.page.js:27:3)
17:50:13     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:104:3)
17:50:21 [0-6] FAILED in chrome - /tests/selenium/specs/page.js (1 retries)
17:50:21 ------------------------------------------------------------------
17:50:21 [Chrome 149.0.0.0 linux #0-6] Running: Chrome (v149.0.0.0) on linux
17:50:21 [Chrome 149.0.0.0 linux #0-6] Session ID: a4004606-90ed-4713-8c72-2c3003f03b5d
17:50:21 [Chrome 149.0.0.0 linux #0-6]
17:50:21 [Chrome 149.0.0.0 linux #0-6] » /tests/selenium/specs/page.js
17:50:21 [Chrome 149.0.0.0 linux #0-6] Page
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be previewable @daily
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be creatable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✖ should be re-creatable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be editable @daily
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should have history @daily
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be deletable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✖ should be restorable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be protectable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be undoable @daily
17:50:21 [Chrome 149.0.0.0 linux #0-6]
17:50:21 [Chrome 149.0.0.0 linux #0-6] 7 passing (24.4s)
17:50:21 [Chrome 149.0.0.0 linux #0-6] 2 failing
17:50:21 [Chrome 149.0.0.0 linux #0-6]
17:50:21 [Chrome 149.0.0.0 linux #0-6] 1) Page should be re-creatable
17:50:21 [Chrome 149.0.0.0 linux #0-6] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.7192496938994285-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:21 [Chrome 149.0.0.0 linux #0-6] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.7192496938994285-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:21 [Chrome 149.0.0.0 linux #0-6] Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.7192496938994285-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at navigate (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:155:23)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async FrameManager.navigateFrame (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:130:21)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Frame.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:501:16)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Page.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:1167:16)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async DevToolsDriver.navigateTo (/workspace/src/node_modules/devtools/build/commands/navigateTo.js:18:5)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Browser.wrappedCommand (/workspace/src/node_modules/devtools/build/devtoolsdriver.js:128:26)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async EditPage.openTitle (/workspace/src/tests/selenium/wdio-mediawiki/Page.js:22:3)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async EditPage.openForEditing (/workspace/src/tests/selenium/pageobjects/edit.page.js:36:3)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async EditPage.edit (/workspace/src/tests/selenium/pageobjects/edit.page.js:58:3)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:68:3)
17:50:21 [Chrome 149.0.0.0 linux #0-6]
17:50:21 [Chrome 149.0.0.0 linux #0-6] 2) Page should be restorable
17:50:21 [Chrome 149.0.0.0 linux #0-6] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?title=Special%3AUndelete%2FBeforeEach-name-0.5382024852754899-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:21 [Chrome 149.0.0.0 linux #0-6] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?title=Special%3AUndelete%2FBeforeEach-name-0.5382024852754899-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:21 [Chrome 149.0.0.0 linux #0-6] Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?title=Special%3AUndelete%2FBeforeEach-name-0.5382024852754899-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at navigate (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:155:23)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async FrameManager.navigateFrame (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:130:21)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Frame.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:501:16)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Page.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:1167:16)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async DevToolsDriver.navigateTo (/workspace/src/node_modules/devtools/build/commands/navigateTo.js:18:5)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Browser.wrappedCommand (/workspace/src/node_modules/devtools/build/devtoolsdriver.js:128:26)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async RestorePage.openTitle (/workspace/src/tests/selenium/wdio-mediawiki/Page.js:22:3)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async RestorePage.restore (/workspace/src/tests/selenium/pageobjects/restore.page.js:23:3)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:119:3)
17:50:21 ------------------------------------------------------------------
17:50:21 [Chrome 149.0.0.0 linux #0-6] Running: Chrome (v149.0.0.0) on linux
17:50:21 [Chrome 149.0.0.0 linux #0-6] Session ID: 5de421e9-b402-4395-8609-475e9839c6a6
17:50:21 [Chrome 149.0.0.0 linux #0-6]
17:50:21 [Chrome 149.0.0.0 linux #0-6] » /tests/selenium/specs/page.js
17:50:21 [Chrome 149.0.0.0 linux #0-6] Page
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be previewable @daily
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be creatable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be re-creatable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be editable @daily
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should have history @daily
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✖ should be deletable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be restorable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be protectable
17:50:21 [Chrome 149.0.0.0 linux #0-6]    ✓ should be undoable @daily
17:50:21 [Chrome 149.0.0.0 linux #0-6]
17:50:21 [Chrome 149.0.0.0 linux #0-6] 8 passing (22.4s)
17:50:21 [Chrome 149.0.0.0 linux #0-6] 1 failing
17:50:21 [Chrome 149.0.0.0 linux #0-6]
17:50:21 [Chrome 149.0.0.0 linux #0-6] 1) Page should be deletable
17:50:21 [Chrome 149.0.0.0 linux #0-6] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=delete&title=BeforeEach-name-0.015470048371037493-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:21 [Chrome 149.0.0.0 linux #0-6] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=delete&title=BeforeEach-name-0.015470048371037493-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:21 [Chrome 149.0.0.0 linux #0-6] Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=delete&title=BeforeEach-name-0.015470048371037493-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at navigate (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:155:23)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async FrameManager.navigateFrame (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:130:21)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Frame.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:501:16)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Page.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:1167:16)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async DevToolsDriver.navigateTo (/workspace/src/node_modules/devtools/build/commands/navigateTo.js:18:5)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Browser.wrappedCommand (/workspace/src/node_modules/devtools/build/devtoolsdriver.js:128:26)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async DeletePage.openTitle (/workspace/src/tests/selenium/wdio-mediawiki/Page.js:22:3)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async DeletePage.delete (/workspace/src/tests/selenium/pageobjects/delete.page.js:27:3)
17:50:21 [Chrome 149.0.0.0 linux #0-6]     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:104:3)

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
Zuul: Disable all browser tests on release branches except Wikibase'sJforresterintegration/configmaster+42 -30
Customize query in gerrit

Related Objects
Search...

Event Timeline

Reedy created this task.Jun 27 2026, 4:54 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 27 2026, 4:54 PM
Reedy triaged this task as High priority.Jun 27 2026, 4:54 PM
Reedy added a parent task: T430413: timeline mediawiki-node24 failure on REL1_45.
Reedy raised the priority of this task from High to Unbreak Now!.Jun 27 2026, 6:34 PM

This is seemingly blocking nearly any MW-1.43-release patches...

This is going to make the security release harder. I guess the broken tests just need skipping/removing... OR turning off quibble-for-mediawiki-core-browser-tests-only-composer-mysql-php81 completely...

Reedy added a comment.EditedJun 27 2026, 6:36 PM

https://gerrit.wikimedia.org/r/c/mediawiki/core/+/1306059

https://integration.wikimedia.org/ci/job/quibble-for-mediawiki-core-browser-tests-only-composer-mysql-php81/1497/console

19:28:38 Execution of 11 workers started at 2026-06-27T18:28:38.320Z
19:28:38 
19:28:38 [0-3] RUNNING in chrome - /tests/selenium/docs/Stack/specs/mocha.js
19:28:38 [0-2] RUNNING in chrome - /tests/selenium/docs/Stack/specs/expect.js
19:28:38 [0-0] RUNNING in chrome - /tests/selenium/docs/Create_a_simple_test/specs/specialpages.js
19:28:39 [0-1] RUNNING in chrome - /tests/selenium/docs/Page_object_pattern/specs/login.js
19:28:41 [0-3] Log in link visible
19:28:42 [0-3] PASSED in chrome - /tests/selenium/docs/Stack/specs/mocha.js
19:28:42 [0-2] PASSED in chrome - /tests/selenium/docs/Stack/specs/expect.js
19:28:42 [0-0] PASSED in chrome - /tests/selenium/docs/Create_a_simple_test/specs/specialpages.js
19:28:43 [0-4] RUNNING in chrome - /tests/selenium/docs/Stack/specs/pageobject.js
19:28:43 [0-5] RUNNING in chrome - /tests/selenium/docs/Use_MediaWiki_API/specs/api.js
19:28:43 [0-6] RUNNING in chrome - /tests/selenium/specs/page.js
19:28:43 [0-1] PASSED in chrome - /tests/selenium/docs/Page_object_pattern/specs/login.js
19:28:44 [0-5] PASSED in chrome - /tests/selenium/docs/Use_MediaWiki_API/specs/api.js
19:28:44 [0-7] RUNNING in chrome - /tests/selenium/specs/recentchanges.js
19:28:44 [0-4] PASSED in chrome - /tests/selenium/docs/Stack/specs/pageobject.js
19:28:45 [0-8] RUNNING in chrome - /tests/selenium/specs/user.js
19:28:45 [0-9] RUNNING in chrome - /tests/selenium/specs/watchlist.js
19:28:47 [0-9] PASSED in chrome - /tests/selenium/specs/watchlist.js
19:28:47 [0-10] RUNNING in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js
19:28:49 [0-7] PASSED in chrome - /tests/selenium/specs/recentchanges.js
19:28:50 [0-10] PASSED in chrome - /tests/selenium/wdio-mediawiki/specs/BlankPage.js
19:28:55 [0-6] Error in "Page.should be re-creatable"
19:28:55 net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.35690547174234855-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
19:28:55 Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.35690547174234855-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
19:28:55     at navigate (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:155:23)
19:28:55     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
19:28:55     at async FrameManager.navigateFrame (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:130:21)
19:28:55     at async Frame.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:501:16)
19:28:55     at async Page.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:1167:16)
19:28:55     at async DevToolsDriver.navigateTo (/workspace/src/node_modules/devtools/build/commands/navigateTo.js:18:5)
19:28:55     at async Browser.wrappedCommand (/workspace/src/node_modules/devtools/build/devtoolsdriver.js:128:26)
19:28:55     at async EditPage.openTitle (/workspace/src/tests/selenium/wdio-mediawiki/Page.js:22:3)
19:28:55     at async EditPage.openForEditing (/workspace/src/tests/selenium/pageobjects/edit.page.js:36:3)
19:28:55     at async EditPage.edit (/workspace/src/tests/selenium/pageobjects/edit.page.js:58:3)
19:28:55     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:68:3)
19:28:58 [0-8] Error in "User.named user should see extra signup form fields when creating an account"
19:28:58 Error: Expect $(`#wpName2`) to exist
19:28:58 
19:28:58 Expected: "exist"
19:28:58 Received: "not exist"
19:28:58     at Context.<anonymous> (/workspace/src/tests/selenium/specs/user.js:51:46)
19:28:58     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
19:29:04 [0-8] RETRYING in chrome - /tests/selenium/specs/user.js
19:29:05 [0-8] RUNNING in chrome - /tests/selenium/specs/user.js
19:29:10 [0-8] Error in "User.should be able to log in"
19:29:10 Error: expect(received).toBe(expected) // Object.is equality
19:29:10 
19:29:10 Expected: "User-0.7835345339069891-Iñtërnâtiônàlizætiøn"
19:29:10 Received: null
19:29:10     at Context.<anonymous> (/workspace/src/tests/selenium/specs/user.js:42:28)
19:29:10     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
19:29:11 [0-6] RETRYING in chrome - /tests/selenium/specs/page.js
19:29:11 [0-6] RUNNING in chrome - /tests/selenium/specs/page.js
19:29:19 [0-8] FAILED in chrome - /tests/selenium/specs/user.js (1 retries)
19:29:38 [0-6] Error in "Page.should be protectable"
19:29:38 Error: Can't call setValue on element with selector "#mwProtect-reason input" because element wasn't found
19:29:38     at async ProtectPage.protect (/workspace/src/tests/selenium/pageobjects/protect.page.js:24:3)
19:29:38     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:132:3)
19:29:39 [0-6] FAILED in chrome - /tests/selenium/specs/page.js (1 retries)
19:29:39 
19:29:39  "spec" Reporter:
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-3] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-3] Session ID: 6b51527d-bf38-4d52-a7f8-8758a08ec2dd
19:29:39 [Chrome 149.0.0.0 linux #0-3]
19:29:39 [Chrome 149.0.0.0 linux #0-3] » /tests/selenium/docs/Stack/specs/mocha.js
19:29:39 [Chrome 149.0.0.0 linux #0-3] Main page
19:29:39 [Chrome 149.0.0.0 linux #0-3]    ✓ should have "Log in" link when using mocha
19:29:39 [Chrome 149.0.0.0 linux #0-3]
19:29:39 [Chrome 149.0.0.0 linux #0-3] 1 passing (1.8s)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-2] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-2] Session ID: 611a8988-9ca9-4854-ae1d-095a4d702d88
19:29:39 [Chrome 149.0.0.0 linux #0-2]
19:29:39 [Chrome 149.0.0.0 linux #0-2] » /tests/selenium/docs/Stack/specs/expect.js
19:29:39 [Chrome 149.0.0.0 linux #0-2] Main page
19:29:39 [Chrome 149.0.0.0 linux #0-2]    ✓ should have "Log in" link when using expect
19:29:39 [Chrome 149.0.0.0 linux #0-2]
19:29:39 [Chrome 149.0.0.0 linux #0-2] 1 passing (1.9s)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-0] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-0] Session ID: 227d0b8a-6582-4783-b31c-b6e960ff8ff3
19:29:39 [Chrome 149.0.0.0 linux #0-0]
19:29:39 [Chrome 149.0.0.0 linux #0-0] » /tests/selenium/docs/Create_a_simple_test/specs/specialpages.js
19:29:39 [Chrome 149.0.0.0 linux #0-0] Special:SpecialPages
19:29:39 [Chrome 149.0.0.0 linux #0-0]    ✓ should not have Edit link
19:29:39 [Chrome 149.0.0.0 linux #0-0]
19:29:39 [Chrome 149.0.0.0 linux #0-0] 1 passing (2s)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-1] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-1] Session ID: bc7bb0b0-7c16-4f97-8fc2-be3fdd5ebb59
19:29:39 [Chrome 149.0.0.0 linux #0-1]
19:29:39 [Chrome 149.0.0.0 linux #0-1] » /tests/selenium/docs/Page_object_pattern/specs/login.js
19:29:39 [Chrome 149.0.0.0 linux #0-1] User
19:29:39 [Chrome 149.0.0.0 linux #0-1]    ✓ should be able to log in without page object
19:29:39 [Chrome 149.0.0.0 linux #0-1]
19:29:39 [Chrome 149.0.0.0 linux #0-1] 1 passing (3.7s)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-5] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-5] Session ID: 55090d8d-652f-4e0c-8a73-cbdd542876e4
19:29:39 [Chrome 149.0.0.0 linux #0-5]
19:29:39 [Chrome 149.0.0.0 linux #0-5] » /tests/selenium/docs/Use_MediaWiki_API/specs/api.js
19:29:39 [Chrome 149.0.0.0 linux #0-5] API
19:29:39 [Chrome 149.0.0.0 linux #0-5]    ✓ Main Page should exist
19:29:39 [Chrome 149.0.0.0 linux #0-5]    ✓ Missing Page should not exist
19:29:39 [Chrome 149.0.0.0 linux #0-5]
19:29:39 [Chrome 149.0.0.0 linux #0-5] 2 passing (374ms)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-4] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-4] Session ID: a01b9019-4c46-4414-9474-34494cb702f8
19:29:39 [Chrome 149.0.0.0 linux #0-4]
19:29:39 [Chrome 149.0.0.0 linux #0-4] » /tests/selenium/docs/Stack/specs/pageobject.js
19:29:39 [Chrome 149.0.0.0 linux #0-4] Main Page
19:29:39 [Chrome 149.0.0.0 linux #0-4]    ✓ should have "Log in" link when using page object
19:29:39 [Chrome 149.0.0.0 linux #0-4]
19:29:39 [Chrome 149.0.0.0 linux #0-4] 1 passing (709ms)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-9] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-9] Session ID: 2501bc70-7b62-4ae0-9b2b-3e900a87c7a9
19:29:39 [Chrome 149.0.0.0 linux #0-9]
19:29:39 [Chrome 149.0.0.0 linux #0-9] » /tests/selenium/specs/watchlist.js
19:29:39 [Chrome 149.0.0.0 linux #0-9] Special:Watchlist
19:29:39 [Chrome 149.0.0.0 linux #0-9]    - should show page with new edit @daily
19:29:39 [Chrome 149.0.0.0 linux #0-9]
19:29:39 [Chrome 149.0.0.0 linux #0-9] 1 skipped (368ms)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-7] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-7] Session ID: f00c0fd6-b988-429a-9e8c-a9b8e5553515
19:29:39 [Chrome 149.0.0.0 linux #0-7]
19:29:39 [Chrome 149.0.0.0 linux #0-7] » /tests/selenium/specs/recentchanges.js
19:29:39 [Chrome 149.0.0.0 linux #0-7] Special:RecentChanges
19:29:39 [Chrome 149.0.0.0 linux #0-7]    ✓ shows page creation
19:29:39 [Chrome 149.0.0.0 linux #0-7]
19:29:39 [Chrome 149.0.0.0 linux #0-7] 1 passing (3.6s)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-10] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-10] Session ID: f4c48209-77ab-4aea-a319-10e530613567
19:29:39 [Chrome 149.0.0.0 linux #0-10]
19:29:39 [Chrome 149.0.0.0 linux #0-10] » /tests/selenium/wdio-mediawiki/specs/BlankPage.js
19:29:39 [Chrome 149.0.0.0 linux #0-10] BlankPage
19:29:39 [Chrome 149.0.0.0 linux #0-10]    ✓ should have its title @daily
19:29:39 [Chrome 149.0.0.0 linux #0-10]
19:29:39 [Chrome 149.0.0.0 linux #0-10] 1 passing (1s)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-8] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-8] Session ID: c0ae5154-598d-4f5e-b504-2ab608619ac7
19:29:39 [Chrome 149.0.0.0 linux #0-8]
19:29:39 [Chrome 149.0.0.0 linux #0-8] » /tests/selenium/specs/user.js
19:29:39 [Chrome 149.0.0.0 linux #0-8] User
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✓ should be able to create account
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✓ should be able to log in
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✖ named user should see extra signup form fields when creating an account
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✓ temporary user should not see signup form fields relevant to named users
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✓ temporary user should be able to create account
19:29:39 [Chrome 149.0.0.0 linux #0-8]
19:29:39 [Chrome 149.0.0.0 linux #0-8] 4 passing (18s)
19:29:39 [Chrome 149.0.0.0 linux #0-8] 1 failing
19:29:39 [Chrome 149.0.0.0 linux #0-8]
19:29:39 [Chrome 149.0.0.0 linux #0-8] 1) User named user should see extra signup form fields when creating an account
19:29:39 [Chrome 149.0.0.0 linux #0-8] Expect $(`#wpName2`) to exist
19:29:39 
19:29:39 Expected: "exist"
19:29:39 Received: "not exist"
19:29:39 [Chrome 149.0.0.0 linux #0-8] Error: Expect $(`#wpName2`) to exist
19:29:39 [Chrome 149.0.0.0 linux #0-8]
19:29:39 [Chrome 149.0.0.0 linux #0-8] Expected: "exist"
19:29:39 [Chrome 149.0.0.0 linux #0-8] Received: "not exist"
19:29:39 [Chrome 149.0.0.0 linux #0-8]     at Context.<anonymous> (/workspace/src/tests/selenium/specs/user.js:51:46)
19:29:39 [Chrome 149.0.0.0 linux #0-8]     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-6] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-6] Session ID: ff91d147-fbca-4183-97d1-f3f4dad06552
19:29:39 [Chrome 149.0.0.0 linux #0-6]
19:29:39 [Chrome 149.0.0.0 linux #0-6] » /tests/selenium/specs/page.js
19:29:39 [Chrome 149.0.0.0 linux #0-6] Page
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be previewable @daily
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be creatable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✖ should be re-creatable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be editable @daily
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should have history @daily
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be deletable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be restorable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be protectable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be undoable @daily
19:29:39 [Chrome 149.0.0.0 linux #0-6]
19:29:39 [Chrome 149.0.0.0 linux #0-6] 8 passing (26.8s)
19:29:39 [Chrome 149.0.0.0 linux #0-6] 1 failing
19:29:39 [Chrome 149.0.0.0 linux #0-6]
19:29:39 [Chrome 149.0.0.0 linux #0-6] 1) Page should be re-creatable
19:29:39 [Chrome 149.0.0.0 linux #0-6] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.35690547174234855-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
19:29:39 [Chrome 149.0.0.0 linux #0-6] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.35690547174234855-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
19:29:39 [Chrome 149.0.0.0 linux #0-6] Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=submit&vehidebetadialog=1&hidewelcomedialog=1&title=BeforeEach-name-0.35690547174234855-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at navigate (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:155:23)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async FrameManager.navigateFrame (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:130:21)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async Frame.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/FrameManager.js:501:16)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async Page.goto (/workspace/src/node_modules/puppeteer-core/lib/cjs/puppeteer/common/Page.js:1167:16)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async DevToolsDriver.navigateTo (/workspace/src/node_modules/devtools/build/commands/navigateTo.js:18:5)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async Browser.wrappedCommand (/workspace/src/node_modules/devtools/build/devtoolsdriver.js:128:26)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async EditPage.openTitle (/workspace/src/tests/selenium/wdio-mediawiki/Page.js:22:3)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async EditPage.openForEditing (/workspace/src/tests/selenium/pageobjects/edit.page.js:36:3)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async EditPage.edit (/workspace/src/tests/selenium/pageobjects/edit.page.js:58:3)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:68:3)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-8] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-8] Session ID: cc8fa205-3839-47af-aa99-21d60cc18100
19:29:39 [Chrome 149.0.0.0 linux #0-8]
19:29:39 [Chrome 149.0.0.0 linux #0-8] » /tests/selenium/specs/user.js
19:29:39 [Chrome 149.0.0.0 linux #0-8] User
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✓ should be able to create account
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✖ should be able to log in
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✓ named user should see extra signup form fields when creating an account
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✓ temporary user should not see signup form fields relevant to named users
19:29:39 [Chrome 149.0.0.0 linux #0-8]    ✓ temporary user should be able to create account
19:29:39 [Chrome 149.0.0.0 linux #0-8]
19:29:39 [Chrome 149.0.0.0 linux #0-8] 4 passing (13s)
19:29:39 [Chrome 149.0.0.0 linux #0-8] 1 failing
19:29:39 [Chrome 149.0.0.0 linux #0-8]
19:29:39 [Chrome 149.0.0.0 linux #0-8] 1) User should be able to log in
19:29:39 [Chrome 149.0.0.0 linux #0-8] expect(received).toBe(expected) // Object.is equality
19:29:39 
19:29:39 Expected: "User-0.7835345339069891-Iñtërnâtiônàlizætiøn"
19:29:39 Received: null
19:29:39 [Chrome 149.0.0.0 linux #0-8] Error: expect(received).toBe(expected) // Object.is equality
19:29:39 [Chrome 149.0.0.0 linux #0-8]
19:29:39 [Chrome 149.0.0.0 linux #0-8] Expected: "User-0.7835345339069891-Iñtërnâtiônàlizætiøn"
19:29:39 [Chrome 149.0.0.0 linux #0-8] Received: null
19:29:39 [Chrome 149.0.0.0 linux #0-8]     at Context.<anonymous> (/workspace/src/tests/selenium/specs/user.js:42:28)
19:29:39 [Chrome 149.0.0.0 linux #0-8]     at process.processTicksAndRejections (node:internal/process/task_queues:104:5)
19:29:39 ------------------------------------------------------------------
19:29:39 [Chrome 149.0.0.0 linux #0-6] Running: Chrome (v149.0.0.0) on linux
19:29:39 [Chrome 149.0.0.0 linux #0-6] Session ID: f9341846-3d09-40c9-85f0-86d36f3a3dce
19:29:39 [Chrome 149.0.0.0 linux #0-6]
19:29:39 [Chrome 149.0.0.0 linux #0-6] » /tests/selenium/specs/page.js
19:29:39 [Chrome 149.0.0.0 linux #0-6] Page
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be previewable @daily
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be creatable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be re-creatable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be editable @daily
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should have history @daily
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be deletable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be restorable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✖ should be protectable
19:29:39 [Chrome 149.0.0.0 linux #0-6]    ✓ should be undoable @daily
19:29:39 [Chrome 149.0.0.0 linux #0-6]
19:29:39 [Chrome 149.0.0.0 linux #0-6] 8 passing (27.1s)
19:29:39 [Chrome 149.0.0.0 linux #0-6] 1 failing
19:29:39 [Chrome 149.0.0.0 linux #0-6]
19:29:39 [Chrome 149.0.0.0 linux #0-6] 1) Page should be protectable
19:29:39 [Chrome 149.0.0.0 linux #0-6] Can't call setValue on element with selector "#mwProtect-reason input" because element wasn't found
19:29:39 [Chrome 149.0.0.0 linux #0-6] Error: Can't call setValue on element with selector "#mwProtect-reason input" because element wasn't found
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async ProtectPage.protect (/workspace/src/tests/selenium/pageobjects/protect.page.js:24:3)
19:29:39 [Chrome 149.0.0.0 linux #0-6]     at async Context.<anonymous> (/workspace/src/tests/selenium/specs/page.js:132:3)
19:29:39 
19:29:39 
19:29:39 Spec Files:	 9 passed, 2 retries, 2 failed, 11 total (100% completed) in 00:01:01
SomeRandomDeveloper subscribed.Jun 27 2026, 9:57 PM
gerritbot added a comment.Jul 3 2026, 10:29 AM

Change #1307374 had a related patch set uploaded (by Jforrester; author: Jforrester):

[integration/config@master] Zuul: Disable all browser tests on release branches except Wikibase's

https://gerrit.wikimedia.org/r/1307374

gerritbot added a project: Patch-For-Review.Jul 3 2026, 10:29 AM
zeljkofilipin added projects: QS-Test-Automation, User-zeljkofilipin.Jul 3 2026, 12:23 PM
zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.
zeljkofilipin subscribed.
zeljkofilipin added a comment.Jul 3 2026, 12:29 PM

I'll try to find out how flaky selenium tests are in the main branch. (First for core, later for all gated repositories.) Then, I'll try to find out how flaky tests are in 1.43.

zeljkofilipin claimed this task.Jul 3 2026, 12:29 PM
zeljkofilipin added a comment.EditedJul 3 2026, 12:32 PM

Similar to T418778#12053976, I'll use Quibble-Local.

For the main branch, run tests first 10 times, then 100.

Setup.

git clone https://gitlab.wikimedia.org/zfilipin/quibble-local.git
cd quibble-local/ 
./fresh_install

Run 10 times.

RUNS=10 ./find_flaky_selenium_tests

Full output.

P94719 RUNS=10 ./find_flaky_selenium_tests
1~/Documents/gitlab/zfilipin/quibble-local$ TIME_ELAPSED=1 RUNS=10 ./find_flaky_selenium_tests
2logs: log/2026-07-10-15-40-15/
3
4OS: Linux 6.14.0-37-generic
5CPU: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz (4 cores)
6RAM: 15.6 GB
7bash: 5.2.21(1)-release
8git: git version 2.43.0
9docker: Docker version 29.6.1, build 8900f1d
10docker CPUs: 4
11docker RAM: 15.6 GB
12
13Running ./run_selenium_tests 10 times
14run-1 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 12s)
15run-2 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 2s)
16run-3 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 5s)
17run-4 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 10s)
18run-5 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 12s)
19run-6 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 8s)
20run-7 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 9s)
21run-8 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 11s)
22run-9 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 10s)
23run-10 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... (1m 5s)
24
25========================================
26= Test results across 10 runs
27========================================
28
29Flaky tests (passed in some runs, failed in others):
30 (none)
31
32Consistently failing tests (failed in every run — likely real failures, not flakes):
33 (none)
34
35Tests that passed in every run:
36 10 passed: chrome.149_0_7827_196.linux.Page: should be previewable
37 10 passed: chrome.149_0_7827_196.linux.Page: should be creatable
38 10 passed: chrome.149_0_7827_196.linux.Page: should be re creatable
39 10 passed: chrome.149_0_7827_196.linux.Page: should be editable
40 10 passed: chrome.149_0_7827_196.linux.Page: should have history
41 10 passed: chrome.149_0_7827_196.linux.Page: should be undoable
42 10 passed: chrome.149_0_7827_196.linux.Special:RecentChanges: shows page creation
43 10 passed: chrome.149_0_7827_196.linux.Page_admin_actions: should be deletable
44 10 passed: chrome.149_0_7827_196.linux.Page_admin_actions: should be restorable
45 10 passed: chrome.149_0_7827_196.linux.Page_admin_actions: should be protectable
46 10 passed: chrome.149_0_7827_196.linux.Temporary_user: should not see signup form fields relevant to named users
47 10 passed: chrome.149_0_7827_196.linux.Temporary_user_account_creation: should be able to create account
48 10 passed: chrome.149_0_7827_196.linux.User: should be able to create account
49 10 passed: chrome.149_0_7827_196.linux.User: should be able to log in
50 10 passed: chrome.149_0_7827_196.linux.User: named user should see extra signup form fields when creating an account
51 10 passed: chrome.149_0_7827_196.linux.User: should be able to block a user
52 10 passed: chrome.149_0_7827_196.linux.Special:SpecialPages: should not have Edit link
53 10 passed: chrome.149_0_7827_196.linux.User: should be able to log in without page object
54 10 passed: chrome.149_0_7827_196.linux.Main_page: should have edit link when using expect
55 10 passed: chrome.149_0_7827_196.linux.Main_page: should have edit link when using mocha
56 10 passed: chrome.149_0_7827_196.linux.Main_Page: should have edit link when using page object
57 10 passed: chrome.149_0_7827_196.linux.API: Main Page should exist
58 10 passed: chrome.149_0_7827_196.linux.API: Missing Page should not exist
59 10 passed: chrome.149_0_7827_196.linux.BlankPage: should have its title
60
61Tests skipped in every run:
62 (none)
63
64Per-run result files: log/2026-07-10-15-40-15/find_flaky_selenium_tests/run-N/
65
66========================================
67= Results
68========================================
69PASSED:
70 run-1
71 run-2
72 run-3
73 run-4
74 run-5
75 run-6
76 run-7
77 run-8
78 run-9
79 run-10
80
81All passed.
82(11m 24s)

Run 100 times.

RUNS=100 ./find_flaky_selenium_tests

Full output.

P94722 RUNS=100 ./find_flaky_selenium_tests
1~/Documents/gitlab/zfilipin/quibble-local$ RUNS=100 ./find_flaky_selenium_tests
2use VERBOSE=1 for full output
3logs: log/silent/
4
5OS: Linux 6.14.0-37-generic
6CPU: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz (4 cores)
7RAM: 15.6 GB
8bash: 5.2.21(1)-release
9git: git version 2.43.0
10docker: Docker version 29.6.1, build 8900f1d
11docker CPUs: 4
12docker RAM: 15.6 GB
13
14Running ./run_selenium_tests 100 times
15run-1 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
16run-2 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
17run-3 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
18run-4 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
19run-5 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
20run-6 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
21run-7 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
22run-8 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
23run-9 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
24run-10 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
25run-11 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
26run-12 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
27run-13 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
28run-14 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
29run-15 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
30run-16 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
31run-17 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
32run-18 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
33run-19 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
34run-20 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
35run-21 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
36run-22 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
37run-23 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
38run-24 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
39run-25 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
40run-26 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
41run-27 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
42run-28 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
43run-29 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
44run-30 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
45run-31 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
46run-32 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
47run-33 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
48run-34 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
49run-35 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
50run-36 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
51run-37 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
52run-38 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
53run-39 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
54run-40 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
55run-41 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
56run-42 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
57run-43 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
58run-44 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
59run-45 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
60run-46 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
61run-47 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
62run-48 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
63run-49 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
64run-50 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
65run-51 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
66run-52 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
67run-53 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
68run-54 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
69run-55 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
70run-56 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
71run-57 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
72run-58 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
73run-59 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
74run-60 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
75run-61 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
76run-62 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
77run-63 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
78run-64 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
79run-65 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
80run-66 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
81run-67 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
82run-68 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
83run-69 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
84run-70 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
85run-71 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
86run-72 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
87run-73 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
88run-74 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
89run-75 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
90run-76 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
91run-77 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
92run-78 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
93run-79 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
94run-80 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
95run-81 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
96run-82 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
97run-83 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
98run-84 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
99run-85 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
100run-86 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
101run-87 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
102run-88 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
103run-89 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
104run-90 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
105run-91 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
106run-92 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
107run-93 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
108run-94 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
109run-95 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
110run-96 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
111run-97 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
112run-98 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
113run-99 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
114run-100 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................
115
116========================================
117= Test results across 100 runs
118========================================
119
120Flaky tests (passed in some runs, failed in others):
121 (none)
122
123Consistently failing tests (failed in every run — likely real failures, not flakes):
124 (none)
125
126Tests that passed in every run:
127 100 passed: chrome.149_0_7827_155.linux.Page: should be previewable
128 100 passed: chrome.149_0_7827_155.linux.Page: should be creatable
129 100 passed: chrome.149_0_7827_155.linux.Page: should be re creatable
130 100 passed: chrome.149_0_7827_155.linux.Page: should be editable
131 100 passed: chrome.149_0_7827_155.linux.Page: should have history
132 100 passed: chrome.149_0_7827_155.linux.Page: should be undoable
133 100 passed: chrome.149_0_7827_155.linux.Special:RecentChanges: shows page creation
134 100 passed: chrome.149_0_7827_155.linux.Page_admin_actions: should be deletable
135 100 passed: chrome.149_0_7827_155.linux.Page_admin_actions: should be restorable
136 100 passed: chrome.149_0_7827_155.linux.Page_admin_actions: should be protectable
137 100 passed: chrome.149_0_7827_155.linux.Temporary_user: should not see signup form fields relevant to named users
138 100 passed: chrome.149_0_7827_155.linux.Temporary_user_account_creation: should be able to create account
139 100 passed: chrome.149_0_7827_155.linux.User: should be able to create account
140 100 passed: chrome.149_0_7827_155.linux.User: should be able to log in
141 100 passed: chrome.149_0_7827_155.linux.User: named user should see extra signup form fields when creating an account
142 100 passed: chrome.149_0_7827_155.linux.User: should be able to block a user
143 100 passed: chrome.149_0_7827_155.linux.Special:SpecialPages: should not have Edit link
144 100 passed: chrome.149_0_7827_155.linux.User: should be able to log in without page object
145 100 passed: chrome.149_0_7827_155.linux.Main_page: should have edit link when using expect
146 100 passed: chrome.149_0_7827_155.linux.Main_page: should have edit link when using mocha
147 100 passed: chrome.149_0_7827_155.linux.Main_Page: should have edit link when using page object
148 100 passed: chrome.149_0_7827_155.linux.API: Main Page should exist
149 100 passed: chrome.149_0_7827_155.linux.API: Missing Page should not exist
150 100 passed: chrome.149_0_7827_155.linux.BlankPage: should have its title
151
152Tests skipped in every run:
153 (none)
154
155Per-run result files: log/find_flaky_selenium_tests/run-N/
156
157========================================
158= Results
159========================================
160PASSED:
161 run-1
162 run-2
163 run-3
164 run-4
165 run-5
166 run-6
167 run-7
168 run-8
169 run-9
170 run-10
171 run-11
172 run-12
173 run-13
174 run-14
175 run-15
176 run-16
177 run-17
178 run-18
179 run-19
180 run-20
181 run-21
182 run-22
183 run-23
184 run-24
185 run-25
186 run-26
187 run-27
188 run-28
189 run-29
190 run-30
191 run-31
192 run-32
193 run-33
194 run-34
195 run-35
196 run-36
197 run-37
198 run-38
199 run-39
200 run-40
201 run-41
202 run-42
203 run-43
204 run-44
205 run-45
206 run-46
207 run-47
208 run-48
209 run-49
210 run-50
211 run-51
212 run-52
213 run-53
214 run-54
215 run-55
216 run-56
217 run-57
218 run-58
219 run-59
220 run-60
221 run-61
222 run-62
223 run-63
224 run-64
225 run-65
226 run-66
227 run-67
228 run-68
229 run-69
230 run-70
231 run-71
232 run-72
233 run-73
234 run-74
235 run-75
236 run-76
237 run-77
238 run-78
239 run-79
240 run-80
241 run-81
242 run-82
243 run-83
244 run-84
245 run-85
246 run-86
247 run-87
248 run-88
249 run-89
250 run-90
251 run-91
252 run-92
253 run-93
254 run-94
255 run-95
256 run-96
257 run-97
258 run-98
259 run-99
260 run-100
261
262All passed.

zeljkofilipin added a comment.EditedJul 3 2026, 12:57 PM

For the 1.43 branch, run tests first 10 times, then 100.

Setup.

BRANCH="REL1_43" ./fresh_install

Run 10 times.

RUNS=10 ./find_flaky_selenium_tests

Full output.

P94720 RUNS=10 ./find_flaky_selenium_tests
1~/Documents/gitlab/zfilipin/quibble-local$ TIME_ELAPSED=1 RUNS=10 ./find_flaky_selenium_tests
2logs: log/2026-07-10-15-55-17/
3
4OS: Linux 6.14.0-37-generic
5CPU: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz (4 cores)
6RAM: 15.6 GB
7bash: 5.2.21(1)-release
8git: git version 2.43.0
9docker: Docker version 29.6.1, build 8900f1d
10docker CPUs: 4
11docker RAM: 15.6 GB
12
13Running ./run_selenium_tests 10 times
14run-1 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL (1m 36s)
15run-2 run_selenium_tests ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL (1m 29s)
16run-3 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL (1m 36s)
17run-4 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL (1m 41s)
18run-5 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL (1m 40s)
19run-6 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL (1m 44s)
20run-7 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL (1m 41s)
21run-8 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL (1m 20s)
22run-9 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL (1m 40s)
23run-10 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL (1m 40s)
24
25========================================
26= Test results across 10 runs
27========================================
28
29Flaky tests (passed in some runs, failed in others):
30 16 passed, 4 failed (failed in runs 2 5 7 8): chrome.149_0_0_0.linux.Page: should be previewable @daily
31 12 passed, 8 failed (failed in runs 1 2 3 5 7 8 9 10): chrome.149_0_0_0.linux.Page: should be creatable
32 5 passed, 15 failed (failed in runs 1 2 3 4 5 6 7 8 9 10): chrome.149_0_0_0.linux.Page: should be re creatable
33 2 passed, 18 failed (failed in runs 1 2 3 4 5 6 7 8 9 10): chrome.149_0_0_0.linux.Page: should be deletable
34 3 passed, 17 failed (failed in runs 1 2 3 4 5 6 7 8 9 10): chrome.149_0_0_0.linux.Page: should be restorable
35 2 passed, 18 failed (failed in runs 1 2 3 4 5 6 7 8 9 10): chrome.149_0_0_0.linux.Page: should be protectable
36 10 passed, 9 failed (failed in runs 1 2 4 5 6 7 9 10): chrome.149_0_0_0.linux.User: should be able to log in
37 6 passed, 13 failed (failed in runs 1 2 3 4 5 6 7 9 10): chrome.149_0_0_0.linux.User: named user should see extra signup form fields when creating an account
38 18 passed, 1 failed (failed in run 9): chrome.149_0_0_0.linux.User: temporary user should be able to create account
39
40Consistently failing tests (failed in every run — likely real failures, not flakes):
41 (none)
42
43Tests that passed in every run:
44 10 passed: chrome.149_0_0_0.linux.User: should be able to log in without page object
45 10 passed: chrome.149_0_0_0.linux.Main_page: should have Log in link when using mocha
46 10 passed: chrome.149_0_0_0.linux.Main_page: should have Log in link when using expect
47 10 passed: chrome.149_0_0_0.linux.Special:SpecialPages: should not have Edit link
48 10 passed: chrome.149_0_0_0.linux.Main_Page: should have Log in link when using page object
49 10 passed: chrome.149_0_0_0.linux.API: Main Page should exist
50 10 passed: chrome.149_0_0_0.linux.API: Missing Page should not exist
51 20 passed: chrome.149_0_0_0.linux.Page: should be editable @daily
52 20 passed: chrome.149_0_0_0.linux.Page: should have history @daily
53 20 passed: chrome.149_0_0_0.linux.Page: should be undoable @daily
54 10 passed: chrome.149_0_0_0.linux.Special:RecentChanges: shows page creation
55 19 passed: chrome.149_0_0_0.linux.User: should be able to create account
56 19 passed: chrome.149_0_0_0.linux.User: temporary user should not see signup form fields relevant to named users
57 10 passed: chrome.149_0_0_0.linux.BlankPage: should have its title @daily
58
59Tests skipped in every run:
60 10 skipped: chrome.149_0_0_0.linux.Special:Watchlist: should show page with new edit @daily
61
62Per-run result files: log/2026-07-10-15-55-17/find_flaky_selenium_tests/run-N/
63
64========================================
65= Results
66========================================
67FAILED:
68 run-1 (run_selenium_tests)
69 run-2 (run_selenium_tests)
70 run-3 (run_selenium_tests)
71 run-4 (run_selenium_tests)
72 run-5 (run_selenium_tests)
73 run-6 (run_selenium_tests)
74 run-7 (run_selenium_tests)
75 run-8 (run_selenium_tests)
76 run-9 (run_selenium_tests)
77 run-10 (run_selenium_tests)
78
79Logs: log/2026-07-10-15-55-17/
80(16m 7s)

Run 100 times.

RUNS=100 ./find_flaky_selenium_tests

Full output.

P94727 RUNS=100 ./find_flaky_selenium_tests
1~/Documents/gitlab/zfilipin/quibble-local$ RUNS=100 ./find_flaky_selenium_tests
2use VERBOSE=1 for full output
3logs: log/silent/
4
5OS: Linux 6.14.0-37-generic
6CPU: Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz (4 cores)
7RAM: 15.6 GB
8bash: 5.2.21(1)-release
9git: git version 2.43.0
10docker: Docker version 29.6.1, build 8900f1d
11docker CPUs: 4
12docker RAM: 15.6 GB
13
14Running ./run_selenium_tests 100 times
15run-1 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
16run-2 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
17run-3 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
18run-4 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
19run-5 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
20run-6 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
21run-7 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
22run-8 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
23run-9 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
24run-10 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
25run-11 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
26run-12 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
27run-13 run_selenium_tests ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
28run-14 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
29run-15 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
30run-16 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
31run-17 run_selenium_tests .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
32run-18 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
33run-19 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
34run-20 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
35run-21 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
36run-22 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
37run-23 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
38run-24 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
39run-25 run_selenium_tests .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
40run-26 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
41run-27 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
42run-28 run_selenium_tests ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
43run-29 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
44run-30 run_selenium_tests ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
45run-31 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
46run-32 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
47run-33 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
48run-34 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
49run-35 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
50run-36 run_selenium_tests .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
51run-37 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
52run-38 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
53run-39 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
54run-40 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
55run-41 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
56run-42 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
57run-43 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
58run-44 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
59run-45 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
60run-46 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
61run-47 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
62run-48 run_selenium_tests ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
63run-49 run_selenium_tests .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
64run-50 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
65run-51 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
66run-52 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
67run-53 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
68run-54 run_selenium_tests .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
69run-55 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
70run-56 run_selenium_tests ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
71run-57 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
72run-58 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
73run-59 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
74run-60 run_selenium_tests ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
75run-61 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
76run-62 run_selenium_tests ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
77run-63 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
78run-64 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
79run-65 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
80run-66 run_selenium_tests ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
81run-67 run_selenium_tests ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
82run-68 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
83run-69 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
84run-70 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
85run-71 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
86run-72 run_selenium_tests .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
87run-73 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
88run-74 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
89run-75 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
90run-76 run_selenium_tests ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
91run-77 run_selenium_tests ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
92run-78 run_selenium_tests ..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
93run-79 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
94run-80 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
95run-81 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
96run-82 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
97run-83 run_selenium_tests .............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
98run-84 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
99run-85 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
100run-86 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
101run-87 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
102run-88 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
103run-89 run_selenium_tests .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
104run-90 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
105run-91 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
106run-92 run_selenium_tests ............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
107run-93 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
108run-94 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
109run-95 run_selenium_tests .................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................. FAIL
110run-96 run_selenium_tests ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ FAIL
111run-97 run_selenium_tests ...................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
112run-98 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
113run-99 run_selenium_tests ................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
114run-100 run_selenium_tests .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... FAIL
115
116========================================
117= Test results across 100 runs
118========================================
119
120Flaky tests (passed in some runs, failed in others):
121 152 passed, 48 failed (failed in runs 1 2 5 7 9 10 11 12 15 16 17 20 21 23 27 29 30 32 37 38 40 42 46 47 49 50 52 54 56 57 58 59 60 63 65 72 74 75 77 85 86 87 88 92 95 97 98): chrome.149_0_0_0.linux.Page: should be previewable @daily
122 105 passed, 95 failed (failed in runs 1 2 6 7 9 10 11 12 13 14 15 16 17 19 20 21 22 23 26 28 29 31 33 34 35 36 37 38 39 40 41 44 45 46 47 49 50 52 53 54 55 56 57 58 59 60 61 63 64 65 66 67 69 70 71 72 73 74 75 77 78 79 80 81 82 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100): chrome.149_0_0_0.linux.Page: should be creatable
123 36 passed, 164 failed (failed in runs 1 2 3 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 90 91 92 93 94 96 97 98 99 100): chrome.149_0_0_0.linux.Page: should be re creatable
124 24 passed, 176 failed (failed in runs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100): chrome.149_0_0_0.linux.Page: should be deletable
125 26 passed, 174 failed (failed in runs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100): chrome.149_0_0_0.linux.Page: should be restorable
126 22 passed, 178 failed (failed in runs 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100): chrome.149_0_0_0.linux.Page: should be protectable
127 87 passed, 93 failed (failed in runs 3 4 5 6 8 11 12 13 14 16 17 18 20 21 23 24 25 26 27 28 29 30 31 32 33 34 37 38 39 41 42 43 44 45 48 49 50 51 53 54 55 56 59 61 62 63 64 65 66 67 68 71 72 75 76 77 78 79 80 81 82 83 85 86 87 88 90 91 92 93 94 95 98 99 100): chrome.149_0_0_0.linux.User: should be able to log in
128 60 passed, 120 failed (failed in runs 1 2 3 5 6 8 11 12 13 14 15 16 17 18 20 21 23 24 25 26 27 28 29 30 31 32 34 37 38 39 41 42 43 44 45 48 49 50 51 53 54 55 56 57 59 60 61 62 63 64 65 66 67 68 72 75 76 77 78 79 80 81 82 83 85 86 87 88 90 91 92 93 94 95 98 99 100): chrome.149_0_0_0.linux.User: named user should see extra signup form fields when creating an account
129 160 passed, 20 failed (failed in runs 3 11 16 24 29 30 33 39 44 56 61 66 72 79 80 87 88 93 94): chrome.149_0_0_0.linux.User: temporary user should be able to create account
130
131Consistently failing tests (failed in every run — likely real failures, not flakes):
132 (none)
133
134Tests that passed in every run:
135 100 passed: chrome.149_0_0_0.linux.Special:SpecialPages: should not have Edit link
136 100 passed: chrome.149_0_0_0.linux.User: should be able to log in without page object
137 100 passed: chrome.149_0_0_0.linux.Main_page: should have Log in link when using expect
138 100 passed: chrome.149_0_0_0.linux.Main_page: should have Log in link when using mocha
139 100 passed: chrome.149_0_0_0.linux.Main_Page: should have Log in link when using page object
140 100 passed: chrome.149_0_0_0.linux.API: Main Page should exist
141 100 passed: chrome.149_0_0_0.linux.API: Missing Page should not exist
142 200 passed: chrome.149_0_0_0.linux.Page: should be editable @daily
143 200 passed: chrome.149_0_0_0.linux.Page: should have history @daily
144 200 passed: chrome.149_0_0_0.linux.Page: should be undoable @daily
145 100 passed: chrome.149_0_0_0.linux.Special:RecentChanges: shows page creation
146 180 passed: chrome.149_0_0_0.linux.User: should be able to create account
147 180 passed: chrome.149_0_0_0.linux.User: temporary user should not see signup form fields relevant to named users
148 100 passed: chrome.149_0_0_0.linux.BlankPage: should have its title @daily
149
150Tests skipped in every run:
151 100 skipped: chrome.149_0_0_0.linux.Special:Watchlist: should show page with new edit @daily
152
153Per-run result files: log/find_flaky_selenium_tests/run-N/
154
155========================================
156= Results
157========================================
158FAILED:
159 run-1 (run_selenium_tests)
160 run-2 (run_selenium_tests)
161 run-3 (run_selenium_tests)
162 run-4 (run_selenium_tests)
163 run-5 (run_selenium_tests)
164 run-6 (run_selenium_tests)
165 run-7 (run_selenium_tests)
166 run-8 (run_selenium_tests)
167 run-9 (run_selenium_tests)
168 run-10 (run_selenium_tests)
169 run-11 (run_selenium_tests)
170 run-12 (run_selenium_tests)
171 run-13 (run_selenium_tests)
172 run-14 (run_selenium_tests)
173 run-15 (run_selenium_tests)
174 run-16 (run_selenium_tests)
175 run-17 (run_selenium_tests)
176 run-18 (run_selenium_tests)
177 run-19 (run_selenium_tests)
178 run-20 (run_selenium_tests)
179 run-21 (run_selenium_tests)
180 run-22 (run_selenium_tests)
181 run-23 (run_selenium_tests)
182 run-24 (run_selenium_tests)
183 run-25 (run_selenium_tests)
184 run-26 (run_selenium_tests)
185 run-27 (run_selenium_tests)
186 run-28 (run_selenium_tests)
187 run-29 (run_selenium_tests)
188 run-30 (run_selenium_tests)
189 run-31 (run_selenium_tests)
190 run-32 (run_selenium_tests)
191 run-33 (run_selenium_tests)
192 run-34 (run_selenium_tests)
193 run-35 (run_selenium_tests)
194 run-36 (run_selenium_tests)
195 run-37 (run_selenium_tests)
196 run-38 (run_selenium_tests)
197 run-39 (run_selenium_tests)
198 run-40 (run_selenium_tests)
199 run-41 (run_selenium_tests)
200 run-42 (run_selenium_tests)
201 run-43 (run_selenium_tests)
202 run-44 (run_selenium_tests)
203 run-45 (run_selenium_tests)
204 run-46 (run_selenium_tests)
205 run-47 (run_selenium_tests)
206 run-48 (run_selenium_tests)
207 run-49 (run_selenium_tests)
208 run-50 (run_selenium_tests)
209 run-51 (run_selenium_tests)
210 run-52 (run_selenium_tests)
211 run-53 (run_selenium_tests)
212 run-54 (run_selenium_tests)
213 run-55 (run_selenium_tests)
214 run-56 (run_selenium_tests)
215 run-57 (run_selenium_tests)
216 run-58 (run_selenium_tests)
217 run-59 (run_selenium_tests)
218 run-60 (run_selenium_tests)
219 run-61 (run_selenium_tests)
220 run-62 (run_selenium_tests)
221 run-63 (run_selenium_tests)
222 run-64 (run_selenium_tests)
223 run-65 (run_selenium_tests)
224 run-66 (run_selenium_tests)
225 run-67 (run_selenium_tests)
226 run-68 (run_selenium_tests)
227 run-69 (run_selenium_tests)
228 run-70 (run_selenium_tests)
229 run-71 (run_selenium_tests)
230 run-72 (run_selenium_tests)
231 run-73 (run_selenium_tests)
232 run-74 (run_selenium_tests)
233 run-75 (run_selenium_tests)
234 run-76 (run_selenium_tests)
235 run-77 (run_selenium_tests)
236 run-78 (run_selenium_tests)
237 run-79 (run_selenium_tests)
238 run-80 (run_selenium_tests)
239 run-81 (run_selenium_tests)
240 run-82 (run_selenium_tests)
241 run-83 (run_selenium_tests)
242 run-84 (run_selenium_tests)
243 run-85 (run_selenium_tests)
244 run-86 (run_selenium_tests)
245 run-87 (run_selenium_tests)
246 run-88 (run_selenium_tests)
247 run-89 (run_selenium_tests)
248 run-90 (run_selenium_tests)
249 run-91 (run_selenium_tests)
250 run-92 (run_selenium_tests)
251 run-93 (run_selenium_tests)
252 run-94 (run_selenium_tests)
253 run-95 (run_selenium_tests)
254 run-96 (run_selenium_tests)
255 run-97 (run_selenium_tests)
256 run-98 (run_selenium_tests)
257 run-99 (run_selenium_tests)
258 run-100 (run_selenium_tests)
259
260Logs: log/silent/

gerritbot added a comment.Jul 10 2026, 11:55 AM

Change #1307374 merged by jenkins-bot:

[integration/config@master] Zuul: Disable all browser tests on release branches except Wikibase's

https://gerrit.wikimedia.org/r/1307374

Stashbot added a comment.Jul 10 2026, 11:56 AM

Mentioned in SAL (#wikimedia-releng) [2026-07-10T11:56:08Z] <James_F> Zuul: Disable all browser tests on release branches except Wikibase's, for T430415

Maintenance_bot removed a project: Patch-For-Review.Jul 10 2026, 12:31 PM
zeljkofilipin removed zeljkofilipin as the assignee of this task.Jul 10 2026, 2:18 PM

When running tests on the main branch, there are no failures, even when the tests are running 10 or 100 times in a row.

When running tests on REL1_43 there's a failure every single time.

I'll mention this during our team meeting, to see if we can spend more time on this.

zeljkofilipin removed a project: User-zeljkofilipin.Jul 10 2026, 2:20 PM
zeljkofilipin added a comment.Jul 10 2026, 2:24 PM

@Reedy if this is important and/or urgent, please leave a comment here, or ping the team on slack (#qs-test-automation).

Reedy added a comment.Jul 10 2026, 2:31 PM

How important/urgent this is, is hard to define.

REL1_43 was branched... October/November 2024, released in December 2024, and we have to support it until December 2027.

It's obviously diverged a lot from master in those ~18 months.

That means, based on quarterly releases, we're going to have to do at least 6 more releases for this. CI failing makes this a harder, more drawn out process, often force merging patches to be able to make progress.

Antoine seems to have merged James' patch, which hopefully unblocks this more widely, but it does raise the wider/larger question of the value of the tests, and more so, what value they have on release branches. Should they be running (and passing?) on release branches etc.

• SLong-WMF moved this task from Backlog to Refined on the QS-Test-Automation board.Jul 16 2026, 2:40 PM
• SLong-WMF subscribed.Jul 16 2026, 3:00 PM

Hey folks!

The impact to our dev process seems clear: if we don't fix this, it will cost us additional time through 2027 - which then indicates lost opportunity cost as others are distracted from primary-branch work or other things.

What's less clear is the impact to our users. Do these tests cover functionality that appears in Wikipedia or other specific projects? If so, how heavily used are those projects? Given our limited capacity and the amount of ground we need to cover, tolerating this inefficiency may be the right call for now. This could also help set precedence for what QS-Test-Automation (Željko, Vaughn, Emeka and Esther from the QS team) help support in the future.

What would be useful is to know what user-facing functionality this testing protects, and ultimately making a call about whether a failure on a non-essential (in terms of Wikipedia + MW_latest) branch build should trigger an unbreak-now, or if it can be tolerated after a cursory review.

Jdforrester-WMF lowered the priority of this task from Unbreak Now! to Medium.Thu, Aug 6, 1:40 PM
Jdforrester-WMF subscribed.

We dropped running browser tests from release branches in 018a98f77 so this is no longer UBN.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits