Page MenuHomePhabricator
Create Task
Maniphest T411134

"On Initial view -> Will not switch to edit mode if the column clicked is not selected" test is flaky
Closed, ResolvedPublic
Actions

Description

The named test has flaked 21 times, thus appearing in the flaky test dashboard.

https://releng-data.wmcloud.org/-/dashboards/selenium/last-thirty

Test reference:

https://integration.wikimedia.org/ci/job/selenium-daily-beta-TwoColConflict/2274/

Purpose of this ticket:

Investigate the flakiness - Is the test still flaky? Does it happen locally? What is the cause of the flakiness?

Based on those answers, plan and implement a resolution for the problematic test.

Failure from most recent flake :

Error: Can't call click on element with selector ".mw-twocolconflict-split-selection-row div:nth-child(1) span" because element wasn't found
    at async Context.<anonymous> (/src/tests/selenium/specs/editmerge.js:93:4)

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
Selenium: Removing some selenium testsmediawiki/extensions/TwoColConflictmaster+22 -781
selenium: Fixing test flake for "Will not switch to edit mode if the column clicked is not selected";mediawiki/extensions/TwoColConflictmaster+19 -2
Customize query in gerrit

Related Objects

Event Timeline

SLong-WMF created this task.Nov 26 2025, 9:18 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 26 2025, 9:18 PM
SLong-WMF moved this task from Backlog to Refined on the QS-Test-Automation board.Jan 28 2026, 5:31 PM
vaughnwalters claimed this task.Jan 28 2026, 9:18 PM
vaughnwalters moved this task from Refined to In Progress on the QS-Test-Automation board.
vaughnwalters added a comment.EditedJan 28 2026, 9:26 PM

most recent reference test: https://integration.wikimedia.org/ci/job/selenium-daily-beta-TwoColConflict/2337/

Error details

Can't call click on element with selector ".mw-twocolconflict-split-selection-row div:nth-child(1) span" because element wasn't found

Standard output

COMMAND: navigateTo - {"url":"https://en.wikipedia.beta.wmcloud.org/w/index.php?action=edit&vehidebetadialog=1&hidewelcomedialog=1&title=conflict-title-0.9242675857996017-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n"}
RESULT: undefined
COMMAND: executeScript - {"script":"return (() => document.readyState === 'complete').apply(null, arguments)","args":[]}
RESULT: undefined
COMMAND: findElement - {"using":"css selector","value":"#wpTextbox1"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":"#wpTextbox1"}
RESULT: undefined
COMMAND: findElement - {"using":"css selector","value":"#wpTextbox1"}
RESULT: undefined
COMMAND: elementClear - {"elementId":"ELEMENT-94"}
RESULT: undefined
COMMAND: elementSendKeys - {"elementId":"ELEMENT-94","text":"Line<span>1</span>\n\nChange <span lang=\"en\">B</span>"}
RESULT: undefined
COMMAND: findElement - {"using":"css selector","value":"#wpSave"}
RESULT: undefined
COMMAND: elementClick - {"elementId":"ELEMENT-95"}
RESULT: undefined
COMMAND: executeScript - {"script":"return (( arg ) => typeof mw !== 'undefined' && mw.loader.getState( arg.name ) === arg.status).apply(null, arguments)","args":[{"status":"ready","name":"ext.TwoColConflict.SplitJs"}]}
RESULT: undefined
COMMAND: executeScript - {"script":"return (( arg ) => typeof mw !== 'undefined' && mw.loader.getState( arg.name ) === arg.status).apply(null, arguments)","args":[{"status":"ready","name":"ext.TwoColConflict.SplitJs"}]}
RESULT: undefined
COMMAND: findElement - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: findElements - {"using":"css selector","value":".mw-twocolconflict-split-selection-row div:nth-child(1) span"}
RESULT: undefined
COMMAND: takeScreenshot - {}
RESULT: undefined

Standard error

Error: Can't call click on element with selector ".mw-twocolconflict-split-selection-row div:nth-child(1) span" because element wasn't found
    at async Context.<anonymous> (/src/tests/selenium/specs/editmerge.js:93:4)
vaughnwalters renamed this task from "On Initial view -> Will not switch to edit mode" test is flaky to "On Initial view -> Will not switch to edit mode if the column clicked is not selected" test is flaky.Jan 29 2026, 11:05 PM
gerritbot added a comment.Jan 29 2026, 11:17 PM

Change #1235165 had a related patch set uploaded (by Vaughn Walters; author: Vaughn Walters):

[mediawiki/extensions/TwoColConflict@master] selenium: Fixing test flake

https://gerrit.wikimedia.org/r/1235165

gerritbot added a project: Patch-For-Review.Jan 29 2026, 11:17 PM
vaughnwalters added a comment.EditedFeb 2 2026, 9:54 PM

In isolation:
Test passes 100/100 on local
Test passes 100/100 in CI

Entire file containing the test (editmerge.js):
Test passes 75/75 on CI before timing out

Entire test suite:
Test passes 64/64 in CI before timing out

From local when running against betacluster 100 times there were two failures, but not the one that was present on the flaky test report.

Execution of 1 workers started at 2026-01-30T23:26:51.502Z

[0-0] RUNNING in chrome - /tests/selenium/specs/editmerge.js
[0-0] Error in "TwoColConflict EditUi.on initial view."before all" hook for "will not switch to edit mode if the column clicked is not selected""
Error: element ("#wpTextbox1") still not existing after 5000ms
    at async EditConflictPage.createConflict (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/tests/selenium/pageobjects/editconflict.page.js:256:3)
    at async EditConflictPage.showSimpleConflict (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/tests/selenium/pageobjects/editconflict.page.js:213:3)
    at async Context.<anonymous> (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/tests/selenium/specs/editmerge.js:17:4)
[0-0] RETRYING in chrome - /tests/selenium/specs/editmerge.js
[0-0] RUNNING in chrome - /tests/selenium/specs/editmerge.js
[0-0] PASSED in chrome - /tests/selenium/specs/editmerge.js (1 retries)

 "spec" Reporter:
------------------------------------------------------------------
[Chrome Headless 144.0.0.0 darwin #0-0] Running: Chrome Headless (v144.0.0.0) on darwin
[Chrome Headless 144.0.0.0 darwin #0-0] Session ID: e90def01-2ea5-496f-b549-afc2ff006a4e
[Chrome Headless 144.0.0.0 darwin #0-0]
[Chrome Headless 144.0.0.0 darwin #0-0] » /tests/selenium/specs/editmerge.js
[Chrome Headless 144.0.0.0 darwin #0-0] TwoColConflict EditUi
[Chrome Headless 144.0.0.0 darwin #0-0]     on initial view
[Chrome Headless 144.0.0.0 darwin #0-0]        ✖ "before all" hook for on initial view
[Chrome Headless 144.0.0.0 darwin #0-0]
[Chrome Headless 144.0.0.0 darwin #0-0] 1 failing (12.3s)
[Chrome Headless 144.0.0.0 darwin #0-0]
[Chrome Headless 144.0.0.0 darwin #0-0] 1) on initial view "before all" hook for on initial view
[Chrome Headless 144.0.0.0 darwin #0-0] element ("#wpTextbox1") still not existing after 5000ms
[Chrome Headless 144.0.0.0 darwin #0-0] Error: element ("#wpTextbox1") still not existing after 5000ms
[Chrome Headless 144.0.0.0 darwin #0-0]     at async EditConflictPage.createConflict (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/tests/selenium/pageobjects/editconflict.page.js:256:3)
[Chrome Headless 144.0.0.0 darwin #0-0]     at async EditConflictPage.showSimpleConflict (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/tests/selenium/pageobjects/editconflict.page.js:213:3)
[Chrome Headless 144.0.0.0 darwin #0-0]     at async Context.<anonymous> (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/tests/selenium/specs/editmerge.js:17:4)

and also the mwbot error, which needs wdio-mediawiki upgraded to 6.0.0 so mwbot can be removed:

0-0] RUNNING in chrome - /tests/selenium/specs/editmerge.js
[0-0] [17:11:53] [E] [MWBOT] Login failed: User-0.5235439919609335-Iñtërnâtiônàlizætiøn@https://en.wikipedia.beta.wmcloud.org/w
[0-0] Error in "TwoColConflict EditUi.on initial view."before all" hook for "will not switch to edit mode if the column clicked is not selected""
Error: Could not login: Failed
    at /Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/mwbot/src/index.js:335:31
    at tryCatcher (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/promise.js:729:18)
    at _drainQueueStep (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/async.js:102:5)
    at Async.drainQueues (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/async.js:15:14)
    at process.processImmediate (node:internal/timers:485:21)
[0-0] RETRYING in chrome - /tests/selenium/specs/editmerge.js
[0-0] RUNNING in chrome - /tests/selenium/specs/editmerge.js
[0-0] PASSED in chrome - /tests/selenium/specs/editmerge.js (1 retries)

 "spec" Reporter:
------------------------------------------------------------------
[Chrome Headless 144.0.0.0 darwin #0-0] Running: Chrome Headless (v144.0.0.0) on darwin
[Chrome Headless 144.0.0.0 darwin #0-0] Session ID: 9c2cc3a2-7aec-4b99-bd21-d868f452b3ff
[Chrome Headless 144.0.0.0 darwin #0-0]
[Chrome Headless 144.0.0.0 darwin #0-0] » /tests/selenium/specs/editmerge.js
[Chrome Headless 144.0.0.0 darwin #0-0] TwoColConflict EditUi
[Chrome Headless 144.0.0.0 darwin #0-0]     on initial view
[Chrome Headless 144.0.0.0 darwin #0-0]        ✖ "before all" hook for on initial view
[Chrome Headless 144.0.0.0 darwin #0-0]
[Chrome Headless 144.0.0.0 darwin #0-0] 1 failing (10.7s)
[Chrome Headless 144.0.0.0 darwin #0-0]
[Chrome Headless 144.0.0.0 darwin #0-0] 1) on initial view "before all" hook for on initial view
[Chrome Headless 144.0.0.0 darwin #0-0] Could not login: Failed
[Chrome Headless 144.0.0.0 darwin #0-0] Error: Could not login: Failed
[Chrome Headless 144.0.0.0 darwin #0-0]     at /Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/mwbot/src/index.js:335:31
[Chrome Headless 144.0.0.0 darwin #0-0]     at tryCatcher (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/util.js:16:23)
[Chrome Headless 144.0.0.0 darwin #0-0]     at Promise._settlePromiseFromHandler (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/promise.js:547:31)
[Chrome Headless 144.0.0.0 darwin #0-0]     at Promise._settlePromise (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/promise.js:604:18)
[Chrome Headless 144.0.0.0 darwin #0-0]     at Promise._settlePromise0 (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/promise.js:649:10)
[Chrome Headless 144.0.0.0 darwin #0-0]     at Promise._settlePromises (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/promise.js:729:18)
[Chrome Headless 144.0.0.0 darwin #0-0]     at _drainQueueStep (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/async.js:93:12)
[Chrome Headless 144.0.0.0 darwin #0-0]     at _drainQueue (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/async.js:86:9)
[Chrome Headless 144.0.0.0 darwin #0-0]     at Async._drainQueues (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/async.js:102:5)
[Chrome Headless 144.0.0.0 darwin #0-0]     at Async.drainQueues (/Users/vaughnwalters/workspace/gerrit/mediawiki/extensions/TwoColConflict/node_modules/bluebird/js/release/async.js:15:14)
[Chrome Headless 144.0.0.0 darwin #0-0]     at process.processImmediate (node:internal/timers:485:21)
vaughnwalters updated the task description. (Show Details)Feb 3 2026, 9:06 PM
vaughnwalters mentioned this in T416633: [EPIC] Update selenium tests in TwoColConflict.Feb 5 2026, 7:34 PM
vaughnwalters moved this task from In Progress to In Code Review on the QS-Test-Automation board.Feb 5 2026, 8:45 PM
vaughnwalters added a comment.EditedFeb 5 2026, 8:54 PM

The issues in this test suite are larger than just this one flaky test. I created T416633 as a follow up from the findings in this task.

This was the last stable build, which was June 15, 2025:
https://integration.wikimedia.org/ci/job/selenium-daily-beta-TwoColConflict/lastStableBuild/

And this was the first time after that the the test suite flaked and it has not passed since then.
https://integration.wikimedia.org/ci/job/selenium-daily-beta-TwoColConflict/2351/

I have added waits and logging to the test in this task Will not switch to edit mode if the column clicked is not selected and it is passing now in my local runs against betacluster, but we need to look at what is causing the more widespread and consistent failure for 7+ months. After this fix is merged, we can look at T416633.

SLong-WMF added a comment.Feb 23 2026, 7:07 PM

Who are we waiting for a code review from? The owner of the test suite?

vaughnwalters added a comment.EditedFeb 23 2026, 7:10 PM
In T411134#11642010, @SLong-WMF wrote:

Who are we waiting for a code review from? The owner of the test suite?

Owner of the test suite or anyone who contributes to TwoColConflict extension. One user was auto added and I added a few others who have contributed to the repo. I'm not sure who owns this extension though.

Aklapper added a comment.Feb 24 2026, 7:26 AM

I'm not sure who owns this extension though.

Per https://www.mediawiki.org/wiki/Developers/Maintainers it is maintained by WMDE Engineering.

vaughnwalters added a project: Two-Column-Edit-Conflict-Merge.Feb 26 2026, 4:01 AM
ARamirez_WMF added subscribers: Lina_Farid_WMDE, ARamirez_WMF.Mar 5 2026, 7:44 PM

Hi @Lina_Farid_WMDE, Lydia pointed us in your team's direction. Do you think the team could help us out here?

Lina_Farid_WMDE added a subscriber: Tobi_WMDE_SW.Mar 6 2026, 7:22 AM
awight subscribed.Mar 6 2026, 9:04 AM

Hi, Tech Wishes dev here! After a brief chat, we would recommend disabling the entire suite for now. We've noticed the flakiness as well and it counteracts any value that might be gotten from having the tests. I'm happy to do that if you agree that it's the right direction to go in.

thiemowmde subscribed.Mar 9 2026, 11:39 AM
vaughnwalters added a comment.Mar 9 2026, 7:01 PM
In T411134#11680641, @awight wrote:

Hi, Tech Wishes dev here! After a brief chat, we would recommend disabling the entire suite for now. We've noticed the flakiness as well and it counteracts any value that might be gotten from having the tests. I'm happy to do that if you agree that it's the right direction to go in.

Hello hello @awight, yeah I think if it's not adding value then that is fine with me. Let me know when they are disabled and we can close this ticket, and I will go ahead and mark https://gerrit.wikimedia.org/r/c/mediawiki/extensions/TwoColConflict/+/1235165 as abandoned as well then. Thank you!

gerritbot added a comment.Mar 9 2026, 7:02 PM

Change #1235165 abandoned by Vaughn Walters:

[mediawiki/extensions/TwoColConflict@master] selenium: Fixing test flake for "Will not switch to edit mode if the column clicked is not selected";

Reason:

Test suite is being disabled by Tech Wishes team

https://gerrit.wikimedia.org/r/1235165

vaughnwalters moved this task from In Code Review to In Validation on the QS-Test-Automation board.Mar 9 2026, 7:15 PM
Maintenance_bot removed a project: Patch-For-Review.Mar 9 2026, 7:31 PM
gerritbot added a comment.Mar 13 2026, 10:46 AM

Change #1251277 had a related patch set uploaded (by WMDE-Fisch; author: WMDE-Fisch):

[mediawiki/extensions/TwoColConflict@master] Selenium: Removing some selenium tests

https://gerrit.wikimedia.org/r/1251277

gerritbot added a project: Patch-For-Review.Mar 13 2026, 10:46 AM
vaughnwalters moved this task from In Validation to In Code Review on the QS-Test-Automation board.Mar 17 2026, 9:42 PM
gerritbot added a comment.Tue, Mar 24, 10:49 AM

Change #1251277 merged by jenkins-bot:

[mediawiki/extensions/TwoColConflict@master] Selenium: Removing some selenium tests

https://gerrit.wikimedia.org/r/1251277

Maintenance_bot removed a project: Patch-For-Review.Tue, Mar 24, 11:31 AM
vaughnwalters moved this task from In Code Review to Done on the QS-Test-Automation board.Tue, Mar 24, 2:38 PM
WMDE-Fisch closed this task as Resolved.Wed, Mar 25, 11:52 AM
WMDE-Fisch subscribed.

I hope it's fine now. :-)

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits