Page MenuHomePhabricator

Flaky selenium test for CodeMirror + VisualEditor
Closed, DuplicatePublic

Description

The highlighting-wikitext2017.js test often fails with either or both of:

20:19:04 [0-2] Error in "CodeMirror (enabled) - VisualEditor 2017 wikitext editor.opens with the CodeMirror view displayed and focus set on the VE surface"
20:19:04 Error: element (".ve-ce-attachedRootNode") still not displayed after 5000ms
20:19:04     at async Context.<anonymous> (/workspace/src/extensions/CodeMirror/tests/selenium/specs/highlighting-wikitext2017.js:23:3)

or

20:19:35 [0-2] Error in "CodeMirror (enabled) - VisualEditor 2017 wikitext editor.should only load necessary modules when the CodeMirror preference is unset"
20:19:35 net::ERR_ABORTED at http://127.0.0.1:9413//index.php?uselang=en&title=Special%3ABlankPage
20:19:35 Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?uselang=en&title=Special%3ABlankPage

Note we've always had issues with the Selenium tests for the 2017 editor (r1012801).

In this case it seems to only happen for the "only load necessary modules…" test case. I suspect ERR_ABORTED is something to do with the page refreshing before the API request to update the user preference finishes. However, we're doing this in the same way as when we set preferences in the beforeEach() hook, and other test cases involve page refreshes as well, so it's a bit of a mystery as to what exactly is going wrong.

Event Timeline

Instead of refreshing the page, perhaps we should open a new edit page.

Also seems to happen more often with the gate-and-submit pipeline.

I will give opening in a new tab a try!

Change #1169299 had a related patch set uploaded (by MusikAnimal; author: MusikAnimal):

[mediawiki/extensions/CodeMirror@master] Selenium: don't change to a new page when setting preferences

https://gerrit.wikimedia.org/r/1169299

Change #1169299 merged by jenkins-bot:

[mediawiki/extensions/CodeMirror@master] Selenium: don't change to a new page when setting preferences

https://gerrit.wikimedia.org/r/1169299

https://gerrit.wikimedia.org/r/c/mediawiki/extensions/CodeMirror/+/1175391
https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php81-selenium/21181/console

05:48:05 [0-2] Error in "CodeMirror (enabled) - VisualEditor 2017 wikitext editor.should only load necessary modules when the CodeMirror preference is unset"
05:48:05 net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=edit&vehidebetadialog=1&hidewelcomedialog=1&cm6enable=1&title=CodeMirror-fixture1-0.3088876129429925-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
05:48:05 Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=edit&vehidebetadialog=1&hidewelcomedialog=1&cm6enable=1&title=CodeMirror-fixture1-0.3088876129429925-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
05:48:05     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
05:48:05     at async DevToolsDriver.navigateTo (/workspace/src/extensions/CodeMirror/node_modules/devtools/build/commands/navigateTo.js:18:5)
05:48:05     at async Browser.wrappedCommand (/workspace/src/extensions/CodeMirror/node_modules/devtools/build/devtoolsdriver.js:128:26)
05:48:05     at async EditPage.openTitle (/workspace/src/extensions/CodeMirror/node_modules/wdio-mediawiki/Page.js:22:3)
05:48:05     at async EditPage.openForEditing (/workspace/src/extensions/CodeMirror/tests/selenium/pageobjects/edit.page.js:14:3)
05:48:05     at async Context.<anonymous> (/workspace/src/extensions/CodeMirror/tests/selenium/specs/highlighting-wikitext2017.js:101:3)
05:48:05 [0-2] RETRYING in chrome - /tests/selenium/specs/highlighting-wikitext2017.js
05:48:06 [0-2] RUNNING in chrome - /tests/selenium/specs/highlighting-wikitext2017.js
05:48:26 [0-2] Error in "CodeMirror (enabled) - VisualEditor 2017 wikitext editor.should only load necessary modules when the CodeMirror preference is unset"
05:48:26 net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=edit&vehidebetadialog=1&hidewelcomedialog=1&cm6enable=1&title=CodeMirror-fixture1-0.2790912764840632-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
05:48:26 Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=edit&vehidebetadialog=1&hidewelcomedialog=1&cm6enable=1&title=CodeMirror-fixture1-0.2790912764840632-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
05:48:26     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
05:48:26     at async DevToolsDriver.navigateTo (/workspace/src/extensions/CodeMirror/node_modules/devtools/build/commands/navigateTo.js:18:5)
05:48:26     at async Browser.wrappedCommand (/workspace/src/extensions/CodeMirror/node_modules/devtools/build/devtoolsdriver.js:128:26)
05:48:26     at async EditPage.openTitle (/workspace/src/extensions/CodeMirror/node_modules/wdio-mediawiki/Page.js:22:3)
05:48:26     at async EditPage.openForEditing (/workspace/src/extensions/CodeMirror/tests/selenium/pageobjects/edit.page.js:14:3)
05:48:26     at async Context.<anonymous> (/workspace/src/extensions/CodeMirror/tests/selenium/specs/highlighting-wikitext2017.js:101:3)
05:48:27 [0-2] FAILED in chrome - /tests/selenium/specs/highlighting-wikitext2017.js (1 retries)
...
05:48:27 [Chrome 120.0.0.0 linux #0-2] 1) CodeMirror (enabled) - VisualEditor 2017 wikitext editor should only load necessary modules when the CodeMirror preference is unset
05:48:27 [Chrome 120.0.0.0 linux #0-2] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=edit&vehidebetadialog=1&hidewelcomedialog=1&cm6enable=1&title=CodeMirror-fixture1-0.3088876129429925-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
05:48:27 [Chrome 120.0.0.0 linux #0-2] net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=edit&vehidebetadialog=1&hidewelcomedialog=1&cm6enable=1&title=CodeMirror-fixture1-0.3088876129429925-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
05:48:27 [Chrome 120.0.0.0 linux #0-2] Error: net::ERR_ABORTED at http://127.0.0.1:9413//index.php?action=edit&vehidebetadialog=1&hidewelcomedialog=1&cm6enable=1&title=CodeMirror-fixture1-0.3088876129429925-I%C3%B1t%C3%ABrn%C3%A2ti%C3%B4n%C3%A0liz%C3%A6ti%C3%B8n
05:48:27 [Chrome 120.0.0.0 linux #0-2]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
05:48:27 [Chrome 120.0.0.0 linux #0-2]     at async DevToolsDriver.navigateTo (/workspace/src/extensions/CodeMirror/node_modules/devtools/build/commands/navigateTo.js:18:5)
05:48:27 [Chrome 120.0.0.0 linux #0-2]     at async Browser.wrappedCommand (/workspace/src/extensions/CodeMirror/node_modules/devtools/build/devtoolsdriver.js:128:26)
05:48:27 [Chrome 120.0.0.0 linux #0-2]     at async EditPage.openTitle (/workspace/src/extensions/CodeMirror/node_modules/wdio-mediawiki/Page.js:22:3)
05:48:27 [Chrome 120.0.0.0 linux #0-2]     at async EditPage.openForEditing (/workspace/src/extensions/CodeMirror/tests/selenium/pageobjects/edit.page.js:14:3)
05:48:27 [Chrome 120.0.0.0 linux #0-2]     at async Context.<anonymous> (/workspace/src/extensions/CodeMirror/tests/selenium/specs/highlighting-wikitext2017.js:101:3)