Page MenuHomePhabricator

Flaky selenium test "Failed to wait for ext.growthExperiments.PostEdit to be ready after 5000 ms."
Closed, ResolvedPublic

Description

Seen several times in a row at
https://gerrit.wikimedia.org/r/c/mediawiki/extensions/GrowthExperiments/+/1064751

[0-0] RUNNING in chrome - /tests/selenium/specs/addimage.js
[0-0] PASSED in chrome - /tests/selenium/specs/addimage.js
[0-1] RUNNING in chrome - /tests/selenium/specs/addlink.js
[0-1] Error in "add link.link inspector can be used to accept/reject links and save an article."
Error: Failed to wait for ext.growthExperiments.PostEdit to be ready after 5000 ms.
    at /workspace/src/extensions/GrowthExperiments/node_modules/webdriverio/build/commands/browser/waitUntil.js:66:23
    at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
    at async Object.waitForModuleState (/workspace/src/extensions/GrowthExperiments/node_modules/wdio-mediawiki/Util.js:16:3)
    at async Context.<anonymous> (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addlink.js:44:3)
[0-1] RETRYING after 3s in chrome - /tests/selenium/specs/addlink.js
[0-1] RUNNING in chrome - /tests/selenium/specs/addlink.js
[0-1] Error in "add link.link inspector can be used to accept/reject links and save an article."
Error: Can't call getText on element with selector ".se-card-title" because element wasn't found
    at implicitWait (/workspace/src/extensions/GrowthExperiments/node_modules/webdriverio/build/utils/implicitWait.js:34:19)
    at async Element.elementErrorHandlerCallbackFn (/workspace/src/extensions/GrowthExperiments/node_modules/webdriverio/build/middlewares.js:20:29)
    at async Element.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
    at async Context.<anonymous> (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addlink.js:19:23)
[0-1] RETRYING after 3s in chrome - /tests/selenium/specs/addlink.js (1 retries)
[0-1] RUNNING in chrome - /tests/selenium/specs/addlink.js
[0-1] Error in "add link.link inspector can be used to accept/reject links and save an article."
Error: Failed to wait for ext.growthExperiments.PostEdit to be ready after 5000 ms.
    at /workspace/src/extensions/GrowthExperiments/node_modules/webdriverio/build/commands/browser/waitUntil.js:66:23
    at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
    at async Object.waitForModuleState (/workspace/src/extensions/GrowthExperiments/node_modules/wdio-mediawiki/Util.js:16:3)
    at async Context.<anonymous> (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addlink.js:44:3)
[0-1] FAILED in chrome - /tests/selenium/specs/addlink.js (2 retries)
[0-2] RUNNING in chrome - /tests/selenium/specs/homepage.js
[0-2] PASSED in chrome - /tests/selenium/specs/homepage.js
[0-3] RUNNING in chrome - /tests/selenium/specs/mentordashboard.js
[0-3] PASSED in chrome - /tests/selenium/specs/mentordashboard.js

This and other GrowthExperiments flaky tests have been holding up MediaWiki development across extensions for 4+ hours today with most merges stuck behind it, and requiring everything to be re-tested after it inevitably fails.

Screenshot 2024-08-23 at 00.20.14.png (1×560 px, 149 KB)

Event Timeline

Mentioned in SAL (#wikimedia-releng) [2024-08-22T23:26:03Z] <Krinkle> Manually abort quibble-phpunit jobs for GrowthExperiments patches in gate-and-submit pipeline given their selenium build has already failed, so that other patches +2'ed can start their retry and get merged. Ref T373157

Michael subscribed.

This is annoying because it keeps happening with quite high frequency. Let's see if we can tackle it in the next sprint.

Michael triaged this task as High priority.Aug 29 2024, 3:59 PM

Change #1069121 had a related patch set uploaded (by Michael Große; author: Michael Große):

[mediawiki/extensions/GrowthExperiments@master] tests: Make addLink selenium test less flaky

https://gerrit.wikimedia.org/r/1069121

I worked on it already because it annoyed me so much.

Change #1069121 merged by jenkins-bot:

[mediawiki/extensions/GrowthExperiments@master] tests: Make addLink selenium test less flaky

https://gerrit.wikimedia.org/r/1069121

This should be done now. Feel free to reopen if it still happens.