Page MenuHomePhabricator
Create Task
Maniphest T334626

Selenium test "add image.mobile: user can close the image suggestion UI" is flaky
Open, Needs TriagePublic
Actions

Description

Failure on https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74-docker/30554/console
The failure was for a patch set of the Translate extension, the test definition is in GrowthExperiments extension.

» /tests/selenium/specs/addimage.js
add image
   ✓ desktop: user can view image info and image details
   ✖ mobile: user can close the image suggestion UI

1 passing (1m 16.6s)
1 failing

1) add image mobile: user can close the image suggestion UI
Timeout of 60000ms exceeded. The execution in the test "add image mobile: user can close the image suggestion UI" took too long. Try to reduce the run time or increase your timeout for test specs (https://webdriver.io/docs/timeouts). (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addimage.js)
Error: Timeout of 60000ms exceeded. The execution in the test "add image mobile: user can close the image suggestion UI" took too long. Try to reduce the run time or increase your timeout for test specs (https://webdriver.io/docs/timeouts). (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addimage.js)
    at createTimeoutError (/workspace/src/extensions/GrowthExperiments/node_modules/mocha/lib/errors.js:498:15)
    at Test.Runnable._timeoutError (/workspace/src/extensions/GrowthExperiments/node_modules/mocha/lib/runnable.js:431:10)
    at Timeout.<anonymous> (/workspace/src/extensions/GrowthExperiments/node_modules/mocha/lib/runnable.js:246:24)
    at listOnTimeout (internal/timers.js:557:17)
    at processTimers (internal/timers.js:500:7)

Video of the timeout
https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74-docker/30554/artifact/log/add-image-mobile%253A-user-can-close-the-image-suggestion-UI-2023-04-12T19-58-40-751Z.mp4

Details

SubjectRepoBranchLines +/-
Remove skipped tests in addImage.js and delete homepage.jsmediawiki/extensions/GrowthExperimentsmaster+0 -115
Temporarily disable failing testsmediawiki/extensions/GrowthExperimentsmaster+3 -3
Customize query in gerrit

Related Objects
Search...

Event Timeline

Umherirrender created this task.Apr 12 2023, 8:48 PM
Restricted Application added a project: Growth-Team. · View Herald TranscriptApr 12 2023, 8:48 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Umherirrender mentioned this in T291254: Find flaky Selenium tests.Apr 12 2023, 8:48 PM
Umherirrender added a parent task: T291254: Find flaky Selenium tests.Apr 12 2023, 11:31 PM
Sgs subscribed.Apr 14 2023, 8:40 AM
kostajh moved this task from Inbox to Triaged on the Growth-Team board.Apr 19 2023, 9:51 AM
kostajh subscribed.

It looks like the content editable surface has changed so browser is blocked from exiting the page. I'm not sure why that only happens sometimes. It seems like a bug in webdriverio.

kostajh added a parent task: T333335: [Epic] Q4 FY 2022-23 Growth Maintenance Work.Apr 19 2023, 9:52 AM
Umherirrender added a comment.Apr 29 2023, 6:49 PM

A fresh one to look into the video etc. - https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php81-docker/6022/console

Dreamy_Jazz subscribed.May 3 2023, 6:13 PM

Happened again: https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74-docker/34134/console

kostajh added a comment.May 4 2023, 9:42 AM
In T334626#8824834, @Dreamy_Jazz wrote:

Happened again: https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74-docker/34134/console

This looks like a different issue from the one I noted in T334626#8792596. In https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74-docker/34134/console it looks like tapping on a link doesn't bring up a dialog as expected.

Nikerabbit subscribed.May 17 2023, 12:35 PM

https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php81-docker/6819/console

Umherirrender added a comment.May 19 2023, 3:15 PM

A fresh one: https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php81-docker/6948/console

gerritbot added a comment.May 20 2023, 6:51 PM

Change 921608 had a related patch set uploaded (by Kosta Harlan; author: Bartosz Dziewoński):

[mediawiki/extensions/GrowthExperiments@master] Temporarily disable failing tests

https://gerrit.wikimedia.org/r/921608

gerritbot added a project: Patch-For-Review.May 20 2023, 6:51 PM
gerritbot added a comment.May 20 2023, 7:54 PM

Change 921608 merged by Bartosz Dziewoński:

[mediawiki/extensions/GrowthExperiments@master] Temporarily disable failing tests

https://gerrit.wikimedia.org/r/921608

Maintenance_bot removed a project: Patch-For-Review.May 20 2023, 8:11 PM
DMburugu edited parent tasks, added: T340455: [Epic] Q1 FY 2023-24 Growth Maintenance Work; removed: T333335: [Epic] Q4 FY 2022-23 Growth Maintenance Work.Jun 26 2023, 5:15 PM
DMburugu edited parent tasks, added: T347840: [Epic] Q2 FY 2023-24 Growth Maintenance Work; removed: T340455: [Epic] Q1 FY 2023-24 Growth Maintenance Work.Oct 2 2023, 10:17 AM
Umherirrender unsubscribed.Oct 3 2023, 1:37 PM
Umherirrender added a comment.Jan 4 2024, 9:46 PM

The desktop version can fail as well:
https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74-docker/75476/console

22:34:44 [Chrome 90.0.4430.212 linux #0-0] Running: Chrome (v90.0.4430.212) on linux
22:34:44 [Chrome 90.0.4430.212 linux #0-0] Session ID: 474704fb-e1c1-4f67-8e0c-d14515444010
22:34:44 [Chrome 90.0.4430.212 linux #0-0]
22:34:44 [Chrome 90.0.4430.212 linux #0-0] » /tests/selenium/specs/addimage.js
22:34:44 [Chrome 90.0.4430.212 linux #0-0] add image
22:34:44 [Chrome 90.0.4430.212 linux #0-0]    ✖ desktop: user can view image info and image details
22:34:44 [Chrome 90.0.4430.212 linux #0-0]    - mobile: user can close the image suggestion UI
22:34:44 [Chrome 90.0.4430.212 linux #0-0]
22:34:44 [Chrome 90.0.4430.212 linux #0-0] 1 failing (12s)
22:34:44 [Chrome 90.0.4430.212 linux #0-0] 1 skipped
22:34:44 [Chrome 90.0.4430.212 linux #0-0]
22:34:44 [Chrome 90.0.4430.212 linux #0-0] 1) add image desktop: user can view image info and image details
22:34:44 [Chrome 90.0.4430.212 linux #0-0] element (".oo-ui-messageDialog-message") still not displayed after 5000ms
22:34:44 [Chrome 90.0.4430.212 linux #0-0] Error: element (".oo-ui-messageDialog-message") still not displayed after 5000ms
22:34:44 [Chrome 90.0.4430.212 linux #0-0]     at /workspace/src/extensions/GrowthExperiments/node_modules/webdriverio/build/commands/browser/waitUntil.js:66:23
Umherirrender added a comment.Jan 5 2024, 9:00 PM

https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php81-docker/16276/console

21:26:42 [Chrome 90.0.4430.212 linux #0-0] Running: Chrome (v90.0.4430.212) on linux
21:26:42 [Chrome 90.0.4430.212 linux #0-0] Session ID: abc381db-c582-4555-b538-239a0cc52dd7
21:26:42 [Chrome 90.0.4430.212 linux #0-0]
21:26:42 [Chrome 90.0.4430.212 linux #0-0] » /tests/selenium/specs/addimage.js
21:26:42 [Chrome 90.0.4430.212 linux #0-0] add image
21:26:42 [Chrome 90.0.4430.212 linux #0-0]    ✖ desktop: user can view image info and image details
21:26:42 [Chrome 90.0.4430.212 linux #0-0]    - mobile: user can close the image suggestion UI
21:26:42 [Chrome 90.0.4430.212 linux #0-0]
21:26:42 [Chrome 90.0.4430.212 linux #0-0] 1 failing (13.6s)
21:26:42 [Chrome 90.0.4430.212 linux #0-0] 1 skipped
21:26:42 [Chrome 90.0.4430.212 linux #0-0]
21:26:42 [Chrome 90.0.4430.212 linux #0-0] 1) add image desktop: user can view image info and image details
21:26:42 [Chrome 90.0.4430.212 linux #0-0] Evaluation failed: notloggedin
21:26:42 [Chrome 90.0.4430.212 linux #0-0] Evaluation failed: notloggedin
21:26:42 [Chrome 90.0.4430.212 linux #0-0] Error: Evaluation failed: notloggedin
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at ExecutionContext._evaluateInternal (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:221:19)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at processTicksAndRejections (node:internal/process/task_queues:96:5)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at async ExecutionContext.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:110:16)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at async ElementHandle.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:107:16)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at async ElementHandle.$eval (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:810:24)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at async DevToolsDriver.executeScript (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/commands/executeScript.js:39:20)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at async Browser.wrappedCommand (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/devtoolsdriver.js:102:26)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at async AddImageArticlePage.setup (/workspace/src/extensions/GrowthExperiments/tests/selenium/pageobjects/addimage.article.page.js:50:3)
21:26:42 [Chrome 90.0.4430.212 linux #0-0]     at async Context.<anonymous> (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addimage.js:19:3)
Lucas_Werkmeister_WMDE subscribed.Sep 16 2024, 9:22 AM

https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74/29156/console

[0-0] Error in "add image.desktop: user can view image info and image details"
Evaluation failed: notloggedin
Error: Evaluation failed: notloggedin
    at ExecutionContext._evaluateInternal (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:221:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ExecutionContext.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:110:16)
    at async ElementHandle.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:107:16)
    at async ElementHandle.$eval (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:810:24)
    at async DevToolsDriver.executeScript (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/commands/executeScript.js:39:20)
    at async Browser.wrappedCommand (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/devtoolsdriver.js:102:26)
    at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
    at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
    at async AddImageArticlePage.setup (/workspace/src/extensions/GrowthExperiments/tests/selenium/pageobjects/addimage.article.page.js:84:3)
    at async Context.<anonymous> (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addimage.js:19:3)
[0-0] RETRYING after 3s in chrome - /tests/selenium/specs/addimage.js
[0-0] RUNNING in chrome - /tests/selenium/specs/addimage.js
[0-0] Error in "add image.desktop: user can view image info and image details"
Evaluation failed: notloggedin
Error: Evaluation failed: notloggedin
    at ExecutionContext._evaluateInternal (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:221:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ExecutionContext.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:110:16)
    at async ElementHandle.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:107:16)
    at async ElementHandle.$eval (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:810:24)
    at async DevToolsDriver.executeScript (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/commands/executeScript.js:39:20)
    at async Browser.wrappedCommand (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/devtoolsdriver.js:102:26)
    at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
    at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
    at async AddImageArticlePage.setup (/workspace/src/extensions/GrowthExperiments/tests/selenium/pageobjects/addimage.article.page.js:84:3)
    at async Context.<anonymous> (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addimage.js:19:3)
[0-0] RETRYING after 3s in chrome - /tests/selenium/specs/addimage.js (1 retries)
[0-0] RUNNING in chrome - /tests/selenium/specs/addimage.js
[0-0] Error in "add image.desktop: user can view image info and image details"
Evaluation failed: notloggedin
Error: Evaluation failed: notloggedin
    at ExecutionContext._evaluateInternal (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:221:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async ExecutionContext.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:110:16)
    at async ElementHandle.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:107:16)
    at async ElementHandle.$eval (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:810:24)
    at async DevToolsDriver.executeScript (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/commands/executeScript.js:39:20)
    at async Browser.wrappedCommand (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/devtoolsdriver.js:102:26)
    at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
    at async Browser.wrapCommandFn (/workspace/src/extensions/GrowthExperiments/node_modules/@wdio/utils/build/shim.js:137:29)
    at async AddImageArticlePage.setup (/workspace/src/extensions/GrowthExperiments/tests/selenium/pageobjects/addimage.article.page.js:84:3)
    at async Context.<anonymous> (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addimage.js:19:3)
[0-0] FAILED in chrome - /tests/selenium/specs/addimage.js (2 retries)
gerritbot added a comment.Nov 11 2024, 3:24 AM

Change #1089338 had a related patch set uploaded (by Jared Blumer; author: Jared Blumer):

[mediawiki/extensions/GrowthExperiments@master] Remove skipped tests in addImage.js and delete homepage.js

https://gerrit.wikimedia.org/r/1089338

gerritbot added a project: Patch-For-Review.Nov 11 2024, 3:24 AM
gerritbot added a comment.Nov 14 2024, 11:47 AM

Change #1089338 abandoned by Zfilipin:

[mediawiki/extensions/GrowthExperiments@master] Remove skipped tests in addImage.js and delete homepage.js

Reason:

Abandoning since the test will be moved to cypress.

https://gerrit.wikimedia.org/r/1089338

Maintenance_bot removed a project: Patch-For-Review.Nov 14 2024, 12:31 PM
• bd808 subscribed.Thu, Dec 19, 4:15 PM

https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php74/46146/console

00:11:20.036 [Chrome 90.0.4430.212 linux #0-0] 1) add image desktop: user can view image info and image details
00:11:20.036 [Chrome 90.0.4430.212 linux #0-0] Evaluation failed: notloggedin
00:11:20.036 [Chrome 90.0.4430.212 linux #0-0] Evaluation failed: notloggedin
00:11:20.036 [Chrome 90.0.4430.212 linux #0-0] Error: Evaluation failed: notloggedin
00:11:20.036 [Chrome 90.0.4430.212 linux #0-0]     at ExecutionContext._evaluateInternal (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:221:19)
00:11:20.036 [Chrome 90.0.4430.212 linux #0-0]     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
00:11:20.037 [Chrome 90.0.4430.212 linux #0-0]     at async ExecutionContext.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/ExecutionContext.js:110:16)
00:11:20.037 [Chrome 90.0.4430.212 linux #0-0]     at async ElementHandle.evaluate (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:107:16)
00:11:20.037 [Chrome 90.0.4430.212 linux #0-0]     at async ElementHandle.$eval (/workspace/src/extensions/GrowthExperiments/node_modules/puppeteer-core/lib/cjs/puppeteer/common/JSHandle.js:810:24)
00:11:20.037 [Chrome 90.0.4430.212 linux #0-0]     at async DevToolsDriver.executeScript (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/commands/executeScript.js:39:20)
00:11:20.037 [Chrome 90.0.4430.212 linux #0-0]     at async Browser.wrappedCommand (/workspace/src/extensions/GrowthExperiments/node_modules/devtools/build/devtoolsdriver.js:102:26)
00:11:20.037 [Chrome 90.0.4430.212 linux #0-0]     at async AddImageArticlePage.setup (/workspace/src/extensions/GrowthExperiments/tests/selenium/pageobjects/addimage.article.page.js:84:3)
00:11:20.038 [Chrome 90.0.4430.212 linux #0-0]     at async Context.<anonymous> (/workspace/src/extensions/GrowthExperiments/tests/selenium/specs/addimage.js:19:3)
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits