Page MenuHomePhabricator

Minerva Selenium tests blocking merges in DiscussionTools
Closed, ResolvedPublic

Description

We're unable to merge https://gerrit.wikimedia.org/r/c/mediawiki/extensions/DiscussionTools/+/767831 because of the following test failures: https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php72-selenium-docker/105577/console

18:02:25 [0-7] 	Video location: /workspace/log/Talk-Talk-on-a-page-that-doesn't-exist-(bug-64268)-2022-03-03T17-02-15-479Z.mp4 
18:02:25 [0-7] 
18:02:25 [0-7] 	ffmpeg exited with code 255 /workspace/log/Talk-Talk-on-a-page-that-doesn't-exist-(bug-64268)-2022-03-03T17-02-15-479Z.mp4
18:02:25 [0-7] AssertionError in "Talk.Add discussion button shows on talk pages for logged in users"
18:02:25 AssertionError [ERR_ASSERTION]: Expected values to be strictly equal:
18:02:25 
18:02:25 false !== true
18:02:25 
18:02:25     at thereShouldBeAnAddDiscussionButton (/workspace/src/skins/MinervaNeue/tests/selenium/features/step_definitions/talk_steps.js:39:9)
18:02:25     at Context.<anonymous> (/workspace/src/skins/MinervaNeue/tests/selenium/specs/talk.js:53:3)
18:02:25 [0-7] 
18:02:25 [0-7] 	Video location: /workspace/log/Talk-Add-discussion-button-shows-on-talk-pages-for-logged-in-users-2022-03-03T17-02-17-508Z.mp4 
18:02:25 [0-7] 
18:02:25 [0-7] 	ffmpeg exited with code 255 /workspace/log/Talk-Add-discussion-button-shows-on-talk-pages-for-logged-in-users-2022-03-03T17-02-17-508Z.mp4
18:02:25 [0-1] PASSED in chrome - /tests/selenium/specs/editor_wikitext_nosave.js
18:02:29 [0-7] Error in "Talk.Add discussion for talk page possible as logged in user"
18:02:29 Error: element (".minerva-talk-add-button") still not displayed after 5000ms
18:02:29     at Element.<anonymous> (/workspace/src/skins/MinervaNeue/node_modules/@wdio/utils/build/shim.js:270:24)
18:02:29     at runMicrotasks (<anonymous>)
18:02:29     at processTicksAndRejections (internal/process/task_queues.js:95:5)
18:02:29     at iClickTheAddTalkButton (/workspace/src/skins/MinervaNeue/tests/selenium/features/step_definitions/talk_steps.js:10:31)
18:02:29     at Context.<anonymous> (/workspace/src/skins/MinervaNeue/tests/selenium/specs/talk.js:60:3)
18:02:29 [0-7] FAILED in chrome - /tests/selenium/specs/talk.js

Event Timeline

This basically exhausts my knowledge about debugging Selenium tests, and this is blocking our work, so I'll submit a patch to disable these tests until someone who knows what they are doing can investigate.

Change 767863 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/skins/MinervaNeue@master] Disable failing talk page tests temporarily

https://gerrit.wikimedia.org/r/767863

Change 767863 merged by jenkins-bot:

[mediawiki/skins/MinervaNeue@master] Disable failing talk page tests temporarily

https://gerrit.wikimedia.org/r/767863

In case you've missed my comment on Gerrit I think it would make sense to remove these browser tests rather than fighting them given y'all are making lots of fundamental changes to how the talk page experience works.

@EAkinloose or @zeljkofilipin, can either of you take a look?

I am struggling to reproduce this locally.

I get a different error even after enabling the disabled tests in talk.js. A different test fails. See:

Screenshot 2022-03-04 at 02.53.32.png (286×2 px, 255 KB)

What is expected:

Screenshot 2022-03-04 at 02.57.01.png (914×636 px, 52 KB)

What comes up:

Screenshot 2022-03-04 at 02.57.59.png (1×632 px, 67 KB)

Note: I am testing against Beta cluster

Change 768798 had a related patch set uploaded (by Bartosz Dziewoński; author: Bartosz Dziewoński):

[mediawiki/skins/MinervaNeue@wmf/1.38.0-wmf.24] Disable failing talk page tests temporarily

https://gerrit.wikimedia.org/r/768798

Change 768798 merged by jenkins-bot:

[mediawiki/skins/MinervaNeue@wmf/1.38.0-wmf.24] Disable failing talk page tests temporarily

https://gerrit.wikimedia.org/r/768798

zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.

@EAkinloose or @zeljkofilipin, can either of you take a look?

Apologies for the late reply, I was on leave. Is this still a problem? Is it urgent?

Not urgent, we mitigated it by disabling the tests. But it's still a problem, assuming that the tests were useful, we'd like to restore them.

(Also, as noted by @Jdlrobson above, we might want to drop some of the tests anyway if they're not applicable to the new talk page experience?)

matmarex claimed this task.

I don't think we're going to do anything else about these tests. We have plans now to remove the feature entirely in medium term (T319145).