Page MenuHomePhabricator

Can't call getText on element with selector ".section-heading" because element wasn't found
Open, Needs TriagePublic

Description

Completely unrelated change
https://gerrit.wikimedia.org/r/c/mediawiki/vendor/+/621952/
https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/58637/console

22:36:10 [Chrome 73.0.3683.75 linux #0-7] Spec: /workspace/src/skins/MinervaNeue/tests/selenium/specs/talk.js
22:36:10 [Chrome 73.0.3683.75 linux #0-7] Running: Chrome (v73.0.3683.75) on linux
22:36:10 [Chrome 73.0.3683.75 linux #0-7] Session ID: 11f27abe-7a31-447f-8bc3-922c99100e6f
22:36:10 [Chrome 73.0.3683.75 linux #0-7]
22:36:10 [Chrome 73.0.3683.75 linux #0-7] Talk
22:36:10 [Chrome 73.0.3683.75 linux #0-7]    ✓ Talk button not visible as logged out user
22:36:10 [Chrome 73.0.3683.75 linux #0-7]    ✓ Talk tab visible as logged in user
22:36:10 [Chrome 73.0.3683.75 linux #0-7]    ✓ Talk on a page that doesn't exist (bug 64268)
22:36:10 [Chrome 73.0.3683.75 linux #0-7]    ✓ Add discussion button shows on talk pages for logged in users
22:36:10 [Chrome 73.0.3683.75 linux #0-7]    ✓ Add discussion for talk page possible as logged in user
22:36:10 [Chrome 73.0.3683.75 linux #0-7]    ✖ A newly created topic appears in the list of topics
22:36:10 [Chrome 73.0.3683.75 linux #0-7]
22:36:10 [Chrome 73.0.3683.75 linux #0-7] 5 passing (36.4s)
22:36:10 [Chrome 73.0.3683.75 linux #0-7] 1 failing
22:36:10 [Chrome 73.0.3683.75 linux #0-7]
22:36:10 [Chrome 73.0.3683.75 linux #0-7] 1) Talk A newly created topic appears in the list of topics
22:36:10 [Chrome 73.0.3683.75 linux #0-7] Can't call getText on element with selector ".section-heading" because element wasn't found
22:36:10 [Chrome 73.0.3683.75 linux #0-7] Error: Can't call getText on element with selector ".section-heading" because element wasn't found
22:36:10 [Chrome 73.0.3683.75 linux #0-7]     at iShouldSeeTheTopicInTheListOfTopics (/workspace/src/skins/MinervaNeue/tests/selenium/features/step_definitions/talk_steps.js:52:37)
22:36:10 [Chrome 73.0.3683.75 linux #0-7]     at Context.it (/workspace/src/skins/MinervaNeue/tests/selenium/specs/talk.js:75:3)

Event Timeline

Reedy created this task.Thu, Sep 10, 9:38 PM
Restricted Application added subscribers: Masumrezarock100, Aklapper. · View Herald TranscriptThu, Sep 10, 9:38 PM
Jdlrobson added a subscriber: Jdlrobson.

I've seen this a few times before. It's a race condition - as the video shows the form doesn't even get filled out:
https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/58637/artifact/log/Talk-Add-discussion-for-talk-page-possible-as-logged-in-user.png

I'm not sure what the underlying cause is and how to deal with this edge case.

I will need some help from @zeljkofilipin - looking at that test run the test is working it's just finishing too early before the page can reload.

zeljkofilipin added a comment.EditedWed, Sep 16, 1:08 PM

Just to make sure I've understood the problem: MinervaNeue Selenium test fails sometimes while running in CI for other repositories? (This task mentions mediawiki/vendor and UniversalLanguageSelector.)

If that is the case, as the first step I would recommend disabling the test, because it's obviously flaky (replace it with it.skip). That will make sure the test doesn't cause trouble while it's being fixed.

As the second step, I would recommend running the test 10-100 times in a row to see if the failure can be reproduced locally. If it can be reproduced, that's a good step towards fixing it. It should be as easy as:

for i in {1..10}; do npm run selenium-test -- --spec tests/selenium/specs/talk.js --mochaOpts.grep newly; done

If it can't be reproduced, as the third step I would recommend running the entire test suite 10-100 times in a row. Maybe the test fails only when running after other tests. (That should not happen, but sometimes does.) It should be as easy as:

for i in {1..10}; do npm run selenium-test; done

Let me know if you need help with any of the suggested steps.

I ran the entire test suite 5 times on my machine, targeting mediawiki-docker:

for i in {1..5}; do npm run selenium-test; done
...
Spec Files:	 10 passed, 10 total (100% completed) in 00:04:33 
...
Spec Files:	 10 passed, 10 total (100% completed) in 00:04:33 
...
Spec Files:	 10 passed, 10 total (100% completed) in 00:04:43 
...
Spec Files:	 10 passed, 10 total (100% completed) in 00:04:28 
...
Spec Files:	 10 passed, 10 total (100% completed) in 00:04:39 
...

For the entire terminal output see P12609.

I ran the failing test 10 times on my machine, targeting mediawiki-docker:

for i in {1..10}; do npm run selenium-test -- --spec tests/selenium/specs/talk.js --mochaOpts.grep newly; done
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:39 
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:36 
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:35 
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:39 
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:37 
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:38 
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:40 
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:38 
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:38 
...
Spec Files:	 1 passed, 1 total (100% completed) in 00:00:38 
...

For the entire terminal output see P12610.

Change 628947 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/MinervaNeue@master] Skip test

https://gerrit.wikimedia.org/r/628947