Flaky selenium test: WikibaseReferenceOnProtectedPage
Closed, ResolvedPublic
Actions

Description

Recently I've been consistently facing a flaky selenium test from Wikibase/client/data-bridge, which is making the build fail on other repos. I've spotted this error roughly 5 times in the last week, for AbuseFilter patches. Recent example:

01:24:19 [chrome 73.0.3683.75 Linux #0-3] Spec: /workspace/src/extensions/Wikibase/repo/tests/selenium/specs/readmode.references.js
01:24:19 [chrome 73.0.3683.75 Linux #0-3] Running: chrome (v73.0.3683.75) on Linux
01:24:19 [chrome 73.0.3683.75 Linux #0-3] Session ID: f78ee0e565857c3801d9ec06375be73c
01:24:19 [chrome 73.0.3683.75 Linux #0-3]
01:24:19 [chrome 73.0.3683.75 Linux #0-3] WikibaseReferenceOnProtectedPage
01:24:19 [chrome 73.0.3683.75 Linux #0-3]    ✖ can expand collapsed references on a protected page as unprivileged user
01:24:19 [chrome 73.0.3683.75 Linux #0-3]
01:24:19 [chrome 73.0.3683.75 Linux #0-3] 1 failing (28.1s)
01:24:19 [chrome 73.0.3683.75 Linux #0-3]
01:24:19 [chrome 73.0.3683.75 Linux #0-3] 1) WikibaseReferenceOnProtectedPage can expand collapsed references on a protected page as unprivileged user
01:24:19 [chrome 73.0.3683.75 Linux #0-3] element (".wikibase-statementview-references-container .wikibase-statementview-references-heading a.ui-toggler") still not displayed after 10000ms
01:24:19 [chrome 73.0.3683.75 Linux #0-3] Error: element (".wikibase-statementview-references-container .wikibase-statementview-references-heading a.ui-toggler") still not displayed after 10000ms
01:24:19 [chrome 73.0.3683.75 Linux #0-3]     at Context.<anonymous> (/workspace/src/extensions/Wikibase/repo/tests/selenium/specs/readmode.references.js:19:111)

and the screenshot doesn't seem to show anything suspicious.

Related Objects

Mentioned Here: T354841: Decide on how to move forward with Wikidata Bridge browser testing
T264926: Wikibase Selenium tests are flaky

Event Timeline

Daimona created this task.Oct 7 2020, 8:12 AM

Restricted Application added a project: Wikidata. · View Herald TranscriptOct 7 2020, 8:12 AM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

I'm seeing this error more and more, e.g. today on r629233. I'm unsure about the status for other gated extensions, but it's quite annoying. Could someone please take a look?

Happened again on https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/634030

See also T264926: Wikibase Selenium tests are flaky which also lists this error and a few others.

https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/65707/console for https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Translate/+/636550

There seem to be more flaky test, not only in WB but also elsewhere (e.g. AbuseFilter and FileImporter at least). I'm convincing myself that it might be a wdio bug, as per comments at T264926.

Still happening https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/74304/console

13:51:56 [Chrome 73.0.3683.75 linux #0-3] 1 failing (17.9s)
13:51:56 [Chrome 73.0.3683.75 linux #0-3]
13:51:56 [Chrome 73.0.3683.75 linux #0-3] 1) WikibaseReferenceOnProtectedPage can expand collapsed references on a protected page as unprivileged user
13:51:56 [Chrome 73.0.3683.75 linux #0-3] element (".wikibase-statementview-references-container .wikibase-statementview-references-heading a.ui-toggler") still not displayed after 10000ms

It's very hard to investigate this without the full logs.
All logs that were linked to in this task have already expired.
Thus I will mark as stalled now waiting for this further data / input

Ammarpad unsubscribed.Mar 8 2021, 6:29 AM

Lens0021 subscribed.Oct 15 2021, 10:42 AM

Michael added a project: Browser-Tests.Aug 24 2022, 11:56 AM

Restricted Application added a project: [DEPRECATED] wdwb-tech. · View Herald TranscriptAug 24 2022, 11:56 AM

Addshore unsubscribed.Jun 27 2023, 12:38 PM

Currently those tests are no longer running, pending the decision on T354841: Decide on how to move forward with Wikidata Bridge browser testing and its implementation. When that is done, this task can be revisited.

Restricted Application added a project: wmde-wikidata-tech. · View Herald TranscriptMar 6 2024, 10:33 AM

Michael moved this task from Incoming to Tech/Ready-ish on the Wikidata-Bridge board.Mar 6 2024, 10:34 AM

Michael moved this task from Backlog to Wikidata Bridge on the MediaWiki-extensions-WikibaseClient board.

I’m confused, why are we saying this is a bridge test? The pasted output says it was repo/tests/selenium/specs/readmode.references.js, which is something in repo unrelated to Bridge (which is in client).

Either way – I don’t remember seeing this test fail. Has anyone else experienced this issue recently?

Oh sorry, I must have gotten confused by the initial paragraph and did not look closely at the quoted error. You're correct, this is not about bridge.

Let’s just close it – if anyone else sees it, please reopen.

Flaky selenium test: WikibaseReferenceOnProtectedPageClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

Flaky selenium test: WikibaseReferenceOnProtectedPage
Closed, ResolvedPublic
Actions