Page MenuHomePhabricator

Flaky wdio test "Rollback with confirmation: should offer a way to cancel"
Closed, ResolvedPublic1 Estimated Story PointsPRODUCTION ERROR

Description

[chrome #0-2] Running: chrome
[chrome #0-2]
[chrome #0-2]   Rollback with confirmation
[chrome #0-2]
[chrome #0-2]   Rollback with confirmation
[chrome #0-2]       ✓ should offer rollback options for admin users
[chrome #0-2]       1) should offer a way to cancel rollbacks
[chrome #0-2]       ✓ should perform rollbacks after confirming intention
[chrome #0-2]       ✓ should verify rollbacks via GET requests are confirmed on a follow-up page
[chrome #0-2]
[chrome #0-2]   Rollback without confirmation
[chrome #0-2]       ✓ should perform rollback via POST request without asking the user to confirm
[chrome #0-2]       ✓ should perform rollback via GET request without asking the user to confirm
[chrome #0-2]
[chrome #0-2]
[chrome #0-2] 5 passing (32s)
[chrome #0-2] 1 failing
[chrome #0-2]
[chrome #0-2] 1) Rollback with confirmation should offer a way to cancel rollbacks:
[chrome #0-2] An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-no").
[chrome #0-2] Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-no").
[chrome #0-2]     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
[chrome #0-2]     at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:53:37)
[chrome #0-2]

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

Change 500051 had a related patch set uploaded (by Tonina Zhelyazkova; owner: Tonina Zhelyazkova):
[mediawiki/core@master] Fix flaky rollback browser test

https://gerrit.wikimedia.org/r/500051

Change 500051 merged by jenkins-bot:
[mediawiki/core@master] Fix flaky rollback browser test

https://gerrit.wikimedia.org/r/500051

There is another one:

1) Rollback with confirmation should perform rollbacks after confirming intention:
An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
running chrome
Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
    at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
    at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:63:38)

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php71-docker/8584/console

And again:

1) Rollback with confirmation should perform rollbacks after confirming intention:
An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
running chrome
Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
    at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
    at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:63:38)

at https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/498310/

Change 502162 had a related patch set uploaded (by Tim Eulitz; owner: Tim Eulitz):
[mediawiki/core@master] Wait for API to load & add additional pauses

https://gerrit.wikimedia.org/r/502162

Change 502375 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] tests: Disable flaky rollback wdio test

https://gerrit.wikimedia.org/r/502375

Tim_WMDE moved this task from Doing to Review on the WMDE-FUN-Sprint-2019-04-01 board.
Tim_WMDE subscribed.

Change 502162 merged by jenkins-bot:
[mediawiki/core@master] selenium: Add API wait and other pauses in rollback test

https://gerrit.wikimedia.org/r/502162

Change 502375 abandoned by Krinkle:
tests: Disable flaky rollback wdio test

Reason:
Other patch landed already. A time-based wait is usually a time bomb, but we'll see.

https://gerrit.wikimedia.org/r/502375

More fails

22:25:06 1 failing
22:25:06 
22:25:06 1) Rollback with confirmation should perform rollbacks after confirming intention:
22:25:06 An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
22:25:06 running chrome
22:25:06 Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
22:25:06     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
22:25:06     at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:68:38)
22:25:06

Change 503916 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/503916

Change 503916 merged by jenkins-bot:
[mediawiki/core@master] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/503916

Sorry for the failing tests, we will address this with priority in this ticket: https://phabricator.wikimedia.org/T220479

Change 512399 had a related patch set uploaded (by Reedy; owner: Krinkle):
[mediawiki/core@REL1_33] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/512399

Change 512399 merged by jenkins-bot:
[mediawiki/core@REL1_33] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/512399

Seems to be more of these in REL1_33 (that doesn't seem fixed/disabled in master)

https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-mysql-php70-docker/19873/console

18:22:40 1 failing
18:22:40 
18:22:40 1) Rollback with confirmation should offer rollback options for admin users:
18:22:40 An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-text").
18:22:40 running chrome
18:22:40 Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-text").
18:22:40     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
18:22:40     at getText() - at Context.<anonymous> (tests/selenium/specs/rollback.js:46:55)
Krinkle lowered the priority of this task from Unbreak Now! to High.Jun 10 2019, 3:15 PM
Krinkle added subscribers: awight, thiemowmde, gabriel-wmde.

I have seen this almost every day now, for weeks. Here is the most recent one: https://gerrit.wikimedia.org/r/514495.

18:21:39 1) Rollback with confirmation should offer a way to cancel rollbacks:
18:21:39 element (".mw-rollback-link .jquery-confirmable-button-no") still not visible after 5000ms
18:21:39 running chrome
18:21:39 Error: element (".mw-rollback-link .jquery-confirmable-button-no") still not visible after 5000ms
18:21:39     at elements(".mw-rollback-link .jquery-confirmable-button-no") - isVisible.js:54:17
18:21:39     at isVisible(".mw-rollback-link .jquery-confirmable-button-no") - waitForVisible.js:73:22

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php70-docker/22363/console

It almost always disappears with a recheck. But that's a really dangerous thing to do: the more it happens, the more people get used to it, and will more and more blindly do it instead of taking failing tests serious.

Change 514686 had a related patch set uploaded (by Krinkle; owner: Awight):
[mediawiki/core@master] selenium: Skip flaky Rollback test

https://gerrit.wikimedia.org/r/514686

Change 514686 merged by jenkins-bot:
[mediawiki/core@master] selenium: Skip flaky Rollback test

https://gerrit.wikimedia.org/r/514686

Change 519440 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] Skip flaky Selenium test

https://gerrit.wikimedia.org/r/519440

Change 519462 had a related patch set uploaded (by Krinkle; owner: Kosta Harlan):
[mediawiki/core@wmf/1.34.0-wmf.11] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/519462

Change 519440 merged by jenkins-bot:
[mediawiki/core@master] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/519440

Change 519462 merged by jenkins-bot:
[mediawiki/core@wmf/1.34.0-wmf.11] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/519462

mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:07 PM

Change 554615 had a related patch set uploaded (by Reedy; owner: Kosta Harlan):
[mediawiki/core@REL1_33] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/554615

Change 554615 merged by jenkins-bot:
[mediawiki/core@REL1_33] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/554615

The test got removed completely see https://gerrit.wikimedia.org/r/c/mediawiki/core/+/681339. Before that it was disabled for over a year it seems. See T280652: Delete tests disabled for more than 1 year. I'll just leave that task open for now so we can have a last thought about re-adding the test.

WMDE-Fisch closed this task as Resolved.EditedJan 26 2022, 2:06 PM

It might make sense to re-add at least one browser tests for that functionality in the future. This currently de-prioritized by WMDE.

See T300141: Add basic browser test for rollback confirmation