Page MenuHomePhabricator

Flaky wdio test "Rollback with confirmation: should offer a way to cancel"
Open, HighPublic1 Story Points

Description

[chrome #0-2] Running: chrome
[chrome #0-2]
[chrome #0-2]   Rollback with confirmation
[chrome #0-2]
[chrome #0-2]   Rollback with confirmation
[chrome #0-2]       ✓ should offer rollback options for admin users
[chrome #0-2]       1) should offer a way to cancel rollbacks
[chrome #0-2]       ✓ should perform rollbacks after confirming intention
[chrome #0-2]       ✓ should verify rollbacks via GET requests are confirmed on a follow-up page
[chrome #0-2]
[chrome #0-2]   Rollback without confirmation
[chrome #0-2]       ✓ should perform rollback via POST request without asking the user to confirm
[chrome #0-2]       ✓ should perform rollback via GET request without asking the user to confirm
[chrome #0-2]
[chrome #0-2]
[chrome #0-2] 5 passing (32s)
[chrome #0-2] 1 failing
[chrome #0-2]
[chrome #0-2] 1) Rollback with confirmation should offer a way to cancel rollbacks:
[chrome #0-2] An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-no").
[chrome #0-2] Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-no").
[chrome #0-2]     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
[chrome #0-2]     at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:53:37)
[chrome #0-2]

Event Timeline

Krinkle created this task.Mar 27 2019, 7:05 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMar 27 2019, 7:05 PM

Change 500051 had a related patch set uploaded (by Tonina Zhelyazkova; owner: Tonina Zhelyazkova):
[mediawiki/core@master] Fix flaky rollback browser test

https://gerrit.wikimedia.org/r/500051

Tonina_Zhelyazkova_WMDE moved this task from Doing to Review on the WMDE-FUN-Sprint-2019-03-18 board.
Tonina_Zhelyazkova_WMDE set the point value for this task to 1.

Change 500051 merged by jenkins-bot:
[mediawiki/core@master] Fix flaky rollback browser test

https://gerrit.wikimedia.org/r/500051

There is another one:

1) Rollback with confirmation should perform rollbacks after confirming intention:
An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
running chrome
Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
    at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
    at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:63:38)

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php71-docker/8584/console

And again:

1) Rollback with confirmation should perform rollbacks after confirming intention:
An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
running chrome
Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
    at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
    at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:63:38)

at https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/498310/

Tim_WMDE claimed this task.Apr 8 2019, 8:50 AM

Change 502162 had a related patch set uploaded (by Tim Eulitz; owner: Tim Eulitz):
[mediawiki/core@master] Wait for API to load & add additional pauses

https://gerrit.wikimedia.org/r/502162

Change 502375 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] tests: Disable flaky rollback wdio test

https://gerrit.wikimedia.org/r/502375

Tim_WMDE removed Tim_WMDE as the assignee of this task.Apr 9 2019, 8:59 AM
Tim_WMDE moved this task from Doing to Review on the WMDE-FUN-Sprint-2019-04-01 board.
Tim_WMDE added a subscriber: Tim_WMDE.

Change 502162 merged by jenkins-bot:
[mediawiki/core@master] selenium: Add API wait and other pauses in rollback test

https://gerrit.wikimedia.org/r/502162

Change 502375 abandoned by Krinkle:
tests: Disable flaky rollback wdio test

Reason:
Other patch landed already. A time-based wait is usually a time bomb, but we'll see.

https://gerrit.wikimedia.org/r/502375

Reedy added a subscriber: Reedy.Apr 14 2019, 9:44 PM

More fails

22:25:06 1 failing
22:25:06 
22:25:06 1) Rollback with confirmation should perform rollbacks after confirming intention:
22:25:06 An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
22:25:06 running chrome
22:25:06 Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
22:25:06     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
22:25:06     at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:68:38)
22:25:06

Change 503916 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/503916

Change 503916 merged by jenkins-bot:
[mediawiki/core@master] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/503916

Sorry for the failing tests, we will address this with priority in this ticket: https://phabricator.wikimedia.org/T220479

kai.nissen closed this task as Resolved.Apr 15 2019, 11:03 AM

Change 512399 had a related patch set uploaded (by Reedy; owner: Krinkle):
[mediawiki/core@REL1_33] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/512399

Change 512399 merged by jenkins-bot:
[mediawiki/core@REL1_33] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/512399

Reedy reopened this task as Open.May 24 2019, 5:34 PM

Seems to be more of these in REL1_33 (that doesn't seem fixed/disabled in master)

https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-mysql-php70-docker/19873/console

18:22:40 1 failing
18:22:40 
18:22:40 1) Rollback with confirmation should offer rollback options for admin users:
18:22:40 An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-text").
18:22:40 running chrome
18:22:40 Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-text").
18:22:40     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
18:22:40     at getText() - at Context.<anonymous> (tests/selenium/specs/rollback.js:46:55)
Krinkle raised the priority of this task from High to Unbreak Now!.May 29 2019, 3:34 PM
Krinkle added subscribers: Liuxinyu970226, Jdlrobson, hashar.

Seen affecting Minerva and AdvancedSearch as well.

Just a note, from the merged task. There is a CI build showing the issue at https://integration.wikimedia.org/ci/job/mediawiki-quibble-composer-mysql-php70-docker/17434/

Krinkle lowered the priority of this task from Unbreak Now! to High.Jun 10 2019, 3:15 PM
Krinkle added subscribers: awight, thiemowmde, gabriel-wmde.

I have seen this almost every day now, for weeks. Here is the most recent one: https://gerrit.wikimedia.org/r/514495.

18:21:39 1) Rollback with confirmation should offer a way to cancel rollbacks:
18:21:39 element (".mw-rollback-link .jquery-confirmable-button-no") still not visible after 5000ms
18:21:39 running chrome
18:21:39 Error: element (".mw-rollback-link .jquery-confirmable-button-no") still not visible after 5000ms
18:21:39     at elements(".mw-rollback-link .jquery-confirmable-button-no") - isVisible.js:54:17
18:21:39     at isVisible(".mw-rollback-link .jquery-confirmable-button-no") - waitForVisible.js:73:22

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php70-docker/22363/console
It almost always disappears with a recheck. But that's a really dangerous thing to do: the more it happens, the more people get used to it, and will more and more blindly do it instead of taking failing tests serious.

Change 514686 had a related patch set uploaded (by Krinkle; owner: Awight):
[mediawiki/core@master] selenium: Skip flaky Rollback test

https://gerrit.wikimedia.org/r/514686

Change 514686 merged by jenkins-bot:
[mediawiki/core@master] selenium: Skip flaky Rollback test

https://gerrit.wikimedia.org/r/514686

Change 519440 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] Skip flaky Selenium test

https://gerrit.wikimedia.org/r/519440

Change 519462 had a related patch set uploaded (by Krinkle; owner: Kosta Harlan):
[mediawiki/core@wmf/1.34.0-wmf.11] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/519462

Change 519440 merged by jenkins-bot:
[mediawiki/core@master] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/519440

Change 519462 merged by jenkins-bot:
[mediawiki/core@wmf/1.34.0-wmf.11] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/519462

mmodell changed the subtype of this task from "Task" to "Production Error".Wed, Aug 28, 11:07 PM