Page MenuHomePhabricator
Create Task
Maniphest T219440

Flaky wdio test "Rollback with confirmation: should offer a way to cancel"
Closed, ResolvedPublic1 Estimated Story PointsPRODUCTION ERROR
Actions

Description

[chrome #0-2] Running: chrome
[chrome #0-2]
[chrome #0-2]   Rollback with confirmation
[chrome #0-2]
[chrome #0-2]   Rollback with confirmation
[chrome #0-2]       ✓ should offer rollback options for admin users
[chrome #0-2]       1) should offer a way to cancel rollbacks
[chrome #0-2]       ✓ should perform rollbacks after confirming intention
[chrome #0-2]       ✓ should verify rollbacks via GET requests are confirmed on a follow-up page
[chrome #0-2]
[chrome #0-2]   Rollback without confirmation
[chrome #0-2]       ✓ should perform rollback via POST request without asking the user to confirm
[chrome #0-2]       ✓ should perform rollback via GET request without asking the user to confirm
[chrome #0-2]
[chrome #0-2]
[chrome #0-2] 5 passing (32s)
[chrome #0-2] 1 failing
[chrome #0-2]
[chrome #0-2] 1) Rollback with confirmation should offer a way to cancel rollbacks:
[chrome #0-2] An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-no").
[chrome #0-2] Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-no").
[chrome #0-2]     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
[chrome #0-2]     at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:53:37)
[chrome #0-2]

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
selenium: Skip flaky rollback testmediawiki/coreREL1_33+1 -1
selenium: Skip flaky rollback testmediawiki/coremaster+1 -1
selenium: Skip flaky rollback testmediawiki/corewmf/1.34.0-wmf.11+1 -1
selenium: Skip flaky Rollback testmediawiki/coremaster+1 -1
selenium: Disable flaky wdio rollback tests that rely on pausesmediawiki/coreREL1_33+2 -2
selenium: Disable flaky wdio rollback tests that rely on pausesmediawiki/coremaster+2 -2
tests: Disable flaky rollback wdio testmediawiki/coremaster+2 -0
selenium: Add API wait and other pauses in rollback testmediawiki/coremaster+5 -0
Fix flaky rollback browser testmediawiki/coremaster+1 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
gerritbot added a comment.Mar 29 2019, 2:32 PM

Change 500051 had a related patch set uploaded (by Tonina Zhelyazkova; owner: Tonina Zhelyazkova):
[mediawiki/core@master] Fix flaky rollback browser test

https://gerrit.wikimedia.org/r/500051

gerritbot added a project: Patch-For-Review.Mar 29 2019, 2:32 PM
• Tonina_Zhelyazkova_WMDE removed • Tonina_Zhelyazkova_WMDE as the assignee of this task.Mar 29 2019, 2:44 PM
• Tonina_Zhelyazkova_WMDE moved this task from Doing to Review on the WMDE-FUN-Sprint-2019-03-18 board.
• Tonina_Zhelyazkova_WMDE set the point value for this task to 1.
• Tonina_Zhelyazkova_WMDE subscribed.
gerritbot added a comment.Mar 29 2019, 2:53 PM

Change 500051 merged by jenkins-bot:
[mediawiki/core@master] Fix flaky rollback browser test

https://gerrit.wikimedia.org/r/500051

ReleaseTaggerBot added a project: MW-1.33-notes (1.33.0-wmf.24; 2019-04-02).Mar 29 2019, 3:01 PM
Krinkle moved this task from Untriaged to Resolved on the Wikimedia-production-error (ARCHIVED -- Shared Build Failure) board.Mar 30 2019, 12:20 AM
• Tonina_Zhelyazkova_WMDE moved this task from Review to Done on the WMDE-FUN-Sprint-2019-03-18 board.Apr 1 2019, 9:13 AM
Umherirrender subscribed.Apr 5 2019, 4:18 PM

There is another one:

1) Rollback with confirmation should perform rollbacks after confirming intention:
An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
running chrome
Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
    at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
    at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:63:38)

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php71-docker/8584/console

Krinkle added a comment.Apr 5 2019, 11:13 PM

And again:

1) Rollback with confirmation should perform rollbacks after confirming intention:
An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
running chrome
Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
    at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
    at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:63:38)

at https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/498310/

Krinkle triaged this task as High priority.Apr 5 2019, 11:13 PM
Krinkle removed projects: MW-1.33-notes (1.33.0-wmf.24; 2019-04-02), Patch-For-Review.
Krinkle moved this task from Resolved to Untriaged on the Wikimedia-production-error (ARCHIVED -- Shared Build Failure) board.
Jdforrester-WMF subscribed.Apr 6 2019, 7:57 PM
• Tim_WMDE claimed this task.Apr 8 2019, 8:50 AM
• Tim_WMDE added a project: WMDE-FUN-Sprint-2019-04-01.Apr 8 2019, 9:02 AM
• Tim_WMDE moved this task from Todo to Doing on the WMDE-FUN-Sprint-2019-04-01 board.
gerritbot added a comment.Apr 8 2019, 9:14 AM

Change 502162 had a related patch set uploaded (by Tim Eulitz; owner: Tim Eulitz):
[mediawiki/core@master] Wait for API to load & add additional pauses

https://gerrit.wikimedia.org/r/502162

gerritbot added a project: Patch-For-Review.Apr 8 2019, 9:14 AM
Krinkle moved this task from Untriaged to In progress on the Wikimedia-production-error (ARCHIVED -- Shared Build Failure) board.Apr 8 2019, 9:56 PM
gerritbot added a comment.Apr 9 2019, 12:45 AM

Change 502375 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] tests: Disable flaky rollback wdio test

https://gerrit.wikimedia.org/r/502375

• Tim_WMDE removed • Tim_WMDE as the assignee of this task.Apr 9 2019, 8:59 AM
• Tim_WMDE moved this task from Doing to Review on the WMDE-FUN-Sprint-2019-04-01 board.
• Tim_WMDE subscribed.
• Tim_WMDE moved this task from Backlog to Doing on the Confirmation prompt for rollback action board.Apr 9 2019, 9:22 AM
gerritbot added a comment.Apr 9 2019, 10:46 AM

Change 502162 merged by jenkins-bot:
[mediawiki/core@master] selenium: Add API wait and other pauses in rollback test

https://gerrit.wikimedia.org/r/502162

ReleaseTaggerBot added a project: MW-1.33-notes (1.33.0-wmf.25; 2019-04-09).Apr 9 2019, 11:01 AM
gerritbot added a comment.Apr 9 2019, 5:13 PM

Change 502375 abandoned by Krinkle:
tests: Disable flaky rollback wdio test

Reason:
Other patch landed already. A time-based wait is usually a time bomb, but we'll see.

https://gerrit.wikimedia.org/r/502375

Krinkle moved this task from In progress to Resolved on the Wikimedia-production-error (ARCHIVED -- Shared Build Failure) board.Apr 9 2019, 5:14 PM
Krinkle removed a project: Patch-For-Review.
• Tonina_Zhelyazkova_WMDE moved this task from Review to Done on the WMDE-FUN-Sprint-2019-04-01 board.Apr 10 2019, 9:38 AM
Krinkle moved this task from Resolved to Untriaged on the Wikimedia-production-error (ARCHIVED -- Shared Build Failure) board.Apr 14 2019, 9:43 PM
Reedy subscribed.Apr 14 2019, 9:44 PM

More fails

22:25:06 1 failing
22:25:06 
22:25:06 1) Rollback with confirmation should perform rollbacks after confirming intention:
22:25:06 An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
22:25:06 running chrome
22:25:06 Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-button-yes").
22:25:06     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
22:25:06     at click() - at Context.<anonymous> (tests/selenium/specs/rollback.js:68:38)
22:25:06
gerritbot added a comment.Apr 14 2019, 10:14 PM

Change 503916 had a related patch set uploaded (by Krinkle; owner: Krinkle):
[mediawiki/core@master] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/503916

gerritbot added a project: Patch-For-Review.Apr 14 2019, 10:14 PM
gerritbot added a comment.Apr 14 2019, 10:54 PM

Change 503916 merged by jenkins-bot:
[mediawiki/core@master] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/503916

Krinkle moved this task from Untriaged to Resolved on the Wikimedia-production-error (ARCHIVED -- Shared Build Failure) board.Apr 14 2019, 10:56 PM
ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.1; 2019-04-16).Apr 14 2019, 11:00 PM
• Tim_WMDE added a comment.Apr 15 2019, 8:05 AM

Sorry for the failing tests, we will address this with priority in this ticket: https://phabricator.wikimedia.org/T220479

kai.nissen closed this task as Resolved.Apr 15 2019, 11:03 AM
gerritbot added a comment.May 24 2019, 4:50 PM

Change 512399 had a related patch set uploaded (by Reedy; owner: Krinkle):
[mediawiki/core@REL1_33] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/512399

gerritbot added a comment.May 24 2019, 5:05 PM

Change 512399 merged by jenkins-bot:
[mediawiki/core@REL1_33] selenium: Disable flaky wdio rollback tests that rely on pauses

https://gerrit.wikimedia.org/r/512399

Reedy removed a project: Patch-For-Review.May 24 2019, 5:12 PM
Reedy reopened this task as Open.May 24 2019, 5:34 PM

Seems to be more of these in REL1_33 (that doesn't seem fixed/disabled in master)

https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-mysql-php70-docker/19873/console

18:22:40 1 failing
18:22:40 
18:22:40 1) Rollback with confirmation should offer rollback options for admin users:
18:22:40 An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-text").
18:22:40 running chrome
18:22:40 Error: An element could not be located on the page using the given search parameters (".mw-rollback-link .jquery-confirmable-text").
18:22:40     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
18:22:40     at getText() - at Context.<anonymous> (tests/selenium/specs/rollback.js:46:55)
ReleaseTaggerBot edited projects, added MW-1.33-notes; removed MW-1.33-notes (1.33.0-wmf.25; 2019-04-09).May 25 2019, 4:01 AM
Krinkle raised the priority of this task from High to Unbreak Now!.May 29 2019, 3:34 PM
Krinkle moved this task from Resolved to Untriaged on the Wikimedia-production-error (ARCHIVED -- Shared Build Failure) board.
Krinkle merged a task: T221461: "Rollback with confirmation should offer rollback options for admin users" test flaky.
Krinkle added subscribers: Liuxinyu970226, Jdlrobson, hashar.

Seen affecting Minerva and AdvancedSearch as well.

hashar added a comment.May 29 2019, 3:53 PM

Just a note, from the merged task. There is a CI build showing the issue at https://integration.wikimedia.org/ci/job/mediawiki-quibble-composer-mysql-php70-docker/17434/

Krinkle lowered the priority of this task from Unbreak Now! to High.Jun 10 2019, 3:15 PM
Krinkle moved this task from Untriaged to Resolved on the Wikimedia-production-error (ARCHIVED -- Shared Build Failure) board.
Krinkle merged a task: T225163: Spurious timeout in MediaWiki's rollback browser test.
Krinkle added a parent task: T225162: Fix flapping tests in gated extensions as of 2019-06-06.
Krinkle added subscribers: awight, thiemowmde, gabriel-wmde.
In T225163#5239674, @thiemowmde wrote:

I have seen this almost every day now, for weeks. Here is the most recent one: https://gerrit.wikimedia.org/r/514495.

18:21:39 1) Rollback with confirmation should offer a way to cancel rollbacks:
18:21:39 element (".mw-rollback-link .jquery-confirmable-button-no") still not visible after 5000ms
18:21:39 running chrome
18:21:39 Error: element (".mw-rollback-link .jquery-confirmable-button-no") still not visible after 5000ms
18:21:39     at elements(".mw-rollback-link .jquery-confirmable-button-no") - isVisible.js:54:17
18:21:39     at isVisible(".mw-rollback-link .jquery-confirmable-button-no") - waitForVisible.js:73:22

https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-php70-docker/22363/console

It almost always disappears with a recheck. But that's a really dangerous thing to do: the more it happens, the more people get used to it, and will more and more blindly do it instead of taking failing tests serious.

gerritbot added a comment.Jun 10 2019, 3:18 PM

Change 514686 had a related patch set uploaded (by Krinkle; owner: Awight):
[mediawiki/core@master] selenium: Skip flaky Rollback test

https://gerrit.wikimedia.org/r/514686

gerritbot added a project: Patch-For-Review.Jun 10 2019, 3:18 PM
gerritbot added a comment.Jun 10 2019, 3:33 PM

Change 514686 merged by jenkins-bot:
[mediawiki/core@master] selenium: Skip flaky Rollback test

https://gerrit.wikimedia.org/r/514686

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.10; 2019-06-18); removed MW-1.34-notes (1.34.0-wmf.1; 2019-04-16).Jun 10 2019, 4:00 PM
xSavitar subscribed.Jun 12 2019, 4:51 PM

Happened again here: https://integration.wikimedia.org/ci/job/mediawiki-quibble-vendor-postgres-php72-docker/803/console

gerritbot added a comment.Jun 27 2019, 4:06 PM

Change 519440 had a related patch set uploaded (by Kosta Harlan; owner: Kosta Harlan):
[mediawiki/core@master] Skip flaky Selenium test

https://gerrit.wikimedia.org/r/519440

gerritbot added a comment.Jun 27 2019, 5:21 PM

Change 519462 had a related patch set uploaded (by Krinkle; owner: Kosta Harlan):
[mediawiki/core@wmf/1.34.0-wmf.11] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/519462

gerritbot added a comment.Jun 27 2019, 5:34 PM

Change 519440 merged by jenkins-bot:
[mediawiki/core@master] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/519440

gerritbot added a comment.Jun 27 2019, 5:34 PM

Change 519462 merged by jenkins-bot:
[mediawiki/core@wmf/1.34.0-wmf.11] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/519462

ReleaseTaggerBot edited projects, added MW-1.34-notes (1.34.0-wmf.11; 2019-06-26); removed MW-1.34-notes (1.34.0-wmf.10; 2019-06-18).Jun 27 2019, 6:00 PM
Maintenance_bot removed a project: Patch-For-Review.Jun 27 2019, 6:10 PM
• mmodell changed the subtype of this task from "Task" to "Production Error".Aug 28 2019, 11:07 PM
gerritbot added a comment.Dec 4 2019, 8:15 PM

Change 554615 had a related patch set uploaded (by Reedy; owner: Kosta Harlan):
[mediawiki/core@REL1_33] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/554615

gerritbot added a project: Patch-For-Review.Dec 4 2019, 8:15 PM
gerritbot added a comment.Dec 4 2019, 8:22 PM

Change 554615 merged by jenkins-bot:
[mediawiki/core@REL1_33] selenium: Skip flaky rollback test

https://gerrit.wikimedia.org/r/554615

Maintenance_bot removed a project: Patch-For-Review.Dec 4 2019, 9:10 PM
hashar unsubscribed.Mar 30 2020, 3:36 PM
Krinkle edited projects, added ci-test-error (WMF-deployed Build Failure); removed Wikimedia-production-error (ARCHIVED -- Shared Build Failure).Apr 18 2020, 10:53 PM
Krinkle removed projects: ci-test-error (WMF-deployed Build Failure), MW-1.34-notes (1.34.0-wmf.11; 2019-06-26), MW-1.33-notes.Apr 19 2020, 5:38 PM
Umherirrender unsubscribed.Feb 27 2021, 9:42 PM
WMDE-Fisch subscribed.Apr 20 2021, 7:03 AM
thiemowmde added projects: WMDE-TechWish-Maintenance, Browser-Tests.Jan 10 2022, 5:26 PM
thiemowmde moved this task from Incoming to Priority backlog on the WMDE-TechWish-Maintenance board.Jan 18 2022, 1:35 PM
WMDE-Fisch added a project: WMDE-TechWish-Sprint-2022-01-19.Jan 24 2022, 3:00 PM

The test got removed completely see https://gerrit.wikimedia.org/r/c/mediawiki/core/+/681339. Before that it was disabled for over a year it seems. See T280652: Delete tests disabled for more than 1 year. I'll just leave that task open for now so we can have a last thought about re-adding the test.

WMDE-Fisch moved this task from Sprint Backlog to Tech Review on the WMDE-TechWish-Sprint-2022-01-19 board.Jan 24 2022, 3:00 PM
WMDE-Fisch moved this task from Priority backlog to In progress on the WMDE-TechWish-Maintenance board.
WMDE-Fisch closed this task as Resolved.EditedJan 26 2022, 2:06 PM

It might make sense to re-add at least one browser tests for that functionality in the future. This currently de-prioritized by WMDE.

See T300141: Add basic browser test for rollback confirmation

WMDE-Fisch moved this task from Tech Review to Done on the WMDE-TechWish-Sprint-2022-01-19 board.Jan 26 2022, 2:23 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits