Page MenuHomePhabricator
Create Task
Maniphest T199644

Selenium failures on wmf-quibble-vendor-mysql-*-docker jobs for gate and submit
Closed, ResolvedPublic
Actions

Description

We currently have 4 patches on AbuseFilter dashboard which have been reviewed, and the two jobs of gate-and-submit (wmf-quibble-vendor-mysql-php70-docker and wmf-quibble-vendor-mysql-hhvm-docker) failed for all of them, for the same test, which is a Selenium one. It's pretty clear to me that those failure are unrelated, also looking at recent build fails. Here is the failing test (from https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-php70-docker/343/consoleFull):

10:08:46 [chrome #0-2] Session ID: 04263bb5fe5fcdc3691bb202514b7485
10:08:46 [chrome #0-2] Spec: /workspace/src/tests/selenium/specs/specialrecentchanges.js
10:08:46 [chrome #0-2] Running: chrome
10:08:46 [chrome #0-2]
10:08:46 [chrome #0-2]   Special:RecentChanges
10:08:46 [chrome #0-2]
10:08:46 [chrome #0-2]   Special:RecentChanges
10:08:46 [chrome #0-2]       1) shows page creation
10:08:46 [chrome #0-2]
10:08:46 [chrome #0-2]
10:08:46 [chrome #0-2] 1 failing (5s)
10:08:46 [chrome #0-2]
10:08:46 [chrome #0-2] 1) Special:RecentChanges shows page creation:
10:08:46 [chrome #0-2] element not visible
10:08:46 [chrome #0-2] Error: An element command could not be completed because the element is not visible on the page.
10:08:46 [chrome #0-2]     at EditPage.edit (tests/selenium/pageobjects/edit.page.js:17:13)
10:08:46 [chrome #0-2]     at Context.<anonymous> (tests/selenium/specs/specialrecentchanges.js:17:12)
10:08:46 [chrome #0-2]     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
10:08:46 [chrome #0-2]     at elementIdClick("0.12618929670787482-2") - click.js:20:22
10:08:46 [chrome #0-2]

and consequently

10:09:36 14 passing (98.10s)
10:09:36 1 failing
10:09:36 
10:09:36 1) Special:RecentChanges shows page creation:
10:09:36 element not visible
10:09:36 running chrome
10:09:36 Error: An element command could not be completed because the element is not visible on the page.
10:09:36     at EditPage.edit (tests/selenium/pageobjects/edit.page.js:17:13)
10:09:36     at Context.<anonymous> (tests/selenium/specs/specialrecentchanges.js:17:12)
10:09:36     at Promise.F (node_modules/core-js/library/modules/_export.js:36:28)
10:09:36     at elementIdClick("0.12618929670787482-2") - click.js:20:22

Details

Related Changes in Gerrit:
SubjectRepoBranchLines +/-
selenium: disable beta dialog popupmediawiki/extensions/VisualEditormaster+3 -0
VisualEditor now disables the beta welcome popupmediawiki/coremaster+1 -14
(DO NOT SUBMIT) Test change for T199644mediawiki/extensions/AbuseFiltermaster+0 -0
(DO NOT SUBMIT) Test change for T199644mediawiki/extensions/Thanksmaster+0 -0
Temporarily skip broken Selenium testmediawiki/coremaster+2 -1
Customize query in gerrit

Related Objects

Event Timeline

Daimona created this task.Jul 15 2018, 10:41 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 15 2018, 10:41 AM
Daimona triaged this task as High priority.Jul 15 2018, 10:42 AM

To me this is surely high, given all these unrelated failures. Not setting to UBN! since I'm not completely sure whether this affects other repos as well.

Daimona added a comment.EditedJul 15 2018, 11:21 AM

I also guess J108 could be related? Actually, that's "14 Jun", while I misread it as "14 Jul"

Daimona raised the priority of this task from High to Unbreak Now!.Jul 15 2018, 12:01 PM

Just noticed https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-hhvm-docker/1242/console for Extension:Thanks, and the job is executed (and fails) for the main build as well. I feel this is really UBN!, at least removing the broken test per standard practice.

Restricted Application added subscribers: Liuxinyu970226, TerraCodes. · View Herald TranscriptJul 15 2018, 12:01 PM
matej_suchanek added a subscriber: • Pablo-WMDE.Jul 15 2018, 12:15 PM
matej_suchanek subscribed.

I suspect T199446 & rMW42f844e04a805b54dfdcb4c802a4dc5d50ecec7f.

Daimona added a comment.Jul 15 2018, 12:27 PM

Very likely. Also, the commit above was merged right between the last successful build and the first faulty one.

gerritbot subscribed.Jul 15 2018, 1:39 PM

Change 445858 had a related patch set uploaded (by Daimona Eaytoy; owner: Daimona Eaytoy):
[mediawiki/core@master] Temporarily skip broken Selenium test

https://gerrit.wikimedia.org/r/445858

gerritbot added a project: Patch-For-Review.Jul 15 2018, 1:39 PM
gerritbot added a comment.Jul 15 2018, 2:30 PM

Change 445858 merged by jenkins-bot:
[mediawiki/core@master] Temporarily skip broken Selenium test

https://gerrit.wikimedia.org/r/445858

Daimona lowered the priority of this task from Unbreak Now! to High.Jul 15 2018, 2:35 PM

The urgent part is done.

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-07-17 (1.32.0-wmf.13)).Jul 15 2018, 3:00 PM
Daimona removed projects: Patch-For-Review, AbuseFilter.Jul 15 2018, 3:52 PM
• Pablo-WMDE added a comment.Jul 16 2018, 7:32 AM

Will try and initiate a little retrospective on why WIP https://gerrit.wikimedia.org/r/445630 got merged.

WMDE-leszek subscribed.Jul 16 2018, 8:17 AM
hashar added a subscriber: zeljkofilipin.Jul 16 2018, 9:10 AM
hashar subscribed.Jul 16 2018, 9:19 AM

When an extension depends on VisualEditor and has Selenium tests, some tests will fail because of a popup showing up from VisualEditor. For example the show page creation

shows-page-creation.png (894×1 px, 97 KB)

That started to hit the CI jobs because of https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/445630/ which changed a mediawiki/core test to create a page using the user interface instead of the API.

To disable the VisualEditor dialog, we have the test suite to set a cookie ': ve-beta-welcome-dialog: 1. Seems we could do that in VisualEditor itself by adding a tests/selenium/LocalSettings.php which would have:

<?php
$wgVisualEditorShowBetaWelcome = false;

Which in turns require Quibble to support injecting those LocalSettings.php files. That is T198201

gerritbot added a comment.Jul 16 2018, 9:35 AM

Change 445963 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/VisualEditor@master] selenium: disable beta dialog popup

https://gerrit.wikimedia.org/r/445963

gerritbot added a project: Patch-For-Review.Jul 16 2018, 9:35 AM
matej_suchanek unsubscribed.Jul 16 2018, 9:35 AM
gerritbot added a comment.Jul 16 2018, 10:06 AM

Change 445967 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/core@master] VisualEditor now disables the beta welcome popup

https://gerrit.wikimedia.org/r/445967

gerritbot added a comment.Jul 16 2018, 10:30 AM

Change 445973 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/Thanks@master] (DO NOT SUBMIT) Test change for T199644

https://gerrit.wikimedia.org/r/445973

hashar mentioned this in T198201: Quibble must include tests/selenium/LocalSettings.php (was ORES webdriver.io selenium test fail on CI due to lack of ORES server).Jul 16 2018, 10:33 AM
gerritbot added a comment.Jul 16 2018, 11:20 AM

Change 445981 had a related patch set uploaded (by Hashar; owner: Hashar):
[mediawiki/extensions/AbuseFilter@master] (DO NOT SUBMIT) Test change for T199644

https://gerrit.wikimedia.org/r/445981

zeljkofilipin added projects: User-zeljkofilipin, MediaWiki-Core-Tests.Jul 16 2018, 12:24 PM
zeljkofilipin moved this task from Inbox to Selenium on the MediaWiki-Core-Tests board.
gerritbot added a comment.Aug 27 2018, 1:42 PM

Change 445973 abandoned by Hashar:
(DO NOT SUBMIT) Test change for T199644

https://gerrit.wikimedia.org/r/445973

gerritbot added a comment.Aug 27 2018, 1:42 PM

Change 445981 abandoned by Hashar:
(DO NOT SUBMIT) Test change for T199644

https://gerrit.wikimedia.org/r/445981

gerritbot added a comment.Aug 27 2018, 1:43 PM

Change 445967 abandoned by Hashar:
VisualEditor now disables the beta welcome popup

https://gerrit.wikimedia.org/r/445967

gerritbot added a comment.Oct 31 2018, 2:27 PM

Change 445963 abandoned by Hashar:
selenium: disable beta dialog popup

Reason:
I have abandoned the dependent change https://gerrit.wikimedia.org/r/#/c/mediawiki/core/ /445971/

The fix is that the testsuite will be run directly from VisualEditor repo (npm selenium-test) instead of centrally from mediawiki/core.

https://gerrit.wikimedia.org/r/445963

hashar removed projects: Patch-For-Review, Continuous-Integration-Config.Aug 27 2019, 12:35 PM

The recent change test in MediaWiki core had been disabled at the time via:

[mediawiki/core@master] Temporarily skip broken Selenium test
https://gerrit.wikimedia.org/r/445858

https://gerrit.wikimedia.org/r/445630 is "selenium: create edit for RecentChanges through GUI"

@Pablo-WMDE mentioned an issue with the post send jobs / deferred updates which were not run. We had the issue several time when writing tests that look at RecentChanges. IIRC the solution has been to add a busy loop that drains the queue of jobs.

So tentatively the faulty test might be enabled again if it has not been done so already.

• Pablo-WMDE added a comment.Aug 27 2019, 4:24 PM

The feature @hashar describes is abstracted behind [[ https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/tests/selenium/wdio-mediawiki/RunJobs.js#64 | RunJobs.run() ]]. Its implementation was discussed at length when introduced and a compromise - but it should work™.

Once again, it was never my wish for https://gerrit.wikimedia.org/r/445630 to get merged (was WIP to try something), consequently [[ https://gerrit.wikimedia.org/r/plugins/gitiles/mediawiki/core/+/master/tests/selenium/specs/specialrecentchanges.js#21 | specs/specialrecentchanges.js ]] today stands as it does, using the API + RunJobs.run() during test setup phase.

What made this resurface? Should probably be closed, right?

hashar assigned this task to • Pablo-WMDE.Aug 29 2019, 7:27 AM

And @Pablo-WMDE patch reenabled the RecentChange test https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/459800/4/tests/selenium/specs/specialrecentchanges.js

So yeah this task can be closed due to:

  • VisualEditor popup is now dismissed when creating a page via the GUI
  • The suite runs background job which populate the recent changes table and thus let us assert whether a page show up there

Thank you for the detailed reply Pablo!

• Pablo-WMDE reassigned this task from • Pablo-WMDE to Daimona.Aug 29 2019, 7:49 AM
Krinkle closed this task as Resolved.Oct 1 2019, 9:21 PM
Krinkle edited projects, added ci-test-error, Wikimedia-production-error (ARCHIVED -- Shared Build Failure); removed MediaWiki-Core-Tests, MW-1.32-notes (WMF-deploy-2018-07-17 (1.32.0-wmf.13)).
Krinkle subscribed.

Seems to have been resolved. Do re-open if this is not the case.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits