Page MenuHomePhabricator
Create Task
Maniphest T187859

Move one Selenium tests from mediawiki/core to mediawiki/skins/Vector
Closed, ResolvedPublic
Actions

Description

Status

Blocked on understanding the problem. See T187859#4054069 for details.

Description

Move Selenium test from 404080 from mediawiki/core to mediawiki/skins/Vector. The test is causing problems for other extensions and skins. See T185928 for more information.

The original commit (413157) caused problems for mediawiki/skins/MinervaNeue. See T188553 for more information. @Jdlrobson has reverted it (415393) until CI is fixed. The original commit is restored (but not merged) in 415614. See topic:T187859 for all commits.

Details

SubjectRepoBranchLines +/-
WIP Move one Selenium tests from mediawiki/core to mediawiki/skins/Vectormediawiki/skins/MinervaNeuemaster+97 -0
WIP: Move Selenium test from mediawiki/coremediawiki/skins/Vectormaster+96 -0
Move Selenium test from mediawiki/coremediawiki/skins/Vectormaster+96 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

zeljkofilipin created this task.Feb 21 2018, 9:58 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 21 2018, 9:58 AM
zeljkofilipin mentioned this in T185928: New usermessage browser test is blocking merges in Minerva skin and Echo extension.Feb 21 2018, 9:59 AM
zeljkofilipin triaged this task as Low priority.Feb 21 2018, 10:05 AM
zeljkofilipin moved this task from Backlog to In-progress on the Release-Engineering-Team (Kanban) board.
zeljkofilipin moved this task from Backlog 🪒 to Deep work 🌊 on the User-zeljkofilipin board.
zeljkofilipin renamed this task from Selenium tests for Vector skin to Move one Selenium tests from mediawiki/core to mediawiki/skins/Vector.Feb 21 2018, 10:12 AM
zeljkofilipin added a parent task: T182421: Q3 Selenium framework improvements.
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)
Addshore awarded a token.Feb 21 2018, 11:20 AM
gerritbot subscribed.Feb 21 2018, 1:02 PM

Change 413157 had a related patch set uploaded (by Zfilipin; owner: Addshore):
[mediawiki/skins/Vector@master] WIP Move one Selenium tests from mediawiki/core

https://gerrit.wikimedia.org/r/413157

gerritbot added a project: Patch-For-Review.Feb 21 2018, 1:02 PM
zeljkofilipin added a parent task: T185928: New usermessage browser test is blocking merges in Minerva skin and Echo extension.Feb 21 2018, 1:16 PM
greg moved this task from In-progress to Blocked (externally) on the Release-Engineering-Team (Kanban) board.Feb 21 2018, 4:52 PM
zeljkofilipin moved this task from Deep work 🌊 to Waiting 🏹 on the User-zeljkofilipin board.Feb 27 2018, 10:29 AM
gerritbot added a comment.Feb 28 2018, 6:52 PM

Change 413157 merged by jenkins-bot:
[mediawiki/skins/Vector@master] Move Selenium test from mediawiki/core

https://gerrit.wikimedia.org/r/413157

zeljkofilipin moved this task from Waiting 🏹 to Q4 👔 on the User-zeljkofilipin board.Mar 1 2018, 9:34 AM
zeljkofilipin added a comment.Mar 1 2018, 10:42 AM

This could help:

useskin=Vector
useskin=Minerva
gerritbot added a comment.Mar 1 2018, 6:21 PM

Change 415614 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/Vector@master] WIP: Move Selenium test from mediawiki/core

https://gerrit.wikimedia.org/r/415614

Jdlrobson added a comment.Mar 1 2018, 7:05 PM

useskin=Vector
useskin=Minerva

I don't follow... the Minerva skin should not be installing Vector in order to run its browser tests, so Vector should not exist and its browser tests should not be running.

zeljkofilipin moved this task from Q4 👔 to Deep work 🌊 on the User-zeljkofilipin board.Mar 2 2018, 2:44 PM
zeljkofilipin added a subtask: T188553: Vector browser test blocking merge in Minerva.Mar 2 2018, 2:52 PM
zeljkofilipin mentioned this in T188553: Vector browser test blocking merge in Minerva.Mar 2 2018, 3:09 PM
zeljkofilipin closed subtask T188553: Vector browser test blocking merge in Minerva as Resolved.
zeljkofilipin added a subscriber: hashar.Mar 2 2018, 3:16 PM
In T187859#4015225, @Jdlrobson wrote:

I don't follow... the Minerva skin should not be installing Vector in order to run its browser tests, so Vector should not exist and its browser tests should not be running.

Sorry, I was talking with @hashar how to fix T188553 and I was leaving notes for myself. I have explained in T188553#4017820 why Vector tests were run for patches in Minerva.

zeljkofilipin updated the task description. (Show Details)Mar 2 2018, 3:20 PM
ReleaseTaggerBot added a project: MW-1.31-release-notes (WMF-deploy-2018-03-06 (1.31.0-wmf.24)).Mar 6 2018, 11:01 AM
zeljkofilipin added a comment.EditedMar 14 2018, 6:18 PM

Jenkins

[[ https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/ | mediawiki-core-qunit-selenium-jessie ]] Jenkins job runs Selenium tests.

integration/config

The job is defined in [[ https://phabricator.wikimedia.org/source/integration-config/browse/master/jjb/mediawiki.yaml;df49dfb867d22137fb6dcf8fe7c693b51eec8280$8 | mediawiki.yaml ]].

It has mw-wdio-nodepool builder defined in [[ https://phabricator.wikimedia.org/source/integration-config/browse/master/jjb/macro.yaml;df49dfb867d22137fb6dcf8fe7c693b51eec8280$260 | macro.yaml ]].

Selenium job is executed by [[ https://phabricator.wikimedia.org/source/integration-config/browse/master/jjb/macro.yaml;df49dfb867d22137fb6dcf8fe7c693b51eec8280$283 | grunt webdriver:test ]].

mediawiki/core

Tests to run are defined in [[ https://phabricator.wikimedia.org/source/mediawiki/browse/master/tests/selenium/wdio.conf.js;312c32319d701f62d49b1509e68dd43d585f3273$47-52 | wdio.conf.js ]]:

 specs: [
   relPath( './tests/selenium/specs/**/*.js' ),
   relPath( './extensions/*/tests/selenium/specs/**/*.js' ),
   relPath( './extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js' ),
   relPath( './skins/*/tests/selenium/specs/**/*.js' )
],

There is some Jenkins-specific configuration in [[ https://phabricator.wikimedia.org/source/mediawiki/browse/master/tests/selenium/wdio.conf.jenkins.js | wdio.conf.jenkins.js ]]. For now it only has username, password and alike. It could potentially be used to override configuration from wdio.conf.js.

gerritbot added a comment.Mar 15 2018, 5:40 PM

Change 419805 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/skins/MinervaNeue@master] WIP Move one Selenium tests from mediawiki/core to mediawiki/skins/Vector

https://gerrit.wikimedia.org/r/419805

zeljkofilipin added a comment.Mar 15 2018, 6:07 PM

@Jdlrobson I would like to reproduce the problem locally (using mediawiki-vagrant) so I could resolve it, but I am not sure how to do that. If I provision minerva role, vector is still the default skin, and vector test passes.

I have created 419805, copy/pasting vector test to minerva repository. It reproduces the problem, but not in a way I was expecting.

The test fails with:

00:03:15.135 An element could not be located on the page using the given search parameters ("div.usermessage").

but when I take a look at the screenshot, it shows vector skin, not minerva.

should-be-able-to-view-new-message-banner.png (899×1 px, 77 KB)

I am confused. Does Minerva change something in Vector skin, causing Vector test to fail?

zeljkofilipin moved this task from Deep work 🌊 to Waiting 🏹 on the User-zeljkofilipin board.Mar 19 2018, 2:29 PM
zeljkofilipin removed zeljkofilipin as the assignee of this task.Mar 21 2018, 4:54 PM
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin awarded a token.
zeljkofilipin updated the task description. (Show Details)Mar 26 2018, 12:19 PM
zeljkofilipin changed the task status from Open to Stalled.Mar 29 2018, 9:12 AM
zeljkofilipin edited parent tasks, added: T190994: Q4 Selenium framework improvements; removed: T182421: Q3 Selenium framework improvements.
zeljkofilipin mentioned this in T191982: Blog posts about new Selenium framework features.Apr 11 2018, 2:53 PM
zeljkofilipin moved this task from Waiting 🏹 to Backlog 🪒 on the User-zeljkofilipin board.Jun 15 2018, 12:04 PM
zeljkofilipin added a project: MediaWiki-Core-Tests.Jun 28 2018, 4:52 PM
zeljkofilipin moved this task from Inbox to Selenium on the MediaWiki-Core-Tests board.
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin edited parent tasks, added: T190995: Someday/maybe Selenium framework improvements; removed: T190994: Q4 Selenium framework improvements.
Jdlrobson added a comment.Aug 2 2018, 3:47 AM
In T187859#4054069, @zeljkofilipin wrote:

@Jdlrobson I would like to reproduce the problem locally (using mediawiki-vagrant) so I could resolve it, but I am not sure how to do that. If I provision minerva role, vector is still the default skin, and vector test passes.

Are you including the LocalSettings.php needed for the browser tests?[1]

I have created 419805, copy/pasting vector test to minerva repository. It reproduces the problem, but not in a way I was expecting.

I'm not sure how that patch helps. The issue is that the test is running despite being outside the Minerva repo. The Vector test is incompatible with the Minerva ones (which has tests which expect it to be the default skin) so there is no way to fix the Vector test. Instead we have to find a way to run skin tests exclusively or disable browser test groups via localsettings.

I am confused. Does Minerva change something in Vector skin, causing Vector test to fail?

Yes it does. Minerva needs to be the default skin for CI
[1].

E.g. be tested where Vector skin is not the default. The test fails as the Vector test is running against Minerva.

[1] https://github.com/wikimedia/mediawiki-skins-MinervaNeue/blob/master/tests/browser/LocalSettings.php

zeljkofilipin moved this task from Blocked (externally) to Backlog on the Release-Engineering-Team (Kanban) board.Sep 25 2018, 10:06 AM
zeljkofilipin removed a project: Release-Engineering-Team (Kanban).Sep 25 2018, 10:09 AM
gerritbot added a comment.Aug 30 2019, 11:34 PM

Change 415614 abandoned by Jdlrobson:
WIP: Move Selenium test from mediawiki/core

https://gerrit.wikimedia.org/r/415614

gerritbot added a comment.Oct 30 2019, 3:22 PM

Change 419805 abandoned by Zfilipin:
WIP Move one Selenium tests from mediawiki/core to mediawiki/skins/Vector

Reason:
no activity in months

https://gerrit.wikimedia.org/r/419805

Restricted Application added a subscriber: Masumrezarock100. · View Herald TranscriptOct 30 2019, 3:22 PM
hashar unsubscribed.Nov 26 2019, 12:01 PM
Jdlrobson moved this task from Needs triage to Feature requests on the Vector (legacy skin) board.Jan 15 2020, 6:22 PM
Jdlrobson moved this task from Feature requests to Technical on the Vector (legacy skin) board.
zeljkofilipin raised the priority of this task from Low to Needs Triage.May 18 2021, 9:55 AM
zeljkofilipin closed this task as Resolved.Oct 7 2021, 3:26 PM
zeljkofilipin claimed this task.
zeljkofilipin edited projects, added Browser-Tests; removed User-zeljkofilipin.

Core tests now run fine without Vector.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL