Page MenuHomePhabricator

Move one Selenium tests from mediawiki/core to mediawiki/skins/Vector
Open, Stalled, LowPublic

Description

Status

Blocked on understanding the problem. See T187859#4054069 for details.

Description

Move Selenium test from 404080 from mediawiki/core to mediawiki/skins/Vector. The test is causing problems for other extensions and skins. See T185928 for more information.

The original commit (413157) caused problems for mediawiki/skins/MinervaNeue. See T188553 for more information. @Jdlrobson has reverted it (415393) until CI is fixed. The original commit is restored (but not merged) in 415614. See topic:T187859 for all commits.

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptFeb 21 2018, 9:58 AM
zeljkofilipin triaged this task as Low priority.Feb 21 2018, 10:05 AM
zeljkofilipin renamed this task from Selenium tests for Vector skin to Move one Selenium tests from mediawiki/core to mediawiki/skins/Vector.Feb 21 2018, 10:12 AM
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)

Change 413157 had a related patch set uploaded (by Zfilipin; owner: Addshore):
[mediawiki/skins/Vector@master] WIP Move one Selenium tests from mediawiki/core

https://gerrit.wikimedia.org/r/413157

Change 413157 merged by jenkins-bot:
[mediawiki/skins/Vector@master] Move Selenium test from mediawiki/core

https://gerrit.wikimedia.org/r/413157

This could help:

useskin=Vector
useskin=Minerva

Change 415614 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/skins/Vector@master] WIP: Move Selenium test from mediawiki/core

https://gerrit.wikimedia.org/r/415614

useskin=Vector
useskin=Minerva

I don't follow... the Minerva skin should not be installing Vector in order to run its browser tests, so Vector should not exist and its browser tests should not be running.

I don't follow... the Minerva skin should not be installing Vector in order to run its browser tests, so Vector should not exist and its browser tests should not be running.

Sorry, I was talking with @hashar how to fix T188553 and I was leaving notes for myself. I have explained in T188553#4017820 why Vector tests were run for patches in Minerva.

zeljkofilipin updated the task description. (Show Details)Mar 2 2018, 3:20 PM
zeljkofilipin added a comment.EditedMar 14 2018, 6:18 PM

Jenkins

mediawiki-core-qunit-selenium-jessie Jenkins job runs Selenium tests.

integration/config

The job is defined in mediawiki.yaml.

It has mw-wdio-nodepool builder defined in macro.yaml.

Selenium job is executed by grunt webdriver:test.

mediawiki/core

Tests to run are defined in wdio.conf.js:

 specs: [
   relPath( './tests/selenium/specs/**/*.js' ),
   relPath( './extensions/*/tests/selenium/specs/**/*.js' ),
   relPath( './extensions/VisualEditor/modules/ve-mw/tests/selenium/specs/**/*.js' ),
   relPath( './skins/*/tests/selenium/specs/**/*.js' )
],

There is some Jenkins-specific configuration in wdio.conf.jenkins.js. For now it only has username, password and alike. It could potentially be used to override configuration from wdio.conf.js.

Change 419805 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/skins/MinervaNeue@master] WIP Move one Selenium tests from mediawiki/core to mediawiki/skins/Vector

https://gerrit.wikimedia.org/r/419805

@Jdlrobson I would like to reproduce the problem locally (using mediawiki-vagrant) so I could resolve it, but I am not sure how to do that. If I provision minerva role, vector is still the default skin, and vector test passes.

I have created 419805, copy/pasting vector test to minerva repository. It reproduces the problem, but not in a way I was expecting.

The test fails with:

00:03:15.135 An element could not be located on the page using the given search parameters ("div.usermessage").

but when I take a look at the screenshot, it shows vector skin, not minerva.

I am confused. Does Minerva change something in Vector skin, causing Vector test to fail?

zeljkofilipin removed zeljkofilipin as the assignee of this task.Mar 21 2018, 4:54 PM
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin awarded a token.

@Jdlrobson I would like to reproduce the problem locally (using mediawiki-vagrant) so I could resolve it, but I am not sure how to do that. If I provision minerva role, vector is still the default skin, and vector test passes.

Are you including the LocalSettings.php needed for the browser tests?[1]

I have created 419805, copy/pasting vector test to minerva repository. It reproduces the problem, but not in a way I was expecting.

I'm not sure how that patch helps. The issue is that the test is running despite being outside the Minerva repo. The Vector test is incompatible with the Minerva ones (which has tests which expect it to be the default skin) so there is no way to fix the Vector test. Instead we have to find a way to run skin tests exclusively or disable browser test groups via localsettings.

I am confused. Does Minerva change something in Vector skin, causing Vector test to fail?

Yes it does. Minerva needs to be the default skin for CI
[1].

E.g. be tested where Vector skin is not the default. The test fails as the Vector test is running against Minerva.

[1] https://github.com/wikimedia/mediawiki-skins-MinervaNeue/blob/master/tests/browser/LocalSettings.php