Page MenuHomePhabricator

Create selenium-daily-beta-MediaWiki daily Jenkins job
Closed, ResolvedPublic

Description

Running selenium-MediaWiki-jessie daily Jenkins job would help detect and prevent problems like Incident documentation/20180111-LegacyEncoding.

It would require refactoring selenium tests. At the moment, mediawiki/core tests target only mediawiki-vagrant and jenkins where no logging is required to create/update users/pages. Running at beta cluster would require logging in for every action.

Done

๐Ÿ’€ 415027 (integration/config): WIP Create selenium-core-jessie daily Jenkins job
๐Ÿ’€ 420040 (integration/config): WIP Create selenium-MediaWiki-jessie daily Jenkins job
๐Ÿ’€ 420057 (mediawiki/core): Selenium: configuration file for beta cluster
๐Ÿ’€ 420674 (mediawiki/core): Revert "Selenium: configuration file for beta cluster"
๐Ÿ’€ 446371 (mediawiki/core) Selenium: add selenium-daily NPM script
๐Ÿ  456131 Selenium: selenium-daily NPM script
๐Ÿ  457854: Selenium: Daily Jenkins job targeting beta cluster should run only tests that pass
๐Ÿ  457881: Create selenium-daily-beta-MediaWiki Jenkins job
๐Ÿ  selenium-daily-beta-MediaWiki

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptJan 16 2018, 5:20 PM
greg triaged this task as Medium priority.Feb 21 2018, 4:59 PM
zeljkofilipin raised the priority of this task from Medium to High.Feb 23 2018, 5:32 PM

@daniel says there are at least a couple extensions that would log you in if you are coming from an IP:

https://www.mediawiki.org/wiki/Extension:NetworkAuth
https://www.mediawiki.org/wiki/Extension:IPAuth

Configuration:

$wgIPAuthUsers = array( "127.0.0.1" => "Admin" );

@hashar: is that something we could enable at beta cluster?

Enabling the extension would mean we do not have to refactor core selenium tests, making them more complicated.

I have just talked with @hashar about this. Another option would be to log in using the API and then inject the cookie in the browser.

Change 415027 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[integration/config@master] WIP Create selenium-core-jessie daily Jenkins job

https://gerrit.wikimedia.org/r/415027

zeljkofilipin added a comment.EditedMar 1 2018, 10:27 AM

Webdriverio has a feature to run selected tests. Unfortunately, it does not work well. ๐Ÿ˜ข

zeljkofilipin renamed this task from Create selenium-core-jessie daily Jenkins job to Create selenium-MediaWiki-jessie daily Jenkins job.Mar 2 2018, 3:41 PM
zeljkofilipin updated the task description. (Show Details)

Change 420040 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[integration/jenkins@master] WIP Create selenium-MediaWiki-jessie daily Jenkins job

https://gerrit.wikimedia.org/r/420040

Change 420040 abandoned by Zfilipin:
WIP Create selenium-MediaWiki-jessie daily Jenkins job

https://gerrit.wikimedia.org/r/420040

Change 420057 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/core@master] WIP Create selenium-MediaWiki-jessie daily Jenkins job

https://gerrit.wikimedia.org/r/420057

I have just talked with @hashar about this. Another option would be to log in using the API and then inject the cookie in the browser.

browser.setCookie(cookie);

Change 420057 merged by jenkins-bot:
[mediawiki/core@master] Selenium: configuration file for beta cluster

https://gerrit.wikimedia.org/r/420057

Krinkle added a subscriber: Krinkle.EditedMar 16 2018, 9:19 PM

Change 420057 merged by jenkins-bot:
[mediawiki/core@master] Selenium: configuration file for beta cluster
https://gerrit.wikimedia.org/r/420057

@zeljkofilipin Files like that generally should not be in MediaWiki core. Could you explain how this file will be used? It seems like the four configuration values are taken directly from environment variables, which should be the default already. We drive the QUnit tests that way as well. The only variance between local, master and beta testing is env variables. The Jenkins job glues it together.

greg lowered the priority of this task from High to Medium.Mar 19 2018, 10:39 PM
zeljkofilipin removed zeljkofilipin as the assignee of this task.Mar 20 2018, 11:09 AM

@zeljkofilipin Files like that generally should not be in MediaWiki core.

I'll revert it.

Change 415027 abandoned by Zfilipin:
WIP Create selenium-core-jessie daily Jenkins job

Reason:
will be implemented in T188742

https://gerrit.wikimedia.org/r/415027

Change 446371 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/core@master] Selenium: add selenium-daily NPM script

https://gerrit.wikimedia.org/r/446371

zeljkofilipin updated the task description. (Show Details)
zeljkofilipin removed zeljkofilipin as the assignee of this task.Jul 31 2018, 2:54 PM

Change 446371 abandoned by Zfilipin:
Selenium: add selenium-daily NPM script

Reason:
not needed

https://gerrit.wikimedia.org/r/446371

zeljkofilipin updated the task description. (Show Details)
zeljkofilipin updated the task description. (Show Details)Aug 7 2018, 4:26 PM

I would require that job to NOT use Nodepool but be run via a Docker container. I am not sure how to achieve that yet though.

zeljkofilipin renamed this task from Create selenium-MediaWiki-jessie daily Jenkins job to Create MediaWiki-npm-browser-run-selenium-test-node-6-docker daily Jenkins job.Aug 22 2018, 11:32 AM

@hashar sorry, I have forgot to rename the task. I've done it now. I have created the job using Docker containers, MediaWiki-npm-browser-run-selenium-test-node-6-docker. It's failing because tests should be refactored, but otherwise runs just fine.

Ahhh great. So maybe we will also be able to migrate the daily tests for extensions so? Namely:

selenium-CirrusSearch-jessie
selenium-Echo-jessie
selenium-Math-jessie
selenium-Popups-jessie
selenium-RelatedArticles-jessie

zeljkofilipin updated the task description. (Show Details)Sep 3 2018, 1:08 PM
zeljkofilipin updated the task description. (Show Details)

Change 457854 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/core@master] Selenium: Daily Jenkins job targeting beta cluster should run only tests that pass

https://gerrit.wikimedia.org/r/457854

zeljkofilipin renamed this task from Create MediaWiki-npm-browser-run-selenium-test-node-6-docker daily Jenkins job to Create selenium-daily-beta-MediaWiki daily Jenkins job.Sep 4 2018, 9:29 AM
zeljkofilipin updated the task description. (Show Details)
zeljkofilipin added a comment.EditedSep 4 2018, 10:10 AM

Videos of failed tests:

Change 457881 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[integration/config@master] Create selenium-daily-beta-MediaWiki Jenkins job

https://gerrit.wikimedia.org/r/457881

Change 457854 merged by jenkins-bot:
[mediawiki/core@master] Selenium: Daily Jenkins job targeting beta cluster should run only tests that pass

https://gerrit.wikimedia.org/r/457854

Change 457881 merged by jenkins-bot:
[integration/config@master] Create selenium-daily-beta-MediaWiki Jenkins job

https://gerrit.wikimedia.org/r/457881