Page MenuHomePhabricator

Selenium tests for VisualEditor in Node.js
Closed, DeclinedPublic8 Story Points

Description

Create Selenium test for loading VE using JS

Event Timeline

Ryasmeen created this task.Nov 10 2016, 3:34 PM
Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptNov 10 2016, 3:34 PM

Change 320829 had a related patch set uploaded (by Ryasmeen):
Selenium JS load test for VE

https://gerrit.wikimedia.org/r/320829

Jdforrester-WMF triaged this task as Normal priority.Nov 14 2016, 10:11 PM
Jdforrester-WMF moved this task from To Triage to TR1: Releases on the VisualEditor board.
Jdforrester-WMF set the point value for this task to 8.
zeljkofilipin renamed this task from Create Selenium test for loading VE using JS to Selenium tests for VisualEditor in Node.js.Nov 28 2016, 4:39 PM

Change 334136 had a related patch set uploaded (by Zfilipin):
Run mediawiki-core-selenium-jessie in experimental pipeline for VisualEditor

https://gerrit.wikimedia.org/r/334136

Change 334136 merged by jenkins-bot:
Run mediawiki-core-selenium-jessie in experimental pipeline for VisualEditor

https://gerrit.wikimedia.org/r/334136

I have proposed a skill share session for Vienna hackathon T159945: Selenium/WebdriverIO tests in JavaScript/Node.js if you would like to pair in person.

zeljkofilipin lowered the priority of this task from Normal to Low.

Change 345146 had a related patch set uploaded (by Zfilipin):
[mediawiki/extensions/VisualEditor@master] WIP WebdriverIO tests

https://gerrit.wikimedia.org/r/345146

Change 320829 abandoned by Zfilipin:
WIP Selenium test for loading VisualEditor

Reason:
We have decided to use webdriverio, this patch uses webdriverjs. Moved the tests to https://gerrit.wikimedia.org/r/#/c/345146/

https://gerrit.wikimedia.org/r/320829

@Ryasmeen I am assigning the task back to you.

I have abandoned the webdriverjs patch (320829) since we have decided to use webdriverio instead.

I have done my best to implement the a from the webdriverjs patch in the webdriverio patch (345146) but I do not know how to assert that VisualEditor is loaded. :(

We will probably need help from @Esanders.

mediawiki-core-qunit-selenium-jessie fails with Timeout of 20000ms exceeded

https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/3472/

The screenshot says VisualEditor is not configured correctly

https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/3472/artifact/log/should-load-by-url.png

@hashar do you know what is the problem?

zeljkofilipin changed the task status from Open to Stalled.Jul 27 2017, 2:08 PM
Krinkle added a subscriber: Krinkle.EditedJul 27 2017, 8:19 PM

mediawiki-core-qunit-selenium-jessie fails with Timeout of 20000ms exceeded

(Re-triggered the job since the screenshot artefact expired).

I can still reproduce the same error at https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/4569/.
Screenshot:

Error loading data from server: no_vrs: The VirtualRESTService for the document server is not defined;

This basically just means that the Jenkins job has set up MediaWiki core and installed the VisualEditor extension, but it has no RESTBase or Parsoid (Node.js service) running. Without Parsoid to convert between wikitext to HTML, VisualEditor can't load or save anything.

I haven't tested the script locally, but test is presumably written fine and would work locally in MediaWiki-Vagrant. The Ruby test presumably worked fine because it ran against Beta Cluster, which already has all these services set-up.

To make it explicit, we did not invest time in updating CI so the example test runs fine there because the repository owner(s) do not have time and/or interest in writing Selenium tests (as far as we can see).

Krinkle added a comment.EditedAug 28 2017, 5:47 PM

@zeljkofilipin As far as I know, we've never had a Jenkins job that installs both a local MediaWiki wiki and the Parsoid service.

The reason the tests worked under Ruby and fail under Node.js now is because part of the Node.js-switch was also to move from testing Beta Cluster to testing Jenkins build localhost as triggered from Gerrit.

As such, for the specific case of VisualEditor, the tests fail on Jenkins because it also requires the Parsoid service to be installed.

The tests do pass on MediaWiki-Vagrant, assuming you have the visualeditor role enabled, which also installs Parsoid.

Change 345146 abandoned by Zfilipin:
WIP WebdriverIO tests

Reason:
No interest from team working on VisualEditor.

https://gerrit.wikimedia.org/r/345146

zeljkofilipin closed this task as Declined.EditedOct 27 2017, 3:53 PM

Unlikely to ever happen. Ruby Selenium tests were deleted in 381448.