Page MenuHomePhabricator

Run browser tests as part of "npm test" of wikidata/query/gui
Open, NormalPublic

Description

In T219749 we've (rather hackily) fixed the issue of failing CI jobs wikidata-query-rdf by skipping to run selenium tests as part of "npm test" of wikidata-query-gui.

We do want to have those tests run as part of standard "npm test" run.
In order to allow that, container when WDQS GUI's "npm test" are run should have wdio/chromedriver installed.

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptApr 30 2019, 4:21 PM

@hashar @zeljkofilipin this seems to beyond our (WMDE) realm. We're happy to provide more context/information, but we'd appreciate advise/help with getting this actually resolved.

zeljkofilipin triaged this task as Normal priority.May 2 2019, 2:03 PM
  • I will be at Wikimedia-Hackathon-2019 next week, if anybody wants to pair on this.
  • The convention is to have Selenium tests (including wdio.conf.js) in tests/selenium, if possible. See Selenium/Node.js#write-tests for examples.
  • I don't think CI supports running Selenium tests in Firefox (wdio.conf.js#33). All other repositories use Chrome.
  • There is no need for selenium-standalone (Gruntfile.js#273). Use Chromedriver instead. It should be available in CI.

@zeljkofilipin I will be at the Hackathon and would gladly pair on this. I arrive on Fri 17.05.

Change 511308 had a related patch set uploaded (by Noa wmde; owner: Noa wmde):
[integration/config@master] Change Dockerfile and wikidata.yml to support running browser tests for the query service gui

https://gerrit.wikimedia.org/r/511308

Change 511309 had a related patch set uploaded (by Noa wmde; owner: Noa wmde):
[wikidata/query/gui@master] Rename separate grunt task from 'browser_test' to 'test_all'

https://gerrit.wikimedia.org/r/511309

Here is an explanation of the issue @Smalyshev encountered on T219749 as I posted it on https://gerrit.wikimedia.org/r/#/c/integration/config/+/511308/

The failure comes from the CI tests for wikidata/query/rdf which are run by the Maven test suite. The job doing that is wikidata-query-rdf-maven-java8-docker which really just ship java8 and then runs maven. There is no chromedriver being spawned.

The wikidata/query/rdf test suite then run the GUI test suite using npm, but since the gui code uses wdio/chromedriver and the maven image does not have it ... The suite fails to connect to chromedriver ( ECONNREFUSED 127.0.0.1:4444 ).

I am not sure whether the GUI test should be run as part of testing wikidata/query/rdf . Probably only a subset of them should?


For wikidata/query/gui, the CI job runs npm test which in turns seems to invoke grunt test. So I guess it is all about adding wdio task to the list of tasks executed when running grunt test. Specially, revert https://gerrit.wikimedia.org/r/#/c/wikidata/query/gui/+/506666/

BUT, in wikidata/query/rdf, as soon as the GUI submodule is updated, the test will fail since they are run with maven and the container does not have chromedriver nor is it intended to run wdio test. My big question is: do you actually need to run the GUI npm test when testing wikidata/query/rdf ? That seems to just rerun tests that already ran perfectly fine in the upstream repo wikidata/query/gui.

awight added a subscriber: awight.May 23 2019, 1:06 PM

This might also be useful:
T199116: Quibble should run `npm install` and `npm run selenium-test` for each extension/skin that has Selenium tests
I have a patch for review which will run browser tests for each extension separately, which makes it possible to run custom setup scripts and install local npm dependencies. This seems like a simpler solution than spinning up the headless browser in another phase of testing. Or maybe I'm misunderstanding why you want the workaround here--would you mind explaining why it's desirable to run browser tests along with npm test?