We'll start the porting over of tests when we run across them, simply because we don't generally go back to older tests until we need them again. We're unsure how much work it'll be to do, overall, until we actually start working on them, but the good news is that we're not blocking any other teams.

Cindy will run all tests on every commit, we might want to start there to figure out which tests we'll need to port over first. The tests don't need to be all converted at once, there can be a mixture of Ruby and Node tests within Cindy's processes (depending on state and other dependancies).

We'll have a meeting to decide what needs to be done and who will tackle it soon.

Rough notes from our meeting on 6 Sep 2017 in regards to Ruby to Node.js Conversion of Selenium tests on Cindy:

Cindy to Jenkins

• something here...about tests on cindy and jenkins (missed it)
• tests are tagged with variety of pages associated with it (to create those pages)
• mediawiki core is using node.js
• a ruby library (very old) called mediawiki selenium (contains a lot of helper functions that don't exist in node)
• https://github.com/wikimedia/mediawiki-selenium/tree/master/lib/mediawiki_selenium
• a ruby gem lives to help with this
• would need to be re-written with it's helper
• maybe in JS, but they're not already pre-made
• need to figure out if it exists for node.js
• node.mw: https://www.mediawiki.org/wiki/Selenium/Node.js/MediaWiki_API
  • a ruby helper that was already ported over to node

do our tests run against the beta cluster?

• yes (2? tests)
  • maybe run automatically with jenkins?
• hardest to write the first full test, but after that, should be much easier
• Jan has already started looking into writing the first few tests
• https://www.mediawiki.org/wiki/Selenium/Node.js/Write

deadline is end of oct 2017 - soft deadline, since we run all our own tests

• http://www.softwaretestinghelp.com/node-js-testing-framework/
• code run for the @commons tag: https://github.com/wikimedia/mediawiki-extensions-CirrusSearch/blob/master/tests/browser/features/support/hooks.rb#L71-L87
• example test using @commons tag: https://github.com/wikimedia/mediawiki-extensions-CirrusSearch/blob/master/tests/browser/features/frozen_index_api.feature#L29-L46

Cucumber would be a layer on top of a basic coded test

• We'll just need to rewrite all ruby hooks and step definitions, but keep feature files the same
• should we get RelEng to support Cucumber? If we decide to use it?
• packages that integrates node and Cucumber:
  • https://www.npmjs.com/package/wdio-cucumber-framework/tutorial
  • https://www.npmjs.com/package/wdio-cucumber-framework/
• in node framework, there might not be specific cucumber help??

writing the tests

• probably don't need a lot of documentation with writing the tests in cucumber
• need to get all the pre-conditions to work first
• keeps feature files
• with existing tags and hooks and world objects

Cucumber-JS implementation: https://github.com/cucumber/cucumber-js
Since it's an official port, I'd expect most of the infrastructure to work, but we'd have to port Wikimedia Ruby parts of course.

Good information all written up by @Jdrewniak here: https://gerrit.wikimedia.org/r/#/c/378688/

[WIP] Ruby to Node conversion of integration tests.

Begininng the conversion of integration tests from Ruby to Node.
Maintaining Cucumber (here Cucumber.js) as the testing framework.
https://github.com/cucumber/cucumber-js

Using MWBot as the API helper
https://github.com/Fannon/mwbot

Requires chromedriver to be installed:
https://sites.google.com/a/chromium.org/chromedriver/

Currently set to run in Chrome (not yet headless).

I will create a simple test following instructions at Selenium/Node.js/Write. I will also set up CI to run the tests.

Change 381785 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/extensions/CirrusSearch@master] WIP The first Selenium test in Node.js

https://gerrit.wikimedia.org/r/381785

I should have tagged this on a few patches, we actually have a small, but reasonable, amount of work done on this: https://gerrit.wikimedia.org/r/#/c/381513/

Looks like this is already happening, for example: https://gerrit.wikimedia.org/r/#/c/381513/

Change 381791 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[integration/config@master] Run mediawiki-core-qunit-selenium-jessie for CirrusSearch

https://gerrit.wikimedia.org/r/381791

Change 381791 merged by jenkins-bot:
[integration/config@master] Run mediawiki-core-qunit-selenium-jessie for CirrusSearch

https://gerrit.wikimedia.org/r/381791

I have set up mediawiki-core-qunit-selenium-jessie job to run webdriverio patches in experimental pipeline.

Example:

https://gerrit.wikimedia.org/r/#/c/381785/
https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/7662/console

@EBernhardson told me on IRC that you are already working on this, but instead of mocha using cucumber. I am not sure how much mediawiki-core-qunit-selenium-jessie would be useful for that currently, but we should be able to make it work. Let me know.

Change 381785 abandoned by Zfilipin:
WIP The first Selenium test in Node.js

Reason:
just a sample

https://gerrit.wikimedia.org/r/381785

@EBernhardson is there a reason CirrusSearch uses mwbot instead of nodemw (like core and a few extensions)?

@Jdrewniak setup the framework for running tests in nodejs, not sure why in particular mwbot was chosen but jan can probably comment. After using it a bit i can say that it's batch() method in particular is quite convenient as our setup creates lots of different pages with varied content to be searched for.

@zeljkofilipin There are a few reasons I chose mwbot instead of nodemw:

• Promises. mwbot does not support promises out of the box, and Cucumber.js, the testing framework used in CirrusSearch, is heavily promise-based, so mwbot worked with it much better than nodemw
• Batch edit/update convenience functions. Like Erik mentioned, there's a lot of article setup with CirrusSearch, (generating many test articles) so having batch edit functionality was a big plus
• Generally nicer logging with mwbot (goes well with the batching)

I spent a little time today with nodemw and i concur with @Jdrewniak. For promises we can probably get away with something like bluebird.promisifyAll, but the batching functionality is key. The combination of succinct call structure, auto-magic concurrency, and operation specific error handling all built into mwbot's batch functionality is incredibly useful for our hooks that set everything up. Error handling for nodemw specifically is pretty difficult, as it doesn't actually return enough information. For example if i delete a page that doesn't exist i get back an error object containing the string Error returned by API: The page you specified doesn't exist.. This is a localized string that varies when we test against, for example, our language specific wikis. I really need the original response which has {code: 'missingtitle', info: 'The page you specified doesn\'t exist', ...}

I think it would be good to pick one tool and standardize on it. I do not have any experience with mwbot. I will play try it out (T181284).

Having a standardized tool is a very good thing! :) @zeljkofilipin let us know if using mwbot works for you.

Out of curiosity, this task is marked as resolved, but I see that Ruby Selenium tests are not deleted?

https://phabricator.wikimedia.org/diffusion/ECIR/browse/master/tests/browser/

Change 399178 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Remove ruby browser tests

https://gerrit.wikimedia.org/r/399178

Change 399184 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[integration/config@master] Do not run Ruby Jenkins jobs for CirrusSearch

https://gerrit.wikimedia.org/r/399184

Change 399184 merged by jenkins-bot:
[integration/config@master] Do not run Ruby Jenkins jobs for CirrusSearch

https://gerrit.wikimedia.org/r/399184

Change 399178 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Remove Ruby browser tests

https://gerrit.wikimedia.org/r/399178

Change 381785 restored by Zfilipin:
WIP The first Selenium test in Node.js

https://gerrit.wikimedia.org/r/381785