Page MenuHomePhabricator

Experiment with browser testing in other software languages
Closed, ResolvedPublic

Description

It was brought up the Testing: Where does it hurt? discussion that we should try out end-to-end testing frameworks in other languages such as JS other languages that MW devs might be more familiar with.

LanguageRubyJavaScript
ImplementationCRubyNode.js
Package managerRubyGems, Bundlernpm
Build automationRakeGrunt
LinterRuboCopJSHint, JSCS
Testing frameworkCucumber, RSpecMocha
Browser driverSeleniumSelenium
Selenium bindingsRuby bindings (github.com, code.google.com, API)WebDriverJS (github.com, code.google.com, API)
Browser driver packagese​lenium-webdriver (rubygems.org)selenium-webdriver (npmjs.com)
Nicer API on top of Seleniumwatir-webdriver​none
Page objectpage-objectin Malu
Shared Selenium codemediawiki_seleniumMalu
Mediawiki APImediawiki_apinodemw

See also Manual:Unit testing.

Details

Related Gerrit Patches:
mediawiki/core : masterWIP End-to-end tests with Malu
mediawiki/core : masterWIP Nightwatch.js
mediawiki/core : masterWIP WebDriverJs + QUnit
mediawiki/core : masterWIP WebDriverJs + Jasmine

Event Timeline

dduvall created this task.Aug 12 2015, 9:21 PM
dduvall raised the priority of this task from to Needs Triage.
dduvall updated the task description. (Show Details)
dduvall added a project: Testing-Initiative.
dduvall added subscribers: dduvall, ggellerman, Jdlrobson.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 12 2015, 9:21 PM
Jdforrester-WMF renamed this task from Experiment with browser testing in other languages to Experiment with browser testing in other software languages.Aug 12 2015, 9:23 PM
Jdforrester-WMF added a subscriber: Jdforrester-WMF.

Tweaked the title as we already do awesome testing in other human languages.

zeljkofilipin triaged this task as Low priority.Aug 25 2015, 3:52 PM
zeljkofilipin moved this task from Inbox to Ruby on the Browser-Tests-Infrastructure board.
zeljkofilipin added a subscriber: zeljkofilipin.

This also came up in the RelEng <-> VE SLU discussion:

zeljkofilipin raised the priority of this task from Low to Medium.
zeljkofilipin moved this task from Backlog to Doing on the Testing-Initiative board.
zeljkofilipin moved this task from Ruby to In Progress on the Browser-Tests-Infrastructure board.
This comment was removed by zeljkofilipin.

Change 255419 had a related patch set uploaded (by Dduvall):
[WIP] Experimenting with JS end-to-end tests

https://gerrit.wikimedia.org/r/255419

This comment was removed by zeljkofilipin.
This comment was removed by zeljkofilipin.
This comment was removed by zeljkofilipin.
This comment was removed by zeljkofilipin.

Change 255984 had a related patch set uploaded (by Zfilipin):
WIP WebDriverJs

https://gerrit.wikimedia.org/r/255984

zeljkofilipin updated the task description. (Show Details)

Change 256404 had a related patch set uploaded (by Zfilipin):
WIP WIP WebDriverJs Mocha

https://gerrit.wikimedia.org/r/256404

hashar updated the task description. (Show Details)Dec 3 2015, 9:45 AM
hashar added a subscriber: hashar.

https://github.com/webdriverio/webdriverio/issues/138#issuecomment-32051980 does a loose comparaisons of selenium-webdriverjs vs WD.js vs webdriver.io. Seems the later two comes on top of selenium-webdriverjs and thus WD.js/webdriver.io would be similar to Watir in the ruby stack.

greg awarded a token.Dec 3 2015, 7:01 PM
zeljkofilipin updated the task description. (Show Details)Dec 9 2015, 2:43 PM
This comment was removed by zeljkofilipin.

Change 255419 abandoned by Zfilipin:
WIP WebDriverJs Jasmine

Reason:
Abandoning in favor of https://gerrit.wikimedia.org/r/#/c/256404/

https://gerrit.wikimedia.org/r/255419

Change 255984 abandoned by Zfilipin:
WIP WebDriverJs QUnit

Reason:
Abandoning in favor of https://gerrit.wikimedia.org/r/#/c/256404/

https://gerrit.wikimedia.org/r/255984

Change 268412 had a related patch set uploaded (by Zfilipin):
WIP Nightwatch.js

https://gerrit.wikimedia.org/r/268412

Change 268412 abandoned by Zfilipin:
WIP Nightwatch.js

https://gerrit.wikimedia.org/r/268412

zeljkofilipin removed zeljkofilipin as the assignee of this task.Feb 16 2016, 10:01 AM
zeljkofilipin moved this task from In Progress to Next on the Browser-Tests-Infrastructure board.
zeljkofilipin moved this task from Doing to Radar on the Testing-Initiative board.
zeljkofilipin moved this task from Next to In Progress on the Browser-Tests-Infrastructure board.
zeljkofilipin moved this task from Radar to Doing on the Testing-Initiative board.Mar 2 2016, 1:08 PM
greg added a subscriber: greg.Mar 28 2016, 11:51 PM

Is this done? Did we decide on mocha (that's what's in malu, at least)?

@zeljkofilipin, should we abandon the WIP core patch for now and close this out? We can still reference it for implementation details but i think the actual end-to-end tests that exercise the new framework's basic features should live in the Malu project.

zeljkofilipin closed this task as Resolved.Apr 6 2016, 3:35 PM
zeljkofilipin updated the task description. (Show Details)

@greg: I think this can be resolved.
@dduvall: I will do the needful.

@dduvall: I am not sure I have understood you: are you suggesting 256404 should be abandoned?

@dduvall: I am not sure I have understood you: are you suggesting 256404 should be abandoned?

Yep! The way I see it, this experiment was successful but merging that patch into core at this point would be premature. I think we should abandon the patch, finish the Malu prototype and prerelease (w/ CI support), then see about fully porting over the end-to-end core tests from MW-Selenium—reviving the patch if that makes sense, or simply starting fresh and using it for reference.

Just in general, I'd like to see us get better about scoping experimental work as such, and being ok with abandoning it as a resolution. That doesn't mean the experiment wasn't fruitful or even informative, just that the code is likely half-baked and merging it would incur more debt than it would pay back.

greg added a comment.Apr 6 2016, 5:05 PM

Well reasoned/said, Dan.

@dduvall: sounds good to me, abandoning 256404.

Change 256404 abandoned by Zfilipin:
WIP End-to-end tests with Malu

Reason:
Abandoning, as agreed in T108874.

https://gerrit.wikimedia.org/r/256404

Change 256404 restored by Zfilipin:
WIP End-to-end tests with Malu

Reason:
need to update the patch

https://gerrit.wikimedia.org/r/256404

Change 256404 abandoned by Zfilipin:
WIP End-to-end tests with Malu

Reason:
just a test

https://gerrit.wikimedia.org/r/256404

Change 256404 restored by Zfilipin:
WIP End-to-end tests with Malu

Reason:
testing

https://gerrit.wikimedia.org/r/256404

Change 256404 abandoned by Zfilipin:
WIP End-to-end tests with Malu

Reason:
just a test

https://gerrit.wikimedia.org/r/256404