Page MenuHomePhabricator

Cypress testing framework evaluation
Open, MediumPublic

Description

Explore feasibility of using cypress.io test framework for automation testing.

Install:

npm install cypress --save-dev

CI dependencies:

apt-get install xvfb libgtk-3-dev libnotify-dev libgconf-2-4 libnss3 libxss1 libasound2

Docs: https://docs.cypress.io

Currently supports only Chrome/Chromium: https://docs.cypress.io/guides/references/trade-offs.html#Temporary-trade-offs

Details

Related Gerrit Patches:
mediawiki/core : masterWIP Cypress

Event Timeline

Restricted Application added a subscriber: Aklapper. ยท View Herald TranscriptAug 19 2019, 3:37 PM
zeljkofilipin updated the task description. (Show Details)

Change 530914 had a related patch set uploaded (by Zfilipin; owner: Zfilipin):
[mediawiki/core@master] WIP Cypress

https://gerrit.wikimedia.org/r/530914

Jpita added a comment.Aug 20 2019, 5:50 PM

nice short video showing main features of cypress
https://youtu.be/l_Q7cbmYjDY

zeljkofilipin removed zeljkofilipin as the assignee of this task.EditedSep 4 2019, 2:52 PM

Status: Initial patch is in gerrit (530914). The patch does not pass CI since it doesn't support cypress. I've talked with @hashar and we agreed that at the moment we don't want to add cypress support in our CI.

Next step: @Jpita and/or @zeljkofilipin should create small but representative test suite in cypress to see if it offers any or significant advantage over selenium/webdriverio.

Even if cypress is slightly better than webdriverio it's unlike we would move to it since we have significant investment in webdriverio, see Selenium/Node.js#write-tests. Additionally, we are still not done with transition from ruby to node, see T139740: Port Selenium tests from Ruby to Node.js.

If cypress is significantly better, then we should reconsider.

A note, from https://opensource.intuit.com/ I found cyphell to convert from wdio to cypress. https://github.com/intuit/cyphfell . Quote:

Cyphfell
Converts WDIO to Cypress
Intuit engineers were running thousands of automation tests with the WebDriverIO framework, but over time, they found this tool no longer met needs. Looking for alternatives, engineers discovered the open source testing tool Cypress. Upon determining Cypress solved a number of issues, the team decided to use it going forward. But with this decision, thousands of tests written for WebDriverIO would have to convert to Cypress. Converting tests manually would be tedious and time-consuming. Software Developer Adil Malik set out to develop an alternative.
Adil envisioned a tool to convert WebDriverIO tests to Cypress tests. With support from a mentor, Principal Engineer Raj Vasikarla, and the Cypress team, Adil developed Cyphfell. Since the testing tools themselves were open source and a conversion tool could have wide-reaching applications, Adil planned to open source his tool from the start. Adil hopes Cyphfell proves useful to anyone looking to convert automation tests and plans to expand its functionality to translate other test frameworks, including Puppeteer and Nightwatch.

Main Contributors: Adil Malik

Jpita added a comment.Sep 17 2019, 7:34 PM

Oh wow, that's amazing.
This way we can actually compare times and error rate between frameworks and easily evaluate which is better/more useful

greg added a subscriber: greg.Oct 22 2019, 3:29 PM

Change 530914 abandoned by Zfilipin:
WIP Cypress

Reason:
just a test

https://gerrit.wikimedia.org/r/530914

I've made a report on comparison of Cypress and WebdriverIO here: https://github.com/simkawaii/GrowthExperiement_Cypress_WebdriverIO

Jpita added a comment.Tue, Jan 14, 9:38 PM

I've made a report on comparison of Cypress and WebdriverIO here: https://github.com/simkawaii/GrowthExperiement_Cypress_WebdriverIO

๐Ÿ‘ amazing work!