Browser testing with Cypress

• Session description:

The WMDE-TechWish team started playing with Cypress browser tests in one of our current projects. In this session we want to give a quick overview on how that was going, what we see as pros and cons and share our knowledge from that ( somewhat ongoing ) experiment.

• Username for contact: @WMDE-Fisch

• Session duration: 25min

• Session type: Short presentation, Q/A, discussion

• Language of session: English

• Prerequisites: Basic understanding of browser tests might be useful. Working local setup if you want to play a bit with the tooling.

Scheduled in room Macao on Friday 2pm

Interested? Add your username below:

• @roti_WMDE
• @AnnWF
• @dom_walden

Notes from session:

Browser testing with Cypress

Date and time: 2024-05-03 14:00

Relevant links

• Phabricator task: https://phabricator.wikimedia.org/T362159
• Session slides:
• (any more?)

Presenter

@[[phab:p/WMDE-Fisch/|WMDE-Fisch]] & @[[phab:p/MareikeHeuerWMDE/|MareikeHeuerWMDE]]

Participants

Notes

• Test pyramid
  • Test automation strategy
  • Hierarchical distribution of tests
    • Unit
    • Integration
    • E2E tests (validate user journey) - slower
  • Getting thorough coverage while efficiently using execution time
• Browser tests
  • Validation from a user persepective
  • Across different browsers and devices
  • Important user workflows
  • Purpose: validation of user workflows, quality assurance
• Current state
  • Selenium
    • We use Selenium JS webdriver
    • Mocha testing framework
    • assert library for assertions
    • We have our own selenium library to help selenium testing with mediawiki
• What is Cypress?
  • All in one JS E2E testing framework
  • Runs in same browser as application
  • Functionalities like real-time debugging, auto waits, DOM manipulation
• Cypress vs. Selenium
  • Cypress runs in browser; Selenium client-server
  • Cypress async by default (auto wait); Selenium waits rely on more explicit waits
  • Cypress only JS; Selenium supports multiple languages
  • Cypress faster; Selenium speed affected by network latency
  • Cypress has a time-travel feature; Selenium relies on traditional logging
• Cypress vs. Selenium code
  • They look very similar
  • Cypress does not need explicit "await"s
• Comparison
  • Selenium
    • needs more dependencies
    • better browser interaction with webdriver
    • need to manually wait for elements
    • cli debugging
    • easy parallisation
    • lots of example code
  • cypress
    • See slides
• Learnings
  • Easy to setup
  • Worked flawlessly with CI
  • Straightforward debugging
  • Easy for less experienced developers
  • Nice way to trigger re-runs
  • Easy to setup screenshots and video
• Struggles
  • Missing helpers
  • No support for mouse interaction
    • e.g. cannot test popups/tooltips
  • No support for copy and paste

Questions

• Can this run on production?
  • Yes, in theory, but not at the moment
• Which projects use this?
  • ...
  • In phabricator, search for "cypress" for a task that lists projects using cypress
• How do you start Cypress
  • Cite extension has "npm run cypress:open"
• Where does it run in CI?
  • Jenkins - existing MW CI