Page MenuHomePhabricator

wikibase/termbox not running Selenium tests in CI
Closed, DeclinedPublic

Description

CI not running Selenium tests, because selenium-test npm script not present in package.json

Event Timeline

Hi @Vidhi-Mody, thanks for looking into our repository.

Selenium tests are indeed not run as part of wikibase/termbox's CI build, but this has been our deliberate decision.
Given the termbox is meant to only be used integrated into Wikibase extension, browser tests are also intended to be run when the termbox is integrated.
This has been documented in the README (Browser tests section), but it might not have been the most clear instructions. We've just updated this part of the README to make it (hopefully) more clear and accurate (https://gerrit.wikimedia.org/r/c/wikibase/termbox/+/626135).

And some explanation of how we have arrived to this decision: Adding tests which would assert our system end to end (that is termbox in integration with wikibase) was an easy decision and we did just that.
While the idea of additionally adding browsertests of termbox with a mocked wikibase/mediawiki dependency certainly crossed our minds, in the end we decided against it due to the combination of the following factors

  • there are robust edge-to-edge inside of termbox (with a mocked wikibase/mediawiki dependency) which test rendered frontend mark-up for scenarios where we are testing with a large(r) numbers of test sets
  • we knew we had browser tests encompassing the whole system (testing behavioral aspects) and did not want to not create fully redundant tests
  • termbox's limited amount of user journeys (basically one read/write view) however meant that the tests run in isolation would be (almost) identical to the ones run against the whole system

Hi @WMDE-leszek! The problem with a repository not running it's tests is - when you make a change, you don't know if it broke anything. (The purpose of continuous integration is to find problems early.) We had this problem with the task @Vidhi-Mody has been working on, T260942: Upgrade WebdriverIO in the wikibase/termbox repository.

Would it make sense to run Wikibase and termbox (Selenium) tests when commits are pushed to termbox?

I've also left a couple of comments on the related commit (626135).