Page MenuHomePhabricator

Consider moving more of our test setup into WDIO and out of shell scripts
Closed, ResolvedPublic3 Estimated Story PointsSpike

Description

Current Situation:
Our current test setup is not very well isolated from our build environment and is almost entirely setup through shell scripts while the actual tests and related setup hooks are in the WDIO/Javascript environment. Making the test setup, including starting of any needed Docker services, happen within Javascript in the same environment the tests are ran would empower us to write better tests, as well as to better control the test environment for each suite or spec file run. Ultimately this setup would give us the pieces we need to potentially run our test suite against arbitrary Wikibase installations regardless of our build setup as we do currently with the Example and Upgrade test suites.

Goal:

Acceptance Criteria:

  • Specifically analyse what it would take to make the example and upgrade test suites not need their own shell script setup (or minimise it).

The reasons they require their own shell scripts vs also just using test_suite.sh is the information to guide this spike.

  • Also dependent on the features of WDIO (8) to see how we can include the shell scripts functionality in js / ts code.

Event Timeline

Restricted Application changed the subtype of this task from "Task" to "Spike". · View Herald TranscriptSep 11 2023, 10:28 AM
lojo_wmde changed the task status from Open to In Progress.Nov 21 2023, 3:27 PM
lojo_wmde claimed this task.

Addressed in https://github.com/wmde/wikibase-release-pipeline/pull/525. I have moved ahead with a working implementation of our test running and services setup outside of shell scripts in the native JS/TS + WDIO environment. The results are here: https://github.com/wmde/wikibase-release-pipeline/pull/525. This setup gains us a number of new capacities in our test runs both for existing suites, as well as opening options for how we write new test suites going forward.

roti_WMDE set the point value for this task to 2.Dec 5 2023, 12:49 PM
roti_WMDE changed the point value for this task from 2 to 3.Dec 19 2023, 12:43 PM