Page MenuHomePhabricator

Consider promoting Wikibase Release Pipeline's acceptance tests to a dedicate software component
Open, Needs TriagePublic

Description

WMDE's infrastructure for creating release packages of Wikibase software suite, so called Wikibase release pipeline, has introduced a framework for acceptance testing of Wikibase and the related software. Those tests cover, among other functionality, e.g. verifying that interaction between Wikibase Repo and Wikbiase Client components works as expected, integration between Wikibase and the Query Service.

While those tests serve a good purpose in the process of preparing the release of Wikibase suite, it seems that, at least some of them, would also be useful for other purposes, e.g. to verify the essential functionality while developing Wikibase code (CI infrastructure of Wikibase, T282479), or to test that the freshly set up Wikibase instance functions correctly.

Acceptance tests are provided as selenium browser tests scenarios (e.g. https://github.com/wmde/wikibase-release-pipeline/blob/10f338e04ff796b59e638d2a5abcebaa8e0e6d7d/Docker/test/selenium/specs/repo_client/item.js) that could potentially be turned into a dedicate npm package. Tests are accompanied by Docker-based infrastructure automation that ensures the intended test set up and run tests that might be considered specific to Wikibase Release Pipeline

Acceptance criteria:

  • Decision on whether to make Wikibase acceptance tests a dedicated software component has been documented on this task and a follow up task for implementing the decision (should it be positive) has been created

Event Timeline

@WMDE-leszek To followup on what you talked about yesterday regarding fedprops I think this kind of testing could greatly benefit us and will be hard to do in our normal php tests.

Something we haven't done so far is to extend/adapt the tests to allow for more to run for the specific scenarios (Might have saved us the time and trouble to earlier identify the problem you described at the retro yesterday).

For example: currently the fedprops "suite" is only concerned with fedprops things but could quite easily be extend to include other things we also claim to be working from other "suites".