Page MenuHomePhabricator

Add Wikibase "acceptance tests" to Wikibase CI
Open, MediumPublic

Description

Wikibase Release Pipeline has introduced acceptance tests that, among others, verify correctness of essential Wikibase functionality (see T282476). Some tests cover functionality that is not tested in the existing Wikibase CI infrastructure, e.g. integration between Wikibase "Repo" and "Client" components. Lack of those tests could lead (and has in the past) to introducing bugs to the functionality without noticing it early enough.

Goal: Using new test suite would reduce the number of bugs introduced and give higher confidence on the correctness of the code

We see 2 possible paths that may make sense:

  • This task could benefit from releasing Wikibase acceptance tests as a dedicate software package (T282476), this could then be used by Wikibase.git
  • The acceptance tests could be triggered where they currently exist (release pipeline) from Gerrit / Wikibase.git

It will likely make sense to split this task up when tackling it, one for investigation & 1 for implementation.

Acceptance criteria🏕️🌟:

  • We know which path makes the most sense to us.
  • Wikibase acceptance tests, have been integrated (as voting tests) into Wikibase CI infrastructure.

Notes:

Event Timeline

It's interesting how github seems to have chosen not to make this super easy to hook up.

  1. The mechanism for triggering a workflow remotely through the api gives at the most a 204 response status code when the workflow is triggered.
  2. We are able to provide input to the workflow but none of this information is directly available through the list endpoint or the details endpoint.
list the workflow runs for a repo and get an id
curl   -H "Accept: application/vnd.github.v3+json"   https://api.github.com/repos/wmde/wikibase-release-pipeline/actions/runs > out.json
But also the details does not contain any of the input data
curl   -H "Accept: application/vnd.github.v3+json"   https://api.github.com/repos/wmde/wikibase-release-pipeline/actions/runs/921094979

It kind of looks like we would have to have the github workflow itself callback somewhere to make this connection, which kind of makes this feel as we might be doing something they don't want us to do. Sounds fun to try that out though.

To some extent this could probably be circumvented by using a dedicated branch with commits that contain this info and could be passed as the "ref".

Example:

  1. Our CI pushes a commit to some branch on wikibase-release-pipeline which contains the information on what is supposed to run.
  2. Our CI triggers a workflow and setting the ref to this commit hash
Addshore triaged this task as Medium priority.Wed, Jun 9, 10:05 AM