Page MenuHomePhabricator

Product Verification coverage: Improve coverage where there are already related specs
Closed, ResolvedPublic2 Estimated Story Points

Description

  1. For Product Verification steps which already have some test coverage in a spec, identify where there are gaps between what we check for and what the user would see. For example, most of our tests will only check for the existence of 1 or 2 fields on the screen while the user manually going through Product Verification will see and implicitly be confirming much more on-screen information. There was for example a past a release made in which all the functionality covered in the tests was passing, but on the actual screens there were some visible errors...
  1. Increase coverage on a selection of 1-3 of these specs which seem most critical, then push a PR for the to consider. We would like to explore the line between too much (brittle and hard to read/maintain) vs too little coverage (not checking enough to capture obvious issues) in our test suite.

Wikibase Docker Release Product Acceptance Testing - Google Sheet

It should be relatively easy to find some areas with existing coverage, but once https://phabricator.wikimedia.org/T344970 is complete there will be a list there of areas with no coverage so the tests in focus here will be the opposite of that list.

Event Timeline

Rows 2-17 of the "Generic Wikibase Scenarios" tab in the "Wikibase Docker Release Product Acceptance Testing" sheet already have related tests in our test suite. Analyse those scenarios to identify any additional coverage needed.

adee_wmde changed the task status from Open to In Progress.Nov 14 2023, 8:28 AM
adee_wmde moved this task from Sprint Backlog to Doing on the Wikibase Suite Team (Sprint-∞) board.
darthmon_wmde set the point value for this task to 2.Nov 21 2023, 2:12 PM