- For Product Verification steps which already have some test coverage in a spec, identify where there are gaps between what we check for and what the user would see. For example, most of our tests will only check for the existence of 1 or 2 fields on the screen while the user manually going through Product Verification will see and implicitly be confirming much more on-screen information. There was for example a past a release made in which all the functionality covered in the tests was passing, but on the actual screens there were some visible errors...
- Increase coverage on a selection of 1-3 of these specs which seem most critical, then push a PR for the to consider. We would like to explore the line between too much (brittle and hard to read/maintain) vs too little coverage (not checking enough to capture obvious issues) in our test suite.
Wikibase Docker Release Product Acceptance Testing - Google Sheet
It should be relatively easy to find some areas with existing coverage, but once https://phabricator.wikimedia.org/T344970 is complete there will be a list there of areas with no coverage so the tests in focus here will be the opposite of that list.