Our tests are sporadic, poorly laid out, and incomplete. We need to rethink and restructure them - this work needs to start with a review of the current situation and the best path forward for improving it.
We're already aware of the following issues, as a starting point:
- Emails have poor test coverage (previously tracked at T196620)
- Our test arrangement may not be ideal - each app has a single `tests.py` file that is often many thousands of lines long.
- We could be using request-mocking, using something like [[ https://github.com/kevin1024/vcrpy | VCR ]], to improve our test coverage of external dependencies.
- [[ https://docs.pytest.org/en/stable/fixture.html | Fixtures ]] might substantially simplify our test design
From this spike, we would like to understand:Process
- Which areas of the code are _missing_ test coverage?Use [[ https://coveralls.io/github/WikipediaLibrary/TWLight?branch=master | coveralls ]] to evaluate the test coverage for this app
- Consider the suitability of using fixtures in these tests to improve efficiency and reduce code
- How could we improve the structure and content of our tests to improve coverage and make future test writing easier?Evaluate whether splitting the test file into multiple would make sense, and if so how that split should happen
- For each test, is there accurate and up to date documentation? If not, add it in.