In T290912#7368801, Sarai described the process WMDE uses in WiKit to perform automated visual regression tests per PR, which includes:
- Using Chromatic, a tool built by the Storybook maintainers, to:
- Publish a temporary Storybook instance for a PR
- Take snapshots for visual regression testing
- Requiring design review of each PR, which is facilitated by the automatically deployed test site, and requiring design signoff on any snapshot changes
See their docs for more info, and an example of the checks configured for PRs.
Integrating an automated visual regression testing process into our patch review process seems like a prudent way to both prevent bugs and ensure that design signoff is prioritized for each change.
Because we're considering moving away from Storybook (see T290912), we should consider other tools we could use to create a similar setup. Ideally we would have:
- Automated deployments of each patch pushed to gerrit
- Automated visual regression testing for each patch