Page MenuHomePhabricator

Consider automated visual regression testing in the new Vue component library
Closed, DuplicatePublic

Description

In T290912#7368801, Sarai described the process WMDE uses in WiKit to perform automated visual regression tests per PR, which includes:

  • Using Chromatic, a tool built by the Storybook maintainers, to:
    • Publish a temporary Storybook instance for a PR
    • Take snapshots for visual regression testing
  • Requiring design review of each PR, which is facilitated by the automatically deployed test site, and requiring design signoff on any snapshot changes

See their docs for more info, and an example of the checks configured for PRs.

Integrating an automated visual regression testing process into our patch review process seems like a prudent way to both prevent bugs and ensure that design signoff is prioritized for each change.

Because we're considering moving away from Storybook (see T290912), we should consider other tools we could use to create a similar setup. Ideally we would have:

  1. Automated deployments of each patch pushed to gerrit
  2. Automated visual regression testing for each patch

Event Timeline

@Sarai-WMDE please feel free to edit the task description if I've missed anything!

Web team has built out a solution here without jenkins that is working well so far.

Reports can be found at https://pixel.wmcloud.org

More information at T302246: [GOAL] Leverage Automated Visual Regression Testing

We will be happy to talk to you about this if you are interested.

STH moved this task from Inbox to Needs Refinement on the Design-System-Team board.