Change Details

== Problem Various bugs hit our users in production due to UI regressions * T108641 - edit and watch icons shifted to the right * T108642 - toast messages lost centering * T159009 - a change to the UI header broke the language overlay These are difficult to detect with unit tests or browser tests and currently rely on manual testing and a user reporting the bug. We believe that by detecting UI regressions programatically we can avoid giving our users bad experiences. == How Various tools exist that allow taking screenshots at certain points in the workflow. These can be done as part of existing browser tests or as standalone tests. Using image diff comparison tools we can detect and report when UIs change. == Open questions [] what level do we test? component level? in middle of Selenium browser tests? [] when do we run? Daily/per commit [] where to run (against bc/vagrant/reading web staging) == Tooling * https://open.segment.com/niffy - a Perceptual diffing suite * gemini - https://github.com/gemini-testing/gemini * wdio - https://github.com/zinserjan/wdio-visual-regression-service * webdrivercss - https://github.com/webdriverio/webdrivercss = Plan [x] Explore currently available tools that claim to help detect UI regressions (e.g. via screenshots) and pick one that looks like the best fit [] ?

== Problem Various bugs hit our users in production due to UI regressions * T108641 - edit and watch icons shifted to the right * T108642 - toast messages lost centering * T159009 - a change to the UI header broke the language overlay These are difficult to detect with unit tests or browser tests and currently rely on manual testing and a user reporting the bug. We believe that by detecting UI regressions programatically we can avoid giving our users bad experiences. == How Various tools exist that allow taking screenshots at certain points in the workflow. These can be done as part of existing browser tests or as standalone tests. Using image diff comparison tools we can detect and report when UIs change. == Open questions [] what level do we test? component level? in middle of Selenium browser tests? [] when do we run? Daily/per commit [] where to run (against bc/vagrant/reading web staging) == Tooling * https://open.segment.com/niffy - a Perceptual diffing suite * gemini - https://github.com/gemini-testing/gemini * wdio - https://github.com/zinserjan/wdio-visual-regression-service * [deprecated] webdrivercss - https://github.com/webdriverio/webdrivercss = Plan [x] Explore currently available tools that claim to help detect UI regressions (e.g. via screenshots) and pick one that looks like the best fit [] ?

== Problem Various bugs hit our users in production due to UI regressions * T108641 - edit and watch icons shifted to the right * T108642 - toast messages lost centering * T159009 - a change to the UI header broke the language overlay These are difficult to detect with unit tests or browser tests and currently rely on manual testing and a user reporting the bug. We believe that by detecting UI regressions programatically we can avoid giving our users bad experiences. == How Various tools exist that allow taking screenshots at certain points in the workflow. These can be done as part of existing browser tests or as standalone tests. Using image diff comparison tools we can detect and report when UIs change. == Open questions [] what level do we test? component level? in middle of Selenium browser tests? [] when do we run? Daily/per commit [] where to run (against bc/vagrant/reading web staging) == Tooling * https://open.segment.com/niffy - a Perceptual diffing suite * gemini - https://github.com/gemini-testing/gemini * wdio - https://github.com/zinserjan/wdio-visual-regression-service * [deprecated] webdrivercss - https://github.com/webdriverio/webdrivercss = Plan [x] Explore currently available tools that claim to help detect UI regressions (e.g. via screenshots) and pick one that looks like the best fit [] ?