Page MenuHomePhabricator

Investigate how we automatically can know that browser tests are working correctly
Open, Needs TriagePublicSpike

Description

Explore a failing switch for webdriver.io

When me and @zeljkofilipin been upgrading webdriver.io we have seen tests that success independently of what the tests expects. In some cases it has been that the user expects something true and get a promise (that is always is true). We then discussed the idea of creating something like a a failing switch where you could break all webdriver.io tests. For example: you have ten tests that pass. Use the switch and invert the conditions for the tests. If the tests was the correctly created, they would all fail. We could then automate this and check that all tests inverted makes the tests fails. If not, we know that the test is broken.

Lets explore the possibility to build this into core, so that we by one configuration could invert the checks.

For your solution make sure only core would be changed and no other library. Also there's a possibility this is not possible, lets find out.

AC

  • Time box the time spent on this spike to X hours to explore different ideas
  • Document your idea(s) and the outcomes and estimate how much work it would take to implement. If it's doable, create a task for it.