Many browser tests fail for reasons other than an issue with the code or the test environment: a flaky timeout, a momentary network glitch, etc. etc.
It would be convenient to build into the command that calls Cucumber to run the tests the ability to:
- track which test is being run
- evaluate the return status of each test, either per Feature or possibly even per Secenario
- if the return value for the individual test is non-zero, do not record the test result. Instead run the single test with the single failure.
- if the second run of the single test returns a non-zero result, record that result as usual
- if the test passes on the second run, record that result and continue running the tests in the build.
Some years ago I wrote a harness like this in Ruby. If it is not convenient to do this directly in a shell script, it might be possible to use Ruby and shell out for the appropriate commands.
A low hanging fruit was T98968 or retry the cucumber test on Net::ReadTimeout