Page MenuHomePhabricator

Upgrade Cucumber from version 1 to version 3
Closed, DeclinedPublic

Description

Cucumber 2.4 has a feature to rerun failed tests:

From https://github.com/cucumber/cucumber-ruby/blob/master/History.md

Add --retry option to retry failed tests as part of the same run (#920 @DanaScheider)

mediawiki_selenium is at the latest release of version 1

spec.add_runtime_dependency 'cucumber', '~> 1.3', '>= 1.3.20'

Event Timeline

Change 341523 had a related patch set uploaded (by Zfilipin):
[mediawiki/selenium] WIP Problem: Can not use --retry option to retry failed tests as part of the same run

https://gerrit.wikimedia.org/r/341523

Looks like the command line option is there in Cucumber 2.4, but the feature does not work. A simple test:

$ bundle exec cucumber --retry 1
Feature: Test

  Scenario: Test # features/test.feature:3
    Given I test # features/step_definitions/test_steps.rb:1
      
      expected false
           got true
       (RSpec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/test_steps.rb:3:in `/^I test$/'
      features/test.feature:4:in `Given I test'

Failing Scenarios:
cucumber features/test.feature:3 # Scenario: Test

1 scenario (1 failed)
1 step (1 failed)
0m0.028s

Retry works as expected with the latest prerelease version of Cucumber (3.0.0.pre.1):

$ bundle exec cucumber --retry 1
Feature: Test

  Scenario: Test # features/test.feature:3
    Given I test # features/step_definitions/test_steps.rb:1
      
      expected false
           got true
       (RSpec::Expectations::ExpectationNotMetError)
      ./features/step_definitions/test_steps.rb:3:in `/^I test$/'
      features/test.feature:4:in `Given I test'

  Scenario: Test # features/test.feature:3

Failing Scenarios:
cucumber features/test.feature:3 # Scenario: Test
cucumber features/test.feature:3 # Scenario: Test

2 scenarios (2 failed)
2 steps (2 failed)
0m0.031s
zeljkofilipin renamed this task from Upgrade Cucumber from version 1 to version 2 to Upgrade Cucumber from version 1 to version 3.Mar 9 2017, 5:21 PM
zeljkofilipin lowered the priority of this task from Medium to Low.May 29 2017, 10:27 AM

Change 341523 abandoned by Zfilipin:
WIP Problem: Can not use --retry option to retry failed tests as part of the same run

Reason:
I do not plan to work on this because of

https://phabricator.wikimedia.org/T173488

Announce Selenium Ruby framework deprecation on appropriate mailing lists (QA, engineering, wikitech-l)

https://gerrit.wikimedia.org/r/341523