SauceLabs job annotation breaks when a scenario is skipped (0.4.x only)
Closed, ResolvedPublic

Description

If a scenario is skipped entirely, there's no SauceLabs session and the API call responsible for annotating the session fails. This only applies to <= 0.4.x versions of MW-Selenium. The teardown method in 1.x should work as it iterates over all open browser sessions in the factory instead of assuming the state of a @browser instance variable.

https://integration.wikimedia.org/ci/view/BrowserTests/view/-Dashboard/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-firefox-sauce/701/consoleFull

02:57:59   @smoke @skip
02:57:59   Scenario: Able to access left navigation in basic non-JavaScript site    # features/no_javascript_site.feature:18
02:57:59     When I click on "Random" in the main navigation menu                   # features/step_definitions/mainmenu_steps.rb:5
02:57:59     And I click on the main navigation button                              # features/step_definitions/mainmenu_steps.rb:1
02:57:59     Then I should see a link to "Home" in the main navigation menu         # features/step_definitions/mainmenu_steps.rb:10
02:57:59     And I should see a link to "Random" in the main navigation menu        # features/step_definitions/mainmenu_steps.rb:10
02:57:59     And I should see a link to "Settings" in the main navigation menu      # features/step_definitions/mainmenu_steps.rb:10
02:57:59     And I should not see a link to "Watchlist" in the main navigation menu # features/step_definitions/mainmenu_steps.rb:14
02:57:59     And I should not see a link to "Log in" in the main navigation menu    # features/step_definitions/mainmenu_steps.rb:14
02:57:59     And I should not see a link to "Uploads" in the main navigation menu   # features/step_definitions/mainmenu_steps.rb:14
02:57:59       404 Resource Not Found (RestClient::ResourceNotFound)
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/rest-client-1.7.3/lib/restclient/abstract_response.rb:48:in `return!'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/rest-client-1.7.3/lib/restclient/request.rb:495:in `process_result'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/rest-client-1.7.3/lib/restclient/request.rb:421:in `block in transmit'
02:57:59       /usr/lib/ruby/2.0.0/net/http.rb:852:in `start'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/rest-client-1.7.3/lib/restclient/request.rb:413:in `transmit'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/rest-client-1.7.3/lib/restclient/request.rb:176:in `execute'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/rest-client-1.7.3/lib/restclient/request.rb:41:in `execute'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/mediawiki_selenium-0.4.2/lib/mediawiki_selenium/support/env.rb:84:in `sauce_api'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/mediawiki_selenium-0.4.2/lib/mediawiki_selenium/support/hooks.rb:99:in `block in <top (required)>'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/rb_support/rb_hook.rb:14:in `invoke'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/language_support/language_methods.rb:114:in `invoke'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/language_support/language_methods.rb:108:in `block in execute_after'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/language_support/language_methods.rb:107:in `reverse_each'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/language_support/language_methods.rb:107:in `execute_after'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/language_support/language_methods.rb:19:in `after'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime/support_code.rb:112:in `block in fire_hook'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime/support_code.rb:111:in `each'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime/support_code.rb:111:in `fire_hook'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime.rb:113:in `after'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime.rb:100:in `before_and_after'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime.rb:82:in `block in with_hooks'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime/support_code.rb:120:in `call'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime/support_code.rb:120:in `block (3 levels) in around'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/language_support/language_methods.rb:9:in `block in around'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/language_support/language_methods.rb:97:in `call'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/language_support/language_methods.rb:97:in `execute_around'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/language_support/language_methods.rb:8:in `around'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime/support_code.rb:119:in `block (2 levels) in around'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime/support_code.rb:123:in `call'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime/support_code.rb:123:in `around'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime.rb:94:in `around'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime.rb:81:in `with_hooks'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:13:in `execute'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/scenario.rb:32:in `block in accept'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/scenario.rb:79:in `with_visitor'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/scenario.rb:31:in `accept'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:58:in `block in visit_feature_element'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:57:in `visit_feature_element'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/feature.rb:38:in `block in accept'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/feature.rb:37:in `each'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/feature.rb:37:in `accept'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:27:in `block in visit_feature'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:26:in `visit_feature'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/features.rb:28:in `block in accept'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/features.rb:17:in `each'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/features.rb:17:in `each'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/features.rb:27:in `accept'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/ast/tree_walker.rb:20:in `visit_features'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/runtime.rb:49:in `run!'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/lib/cucumber/cli/main.rb:47:in `execute!'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/gems/cucumber-1.3.19/bin/cucumber:13:in `<top (required)>'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/bin/cucumber:23:in `load'
02:57:59       /mnt/jenkins-workspace/workspace/gems/2.0.0/bin/cucumber:23:in `<main>'
dduvall created this task.Jun 3 2015, 8:27 PM
dduvall updated the task description. (Show Details)
dduvall raised the priority of this task from to Unbreak Now!.
dduvall added subscribers: dduvall, Jdlrobson.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJun 3 2015, 8:27 PM

Change 215796 had a related patch set uploaded (by Dduvall):
Check for session ID before updating SauceLabs job

https://gerrit.wikimedia.org/r/215796

Change 215796 merged by jenkins-bot:
Check for session ID before updating SauceLabs job

https://gerrit.wikimedia.org/r/215796

dduvall closed this task as Resolved.Jun 5 2015, 6:40 PM
dduvall claimed this task.
dduvall moved this task from In Progress to Done on the Browser-Tests-Infrastructure board.