Page MenuHomePhabricator

[1 hour] Scenario: Search with search in pages button sometimes fails
Closed, ResolvedPublic2 Story Points

Description


A timing issue seems to occasionally cause cucumber features/search.feature:24 # Scenario: Search with search in pages button to fail.
This has been seen on the per commit browser test job as well as the job that runs against the beta cluster.

The error appears to occur when during the search overlay browser test, the search overlay mysteriously closes
https://integration.wikimedia.org/ci/job/mwext-mw-selenium-jessie/2590/console (please update link if it breaks)

Viewing the video (using VLC) the issue is that the search overlay closes somewhere inside the "I type into search box "Test is used by Selenium web driver" step due to the search overlay closing unexpectedly.

When(/^I type into search box "(.+)"$/) do |search_term|

We should make this more failproof as it causes lots of false positives in Jenkins.

Error Message

failed Search with search in pages button
Stacktrace

    Scenario: Search with search in pages button

Given I am using the mobile site
And I am in beta mode
And the page "Selenium search test" exists
And I am on the "Main Page" page
And I am viewing the site in mobile mode
And I click the search icon
And I see the search overlay
When I type into search box "Test is used by Selenium web driver"
And I see the search in pages button
And I click the search in pages button
Then I should see a list of search results

Message:

    Sauce Labs job URL: http://saucelabs.com/jobs/a296d0223c4945718d9de28932a6969d

timed out after 10 seconds, waiting for {:css=>".mw-search-results", :tag_name=>"ul"} to become present (Watir::Wait::TimeoutError)
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/watir-webdriver-0.9.1/lib/watir-webdriver/wait.rb:44:in `until'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/watir-webdriver-0.9.1/lib/watir-webdriver/wait.rb:223:in `wait_until_present'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/page-object-1.1.0/lib/page-object/platforms/watir_webdriver/element.rb:160:in `when_present'
/srv/jenkins-workspace/workspace/selenium-MobileFrontend/BROWSER/firefox/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/tests/browser/features/step_definitions/search_steps.rb:63:in `block in <top (required)>'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_step_definition.rb:97:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/step_match.rb:25:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:60:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:38:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:106:in `block in visit_step'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:105:in `visit_step'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:19:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:100:in `block in visit_steps'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:99:in `visit_steps'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:15:in `block in execute'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:83:in `block (2 levels) in with_hooks'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:99:in `before_and_after'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:82:in `block in with_hooks'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:120:in `call'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:120:in `block (3 levels) in around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:9:in `block in around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:97:in `call'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:97:in `execute_around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:8:in `around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:119:in `block (2 levels) in around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:123:in `call'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:123:in `around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:94:in `around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:81:in `with_hooks'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:13:in `execute'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:32:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:79:in `with_visitor'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:31:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:58:in `block in visit_feature_element'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:57:in `visit_feature_element'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:38:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:37:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:37:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:27:in `block in visit_feature'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:26:in `visit_feature'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:28:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:27:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:20:in `visit_features'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:49:in `run!'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/cli/main.rb:47:in `execute!'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/bin/cucumber:13:in `<top (required)>'
/mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `load'
/mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `<main>'
tests/browser/features/search.feature:30:in `Then I should see a list of search results'

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 20 2017, 9:26 PM
Jdlrobson updated the task description. (Show Details)Jan 20 2017, 9:26 PM
Jdlrobson triaged this task as High priority.Jan 24 2017, 8:17 PM
Jdlrobson moved this task from To Triage to Upcoming on the Readers-Web-Backlog board.

I'm nominating this for next sprint and marking this as high as it's slowing down code review. Seen again here -> https://gerrit.wikimedia.org/r/#/c/333834/

Jdlrobson added subscribers: ovasileva, bmansurov.

Still failing occassionally. https://integration.wikimedia.org/ci/job/mwext-mw-selenium-jessie/2590/console
Viewing the video (I had to use VLC) I see the issue is that the search overlay closes somewhere inside the "I type into search box "Test is used by Selenium web driver" step.

I imagine something is triggering a blur event under certain circumstances.
Let's take a closer look at the rule.

When(/^I type into search box "(.+)"$/) do |search_term|

@ovasileva @bmansurov since this is causing friction in sprints, please can we give this some time? e.g. timeboxed 2 pointer.

bmansurov renamed this task from Scenario: Search with search in pages button sometimes fails to [1 hour] Scenario: Search with search in pages button sometimes fails.Apr 5 2017, 3:19 PM
bmansurov set the point value for this task to 2.
phuedx added a subscriber: phuedx.Apr 10 2017, 8:02 AM

How does a timeboxed pointed story work?

bmansurov updated the task description. (Show Details)Apr 10 2017, 7:41 PM

I see the following message at the end of the screencast of the failure (attached in case it's taken down):

Service Temporarily Unavailable
Our servers are currently under maintenance or experiencing a technical problem. Please try again in a few minutes.

To conclude, the problem is with the back-end and not related to our code.

bmansurov removed bmansurov as the assignee of this task.Apr 10 2017, 7:58 PM

Engineers, would someone review T155855#3169537 and sign off if it looks right?

phuedx closed this task as Resolved.Apr 11 2017, 11:36 AM
phuedx claimed this task.

👍

Jdlrobson reopened this task as Open.Apr 11 2017, 2:37 PM

See comment and link to video above.
The issue happens occasionally due to a blur event. The service unavailable issue is not the same.

https://integration.wikimedia.org/ci/job/mwext-mw-selenium-jessie/2590/console

phuedx removed phuedx as the assignee of this task.Apr 11 2017, 3:32 PM
phuedx moved this task from Needs More Work to To Do on the Reading-Web-Sprint-95 board.
Jdlrobson updated the task description. (Show Details)Apr 11 2017, 4:00 PM

Updated description to avoid further confusion.

bmansurov moved this task from To Do to Doing on the Reading-Web-Sprint-95 board.
Jdlrobson updated the task description. (Show Details)Apr 12 2017, 8:41 PM

Change 348093 had a related patch set uploaded (by Bmansurov):
[mediawiki/extensions/MobileFrontend@master] QA: Make sure to use correct selector

https://gerrit.wikimedia.org/r/348093

bmansurov reassigned this task from bmansurov to Jdlrobson.Apr 13 2017, 5:11 PM

Change 348093 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] QA: Make sure to use correct selector

https://gerrit.wikimedia.org/r/348093

Jdlrobson closed this task as Resolved.Apr 14 2017, 12:17 AM

We can reopen this task if this bug shows its face again, but for the time being let's just hope.

Jdlrobson reopened this task as Open.May 3 2017, 5:54 PM
Jdlrobson removed a project: Patch-For-Review.

and https://gerrit.wikimedia.org/r/#/c/349261/
So I think this is still a problem :/

The behaviour of onClickSearchContent is very suspicious. It triggers a back button press before submitting the form after a setTimeout. This seems suspicious to me. From watching the video it looks like what happens is that the back button is triggered but the form is not submitted - maybe cancelled by the back navigation.

Given the #/search route is harmless I wonder if we need to do history.back at all in this case...

@Jdlrobson - seems like this should be in triaged but future - does this still need analysis or just a longer timebox?

@ovasileva no it's purposely in needs analysis. Last time we tried to fix this it didn't work so I'd like us to understand the root cause better before committing to it.

Jdlrobson lowered the priority of this task from High to Normal.May 26 2017, 5:46 PM
Jdlrobson lowered the priority of this task from Normal to Low.

It's irregular.

Jdlrobson closed this task as Resolved.Jul 10 2017, 10:28 PM

Not seen this in a while

Jdlrobson reopened this task as Open.Jul 11 2017, 12:35 AM

I spoke too soon....
https://gerrit.wikimedia.org/r/364348 QA: Fix flakey tests by adding additional steps
should fix it.

This is hitting me again today. I believe I've worked out why it's happening so pulling into sprint. Hopefully this time it will go away for real..

Change 364348 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] QA: Fix flakey tests by adding additional steps

https://gerrit.wikimedia.org/r/364348

Change 364348 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] QA: Fix flakey tests by adding additional steps

https://gerrit.wikimedia.org/r/364348