Page MenuHomePhabricator
Create Task
Maniphest T155855

[1 hour] Scenario: Search with search in pages button sometimes fails
Closed, ResolvedPublic2 Estimated Story Points
Actions

Assigned To
Jdlrobson
Authored By
Jdlrobson
Jan 20 2017, 9:26 PM
Tags
Referenced Files
F7512011: Search: Search with search in pages button.mp4
Apr 12 2017, 8:41 PM
F7465533: video.flv
Apr 10 2017, 7:57 PM
Subscribers
Aklapper
• bmansurov
gerritbot
Jdlrobson
• Jhernandez
ovasileva
phuedx

Description

Search: Search with search in pages button.mp4207 KBDownload

A timing issue seems to occasionally cause cucumber features/search.feature:24 # Scenario: Search with search in pages button to fail.
This has been seen on the per commit browser test job as well as the job that runs against the beta cluster.

The error appears to occur when during the search overlay browser test, the search overlay mysteriously closes
https://integration.wikimedia.org/ci/job/mwext-mw-selenium-jessie/2590/console (please update link if it breaks)

Viewing the video (using VLC) the issue is that the search overlay closes somewhere inside the "I type into search box "Test is used by Selenium web driver" step due to the search overlay closing unexpectedly.

When(/^I type into search box "(.+)"$/) do |search_term|

We should make this more failproof as it causes lots of false positives in Jenkins.

Error Message

failed Search with search in pages button
Stacktrace

    Scenario: Search with search in pages button

Given I am using the mobile site
And I am in beta mode
And the page "Selenium search test" exists
And I am on the "Main Page" page
And I am viewing the site in mobile mode
And I click the search icon
And I see the search overlay
When I type into search box "Test is used by Selenium web driver"
And I see the search in pages button
And I click the search in pages button
Then I should see a list of search results

Message:

    Sauce Labs job URL: http://saucelabs.com/jobs/a296d0223c4945718d9de28932a6969d

timed out after 10 seconds, waiting for {:css=>".mw-search-results", :tag_name=>"ul"} to become present (Watir::Wait::TimeoutError)
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/watir-webdriver-0.9.1/lib/watir-webdriver/wait.rb:44:in `until'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/watir-webdriver-0.9.1/lib/watir-webdriver/wait.rb:223:in `wait_until_present'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/page-object-1.1.0/lib/page-object/platforms/watir_webdriver/element.rb:160:in `when_present'
/srv/jenkins-workspace/workspace/selenium-MobileFrontend/BROWSER/firefox/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/tests/browser/features/step_definitions/search_steps.rb:63:in `block in <top (required)>'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_step_definition.rb:97:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/step_match.rb:25:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:60:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:38:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:106:in `block in visit_step'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:105:in `visit_step'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:19:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:100:in `block in visit_steps'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:99:in `visit_steps'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:15:in `block in execute'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:83:in `block (2 levels) in with_hooks'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:99:in `before_and_after'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:82:in `block in with_hooks'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:120:in `call'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:120:in `block (3 levels) in around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:9:in `block in around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:97:in `call'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:97:in `execute_around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:8:in `around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:119:in `block (2 levels) in around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:123:in `call'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:123:in `around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:94:in `around'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:81:in `with_hooks'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:13:in `execute'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:32:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:79:in `with_visitor'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:31:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:58:in `block in visit_feature_element'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:57:in `visit_feature_element'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:38:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:37:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:37:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:27:in `block in visit_feature'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:26:in `visit_feature'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:28:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:27:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:20:in `visit_features'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:49:in `run!'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/cli/main.rb:47:in `execute!'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/bin/cucumber:13:in `<top (required)>'
/mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `load'
/mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `<main>'
tests/browser/features/search.feature:30:in `Then I should see a list of search results'

Details

SubjectRepoBranchLines +/-
QA: Fix flakey tests by adding additional stepsmediawiki/extensions/MobileFrontendmaster+3 -1
QA: Make sure to use correct selectormediawiki/extensions/MobileFrontendmaster+1 -1
Customize query in gerrit

Related Objects

Event Timeline

Jdlrobson created this task.Jan 20 2017, 9:26 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 20 2017, 9:26 PM
Jdlrobson updated the task description. (Show Details)Jan 20 2017, 9:26 PM
Jdlrobson triaged this task as High priority.Jan 24 2017, 8:17 PM
Jdlrobson moved this task from Incoming to Upcoming on the Web-Team-Backlog board.

I'm nominating this for next sprint and marking this as high as it's slowing down code review. Seen again here -> https://gerrit.wikimedia.org/r/#/c/333834/

ovasileva moved this task from Upcoming to Triaged but Future on the Web-Team-Backlog board.Mar 7 2017, 12:05 PM
Jdlrobson moved this task from Triaged but Future to Upcoming on the Web-Team-Backlog board.Apr 4 2017, 7:11 PM
Jdlrobson added subscribers: ovasileva, • bmansurov.

Still failing occassionally. https://integration.wikimedia.org/ci/job/mwext-mw-selenium-jessie/2590/console
Viewing the video (I had to use VLC) I see the issue is that the search overlay closes somewhere inside the "I type into search box "Test is used by Selenium web driver" step.

I imagine something is triggering a blur event under certain circumstances.
Let's take a closer look at the rule.

When(/^I type into search box "(.+)"$/) do |search_term|

@ovasileva @bmansurov since this is causing friction in sprints, please can we give this some time? e.g. timeboxed 2 pointer.

ovasileva added a project: Reading-Web-Sprint-95.Apr 5 2017, 1:46 PM
• bmansurov renamed this task from Scenario: Search with search in pages button sometimes fails to [1 hour] Scenario: Search with search in pages button sometimes fails.Apr 5 2017, 3:19 PM
• bmansurov set the point value for this task to 2.
ovasileva moved this task from Upcoming to 2016-17 Q3 on the Web-Team-Backlog board.Apr 5 2017, 5:53 PM
phuedx subscribed.Apr 10 2017, 8:02 AM

How does a timeboxed pointed story work?

Jdlrobson moved this task from Needs Analysis to To Do on the Reading-Web-Sprint-95 board.Apr 10 2017, 5:16 PM
• bmansurov claimed this task.Apr 10 2017, 7:37 PM
• bmansurov updated the task description. (Show Details)Apr 10 2017, 7:41 PM
• bmansurov moved this task from To Do to Ready for Signoff on the Reading-Web-Sprint-95 board.Apr 10 2017, 7:57 PM

I see the following message at the end of the screencast of the failure (attached in case it's taken down):

Service Temporarily Unavailable
Our servers are currently under maintenance or experiencing a technical problem. Please try again in a few minutes.

To conclude, the problem is with the back-end and not related to our code.

video.flv1 MBDownload

• bmansurov removed • bmansurov as the assignee of this task.Apr 10 2017, 7:58 PM

Engineers, would someone review T155855#3169537 and sign off if it looks right?

phuedx closed this task as Resolved.Apr 11 2017, 11:36 AM
phuedx claimed this task.

👍

Jdlrobson reopened this task as Open.Apr 11 2017, 2:37 PM

See comment and link to video above.
The issue happens occasionally due to a blur event. The service unavailable issue is not the same.

https://integration.wikimedia.org/ci/job/mwext-mw-selenium-jessie/2590/console

Jdlrobson moved this task from Ready for Signoff to Needs More Work on the Reading-Web-Sprint-95 board.Apr 11 2017, 2:37 PM
phuedx removed phuedx as the assignee of this task.Apr 11 2017, 3:32 PM
phuedx moved this task from Needs More Work to To Do on the Reading-Web-Sprint-95 board.
Jdlrobson updated the task description. (Show Details)Apr 11 2017, 4:00 PM

Updated description to avoid further confusion.

• bmansurov claimed this task.Apr 12 2017, 7:58 PM
• bmansurov moved this task from To Do to Doing on the Reading-Web-Sprint-95 board.
Jdlrobson updated the task description. (Show Details)Apr 12 2017, 8:41 PM
gerritbot subscribed.Apr 13 2017, 4:21 PM

Change 348093 had a related patch set uploaded (by Bmansurov):
[mediawiki/extensions/MobileFrontend@master] QA: Make sure to use correct selector

https://gerrit.wikimedia.org/r/348093

gerritbot added a project: Patch-For-Review.Apr 13 2017, 4:21 PM
• bmansurov moved this task from Doing to Needs Code Review on the Reading-Web-Sprint-95 board.Apr 13 2017, 4:21 PM
• bmansurov reassigned this task from • bmansurov to Jdlrobson.Apr 13 2017, 5:11 PM
MBinder_WMF moved this task from 2016-17 Q3 to 2016-17 Q4 on the Web-Team-Backlog board.Apr 13 2017, 7:10 PM
gerritbot added a comment.Apr 13 2017, 11:05 PM

Change 348093 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] QA: Make sure to use correct selector

https://gerrit.wikimedia.org/r/348093

ReleaseTaggerBot added a project: MW-1.29-release (WMF-deploy-2017-04-25_(1.29.0-wmf.21)).Apr 14 2017, 12:00 AM
Jdlrobson closed this task as Resolved.Apr 14 2017, 12:17 AM
Jdlrobson moved this task from Needs Code Review to Ready for Signoff on the Reading-Web-Sprint-95 board.

We can reopen this task if this bug shows its face again, but for the time being let's just hope.

Jdlrobson merged a task: T98476: QA: Search with search in pages button test broken.May 2 2017, 2:30 PM
Jdlrobson added a subscriber: • Jhernandez.
Jdlrobson added a comment.May 2 2017, 2:35 PM

This failed again (https://integration.wikimedia.org/ci/job/mwext-mw-selenium-jessie/3380/consoleFull in https://gerrit.wikimedia.org/r/#/c/351248/3

Jdlrobson reopened this task as Open.May 3 2017, 5:54 PM
Jdlrobson removed a project: Patch-For-Review.

and https://gerrit.wikimedia.org/r/#/c/349261/
So I think this is still a problem :/

Jdlrobson moved this task from 2016-17 Q4 to Needs Prioritization on the Web-Team-Backlog board.May 3 2017, 7:31 PM
Jdlrobson added a comment.May 3 2017, 10:00 PM

The behaviour of onClickSearchContent is very suspicious. It triggers a back button press before submitting the form after a setTimeout. This seems suspicious to me. From watching the video it looks like what happens is that the back button is triggered but the form is not submitted - maybe cancelled by the back navigation.

Given the #/search route is harmless I wonder if we need to do history.back at all in this case...

ovasileva added a comment.May 9 2017, 3:46 PM

@Jdlrobson - seems like this should be in triaged but future - does this still need analysis or just a longer timebox?

Jdlrobson added a comment.May 10 2017, 7:48 AM

@ovasileva no it's purposely in needs analysis. Last time we tried to fix this it didn't work so I'd like us to understand the root cause better before committing to it.

Krinkle edited projects, added MW-1.29-release-notes; removed MW-1.29-release (WMF-deploy-2017-04-25_(1.29.0-wmf.21)).May 25 2017, 2:24 PM
Jdlrobson lowered the priority of this task from High to Medium.May 26 2017, 5:46 PM
Jdlrobson lowered the priority of this task from Medium to Low.

It's irregular.

Jdlrobson added a comment.Jun 5 2017, 11:52 PM

Still happening though - https://integration.wikimedia.org/ci/job/mwext-mw-selenium-jessie/4323/

Jdlrobson closed this task as Resolved.Jul 10 2017, 10:28 PM

Not seen this in a while

Jdlrobson reopened this task as Open.Jul 11 2017, 12:35 AM
Jdlrobson added a project: Readers-Web-Kanbanana-Board-Old.

I spoke too soon....
https://gerrit.wikimedia.org/r/364348 QA: Fix flakey tests by adding additional steps
should fix it.

Jdlrobson added a project: Unplanned-Sprint-Work.Jul 11 2017, 12:35 AM

This is hitting me again today. I believe I've worked out why it's happening so pulling into sprint. Hopefully this time it will go away for real..

Jdlrobson moved this task from To Do to Needs Code Review on the Readers-Web-Kanbanana-Board-Old board.Jul 11 2017, 12:36 AM
gerritbot added a comment.Jul 11 2017, 12:36 AM

Change 364348 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] QA: Fix flakey tests by adding additional steps

https://gerrit.wikimedia.org/r/364348

gerritbot added a project: Patch-For-Review.Jul 11 2017, 12:36 AM
gerritbot added a comment.Jul 11 2017, 3:35 PM

Change 364348 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] QA: Fix flakey tests by adding additional steps

https://gerrit.wikimedia.org/r/364348

Jdlrobson moved this task from Needs Code Review to Ready for Signoff on the Readers-Web-Kanbanana-Board-Old board.Jul 11 2017, 5:02 PM
Jdlrobson closed this task as Resolved.Jul 11 2017, 10:09 PM

Build is green again: https://integration.wikimedia.org/ci/view/Reading-Web/job/selenium-MobileFrontend/

MBinder_WMF moved this task from Needs Prioritization to 2017-18 Q1 on the Web-Team-Backlog board.Oct 10 2017, 7:28 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL