Page MenuHomePhabricator

Selenium test 'Scenario: Check components in diff summary' fails with edit conflict
Closed, ResolvedPublic

Description

On May 22nd, 2018 Jenkins tests for MobileFrontend started to fail with:

Scenario: Check components in diff summary

Given I am using the mobile site
And the page "Selenium diff test" exists and has at least "51" edits
And I am on the "Selenium diff test" page
When I click on the history link in the last modified bar
When I click the link in the header bar
Then the text of the first heading should be "Selenium diff test"

Message:

    Uh-oh. Could not find link to Sauce Labs job URL.
Edit conflict. (editconflict) (MediawikiApi::ApiError)
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.0/lib/mediawiki_api/client.rb:211:in `send_request'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.0/lib/mediawiki_api/client.rb:232:in `raw_action'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.0/lib/mediawiki_api/client.rb:36:in `action'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.0/lib/mediawiki_api/client.rb:111:in `edit'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.0/lib/mediawiki_api/client.rb:103:in `create_page'
/srv/jenkins-workspace/workspace/selenium-MobileFrontend/BROWSER/chrome/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/tests/browser/features/step_definitions/create_page_api_steps.rb:12:in `block (3 levels) in <top (required)>'
/srv/jenkins-workspace/workspace/selenium-MobileFrontend/BROWSER/chrome/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/tests/browser/features/step_definitions/create_page_api_steps.rb:11:in `upto'
/srv/jenkins-workspace/workspace/selenium-MobileFrontend/BROWSER/chrome/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/tests/browser/features/step_definitions/create_page_api_steps.rb:11:in `block (2 levels) in <top (required)>'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/page_factory.rb:43:in `block in on_page'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/page-object-1.1.0/lib/page-object/page_factory.rb:75:in `call'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/page-object-1.1.0/lib/page-object/page_factory.rb:75:in `on_page'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_selenium-1.7.3/lib/mediawiki_selenium/page_factory.rb:28:in `on_page'
/srv/jenkins-workspace/workspace/selenium-MobileFrontend/BROWSER/chrome/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/tests/browser/features/step_definitions/create_page_api_steps.rb:10:in `block in <top (required)>'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_step_definition.rb:97:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/step_match.rb:25:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:60:in `invoke'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:38:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:106:in `block in visit_step'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:105:in `visit_step'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:19:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:100:in `block in visit_steps'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:99:in `visit_steps'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/background.rb:44:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/background.rb:54:in `block in with_visitor'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:79:in `with_visitor'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/background.rb:53:in `with_visitor'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/background.rb:41:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:64:in `block in visit_background'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:63:in `visit_background'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:36:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:27:in `block in visit_feature'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:26:in `visit_feature'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:28:in `block in accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:27:in `accept'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:20:in `visit_features'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:49:in `run!'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/cli/main.rb:47:in `execute!'
/mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/bin/cucumber:13:in `<top (required)>'
/mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `load'
/mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `<main>'
tests/browser/features/special_history.feature:6:in `And the page "Selenium diff test" exists and has at least "51" edits'

It's not consistent, sometimes Chrome tests fail, sometimes Firefox, sometimes both.

Details

Related Gerrit Patches:
mediawiki/extensions/MobileFrontend : masterQA: Drop failing test from beta cluster build
mediawiki/extensions/MobileFrontend : masterQA: Do not edit 51 pages on every browser test run

Event Timeline

pmiazga created this task.May 29 2018, 2:28 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptMay 29 2018, 2:28 PM

@Jdlrobson @ovasileva UBN? Jenkins keeps failing for a week.

pmiazga updated the task description. (Show Details)May 29 2018, 2:29 PM

Nope, not UBN given nothing is actually broken. High at most. Remember UBN notifies various people in Release Engineering for consideration as a deploy blocker :)

I've seen this happen before - usually because it starts in an error state. I've edited the page that's causing the problem manually and restarted the build. Might fix it...

Change 436032 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] QA: Do not edit 51 pages on every browser test run

https://gerrit.wikimedia.org/r/436032

@zeljkofilipin it looks like browser tests are running concurrently to me (and causing edit conflicts) OR something changed with the API code to allow for edit conflicts. It seems like the above patch is the most straightforward way to fix this while we're still using Ruby. Those edits are unnecessary on the beta cluster.

Jdlrobson triaged this task as High priority.May 29 2018, 3:18 PM

Change 436032 abandoned by Jdlrobson:
QA: Do not edit 51 pages on every browser test run

Reason:
Looks like this is happening anyway... it checks existing edit count.

https://gerrit.wikimedia.org/r/436032

What's weird is that "Selenium diff test" has 51 edits so I'm not even sure why it's trying to edit this page at all..

Change 436309 had a related patch set uploaded (by Jdlrobson; owner: Jdlrobson):
[mediawiki/extensions/MobileFrontend@master] QA: Drop failing test from beta cluster build

https://gerrit.wikimedia.org/r/436309

Change 436309 merged by jenkins-bot:
[mediawiki/extensions/MobileFrontend@master] QA: Drop failing test from beta cluster build

https://gerrit.wikimedia.org/r/436309

Moving into the sprint to determine whether any additional work on this task is necessary.

pmiazga claimed this task.Jun 5 2018, 5:08 PM
pmiazga moved this task from To Do to Doing on the Readers-Web-Kanbanana-Board-Old board.

This test should not have been running on the Firefox build - it is not tagged @firefox - cc @zeljkofilipin it seems tests are being run for both Firefox and Chrome (but that's another problem)

https://gerrit.wikimedia.org/r/436309 seems to have addressed the problem now - the builds are green and as I said in the commit message "given we already have coverage in the integration tests, we don't need to run this in beta cluster. Not worth investing the time debugging this problem."

So there shouldn't be anything left to do here...

Mobile frontend CI jobs are passing for last couple days, all patches are merged. As @Jdlrobson says, there is nothing else to do. I'm resolving this task.

pmiazga closed this task as Resolved.Jun 6 2018, 3:38 PM
pmiazga moved this task from Doing to Ready for Signoff on the Readers-Web-Kanbanana-Board-Old board.
pmiazga removed a project: Patch-For-Review.
Vvjjkkii renamed this task from Selenium test 'Scenario: Check components in diff summary' fails with edit conflict to 12baaaaaaa.Jul 1 2018, 1:07 AM
Vvjjkkii reopened this task as Open.
Vvjjkkii removed pmiazga as the assignee of this task.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
CommunityTechBot renamed this task from 12baaaaaaa to Selenium test 'Scenario: Check components in diff summary' fails with edit conflict.Jul 2 2018, 1:09 PM
CommunityTechBot closed this task as Resolved.
CommunityTechBot assigned this task to pmiazga.
CommunityTechBot updated the task description. (Show Details)
CommunityTechBot added subscribers: gerritbot, Aklapper.