Page MenuHomePhabricator

Automated browser tests cannot create pages on the Beta Cluster
Closed, ResolvedPublic

Description

MinervaNeue, MobileFrontend, and QuickSurveys tests failed due to being unable to create pages in mainspace and/or the MediaWiki namespace.

Note that the failure message includes "Uh-oh. Could not find link to Sauce Labs job URL.". I suspect this is due to the Sauce Labs UA not being instructed to navigate before the test failed and so Sauce Labs can't provide a screencast for the test [0].

Failing QuickSurveys test
Failing MinervaNeue test


[0]
> Failed
> 
> Integration.Survey is visible with quicksurvey=true flag
> 
> Failing for the past 3 builds (Since Failed#534 )
> Took 1.7 sec.
> add description
> Error Message
> 
> failed Survey is visible with quicksurvey=true flag
> Stacktrace
> 
>     Scenario: Survey is visible with quicksurvey=true flag
> 
> Given the quick survey test pages are installed
> When I am on the "Quick survey test page stub" page with the quick survey flag enabled
> And the page has fully loaded
> And the survey code has fully loaded
> Then I should see the survey
> 
> Message:
> 
>     Uh-oh. Could not find link to Sauce Labs job URL.
> This page provides interface text for the software on this wiki, and is protected to prevent abuse. To add or change translations for all wikis, please use [https://translatewiki.net/ translatewiki.net], the MediaWiki localisation project. (protectednamespace-interface) (MediawikiApi::ApiError)
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.1/lib/mediawiki_api/client.rb:211:in `send_request'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.1/lib/mediawiki_api/client.rb:232:in `raw_action'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.1/lib/mediawiki_api/client.rb:36:in `action'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.1/lib/mediawiki_api/client.rb:111:in `edit'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/mediawiki_api-0.7.1/lib/mediawiki_api/client.rb:103:in `create_page'
> /srv/jenkins-workspace/workspace/selenium-QuickSurveys/BROWSER/chrome/MEDIAWIKI_ENVIRONMENT/beta/PLATFORM/Linux/label/BrowserTests/tests/browser/features/support/step_definitions/common_steps.rb:2:in `block in <top (required)>'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `instance_exec'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:48:in `block in cucumber_instance_exec'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:69:in `cucumber_run_with_backtrace_filtering'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/core_ext/instance_exec.rb:36:in `cucumber_instance_exec'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/rb_support/rb_step_definition.rb:97:in `invoke'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/step_match.rb:25:in `invoke'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:60:in `invoke'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_invocation.rb:38:in `accept'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:106:in `block in visit_step'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:105:in `visit_step'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:19:in `block in accept'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `each'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/step_collection.rb:18:in `accept'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:100:in `block in visit_steps'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:99:in `visit_steps'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:15:in `block in execute'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:83:in `block (2 levels) in with_hooks'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:99:in `before_and_after'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:82:in `block in with_hooks'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:120:in `call'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:120:in `block (3 levels) in around'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:9:in `block in around'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:97:in `call'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:97:in `execute_around'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/language_support/language_methods.rb:8:in `around'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:119:in `block (2 levels) in around'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:123:in `call'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime/support_code.rb:123:in `around'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:94:in `around'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:81:in `with_hooks'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:13:in `execute'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:32:in `block in accept'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:79:in `with_visitor'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/scenario.rb:31:in `accept'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:58:in `block in visit_feature_element'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:57:in `visit_feature_element'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:38:in `block in accept'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:37:in `each'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/feature.rb:37:in `accept'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:27:in `block in visit_feature'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:26:in `visit_feature'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:28:in `block in accept'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:17:in `each'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/features.rb:27:in `accept'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:21:in `block in visit_features'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:170:in `broadcast'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/ast/tree_walker.rb:20:in `visit_features'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/runtime.rb:49:in `run!'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/lib/cucumber/cli/main.rb:47:in `execute!'
> /mnt/home/jenkins-deploy/.gem/2.1.0/gems/cucumber-1.3.20/bin/cucumber:13:in `<top (required)>'
> /mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `load'
> /mnt/home/jenkins-deploy/.gem/2.1.0/bin/cucumber:23:in `<main>'
> tests/browser/features/integration.feature:7:in `Given the quick survey test pages are installed'

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptSep 20 2017, 1:32 PM
bmansurov updated the task description. (Show Details)
bmansurov added a project: Readers-Web-Backlog.

The Sauce Labs link missing but this isn't strictly the error: it looks like the MediaWiki namespace isn't editable.

@zeljkofilipin @hashar: Has the Beta Cluster been locked down recently (circa 18th September)?

phuedx added a comment.EditedSep 21 2017, 4:14 PM

This might be fixed now per T176222#3624781 onwards. I've started a new build.

phuedx added a subscriber: greg.Sep 21 2017, 4:30 PM

@greg: Following on from your comment on T176222, did the permissions for the MediaWiki namespace also get changed? That seems to be the reason that these tests are failing.

greg added a comment.Sep 21 2017, 4:38 PM

Maybe? I'm lost in MediaWiki permissions trying to figure that out. Help? :)

What page specifically can't be edited? A bunch of pages were salted recently so we might need to undo that.

@Legoktm: Looking at the step that's failing, the following pages can't be created/edited:

  • MediaWiki:anne-survey-question
  • MediaWiki:anne-survey-description
  • MediaWiki:anne-survey-answer-one
  • MediaWiki:anne-survey-answer-two
  • MediaWiki:anne-survey-answer-three
  • MediaWiki:anne-survey-answer-four

16:44, 21 September 2017 Greg (WMF) (talk | contribs) changed group membership for Selenium user from extended confirmed user to extended confirmed user, bot, administrator, oversight, confirmed user and Structured Discussions bot (Restore, not a spam bot, used by browser tests)
18:49, 19 September 2017 Selenium user (talk | contribs) was automatically promoted from (none) to extended confirmed user
08:14, 18 September 2017 Sau226 (talk | contribs) changed group membership for Selenium user from bot, confirmed user, Structured Discussions bot, oversight and administrator to (none) (Spam only user)

So @greg 's change should fix it.

Build #541 passed. Thanks, @greg and @Legoktm!

phuedx closed this task as Resolved.Sep 21 2017, 5:53 PM
phuedx claimed this task.

This appears to still be an intermittent issue for MobileFrontend.

Niedzielski reopened this task as Open.Sep 22 2017, 2:59 PM

Build #571 (started by @Niedzielski) passed.

Build #569 failed due to the Beta Cluster being under too much load (HTTP 503 from the MediaWiki API). I suspect that Sauce Labs doesn't provide a link for the test because the first step is failing, at which point no navigation has actually happened and so there's nothing to screenshot.

phuedx renamed this task from Tests cannot find link to Sauce Labs job URL to Automated browser tests cannot create pages on the Beta Cluster.Sep 25 2017, 9:31 AM
phuedx updated the task description. (Show Details)
phuedx closed this task as Resolved.Sep 25 2017, 9:34 AM

@Niedzielski: I've retitled this task to capture the problem that we were seeing initially (that the MobileFrontend/MinervaNeue, and QuickSurveys tests couldn't edit pages - see the trace in [0] in the description). AFAICT the "Uh-oh" message is a red herring here. Also, the MobileFrontend automated browser tests have passed four times in a row 🎉🎉🎉