Page MenuHomePhabricator

Sitelink browser test sometimes fails with firefox because of rate limit?
Closed, ResolvedPublic

Description

Edit sitelinks/Remove multiple sitelinks unreliably fails on sauce and locally in firefox:

And I remove all sitelinks                   # features/step_definitions/sitelinks_steps.rb:51
  unable to locate element, using {:css=>"div[data-wb-sitelinks-group='wikipedia'] .wikibase-toolbar-container span.wikibase-toolbar-button-edit:not(.wikibase-toolbarbutton-disabled) > a"} (Watir::Exception::UnknownObjectException)
  (eval):1:in `process_watir_call'
  ./features/support/modules/sitelink_module.rb:71:in `remove_all_sitelinks'
  ./features/step_definitions/sitelinks_steps.rb:52:in `/^I remove all sitelinks$/'
  features/sitelinks.feature:204:in `And I remove all sitelinks'

Event Timeline

adrianheine raised the priority of this task from to Normal.
adrianheine updated the task description. (Show Details)
adrianheine added a subscriber: adrianheine.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 11 2016, 9:18 AM
adrianheine renamed this task from Sitelink browser test fails with firefox to Sitelink browser test sometimes fails with firefox.Feb 11 2016, 9:24 AM
adrianheine updated the task description. (Show Details)
adrianheine set Security to None.

This is a rate limiting problem. Whenever the test fails, I have the following in my debug log:

[ratelimit] User '0:0:0:0:0:0:0:1' (IP ::1) tripped my_wiki:limiter:edit:ip:%3A%3A1 at 8 (limit 8 in 60s)

@Tobi_WMDE_SW @Addshore @JanZerebecki Rate limiting settings for beta need to be adapted, either by IP or in general.

Probably @aude can help with this?

aude added a comment.Feb 11 2016, 11:01 AM

do we need more than saucelabs ip range for this? e.g. our office IP range if we run these also from the office?

Change 269944 had a related patch set uploaded (by Aude):
Exclude Sauce Labs IP ranges from rate limits on beta

https://gerrit.wikimedia.org/r/269944

Change 269944 merged by jenkins-bot:
Exclude Sauce Labs IP ranges from rate limits on beta

https://gerrit.wikimedia.org/r/269944

hoo added a subscriber: hoo.Feb 16 2016, 9:33 AM

Change deployed, please verify.

The failures seem unrelated to this. It seems that the some of the articles used for testing do not really exist on the corresponding test wikis. Either we need to change the test data or ideally make the test cases more robust.

Locally this failed because of rate limiting and stopped failing after I changed config. I'll try again locally.

I checked again: This works locally.

JanZerebecki renamed this task from Sitelink browser test sometimes fails with firefox to Sitelink browser test sometimes fails with firefox because of rate limit?.May 7 2016, 8:41 PM
JanZerebecki removed a project: Patch-For-Review.

Looking at the actual screenshots on Saucelabs, e.g. https://assets.saucelabs.com/jobs/e6cd0f782ac54c04bfaa75923bff8f00/0022screenshot.png suggested that it is a problem with the corresponding article on beta Wikipedia. I've just checked that, and indeed the article "Test" was not existing on German beta Wikipedia. I've re-created it, and I assume this test failure is fixed now.
However, as I've stated before, we need to make the test more robust, e.g. by ensuring the used test data does actually exist. We could do this in an extra Given-step.

adrianheine closed this task as Resolved.Jun 14 2016, 7:17 AM
adrianheine claimed this task.

Nice catch. When I tried it locally it probably ran against enwp, not beta.