Sitelink browser test sometimes fails with firefox because of rate limit?
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• adrianheine
	Feb 11 2016, 9:18 AM

Description

Edit sitelinks/Remove multiple sitelinks unreliably fails on sauce and locally in firefox:

And I remove all sitelinks                   # features/step_definitions/sitelinks_steps.rb:51
  unable to locate element, using {:css=>"div[data-wb-sitelinks-group='wikipedia'] .wikibase-toolbar-container span.wikibase-toolbar-button-edit:not(.wikibase-toolbarbutton-disabled) > a"} (Watir::Exception::UnknownObjectException)
  (eval):1:in `process_watir_call'
  ./features/support/modules/sitelink_module.rb:71:in `remove_all_sitelinks'
  ./features/step_definitions/sitelinks_steps.rb:52:in `/^I remove all sitelinks$/'
  features/sitelinks.feature:204:in `And I remove all sitelinks'

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	Exclude Sauce Labs IP ranges from rate limits on beta	operations/mediawiki-config	master	+5 -1

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Invalid	None	T108946 [Epic] Improve the development infrastructure
Resolved	zeljkofilipin	T94150 Fix scenarios that fail at en.wikipedia.beta.wmflabs.org or do not run them daily
Duplicate	None	T105985 Run Wikidata browser tests on testwikidata via Jenkins
Resolved	• Jonas	T88541 [DO NOT USE] Wikidata Browsertests (tracking) [superseded by #Browser-Tests]
Resolved	Tobi_WMDE_SW	T101497 [Story] Run browsertests regularly on test.wikidata.org via Jenkins
Resolved	JanZerebecki	T101498 [Task] Create a user with necessary privileges for browsertests on test.wikidata.org
Resolved	Tobi_WMDE_SW	T101499 [Task] Setup a Jenkins job to run Wikidata browsertests on test.wikidata.org
Resolved	Tobi_WMDE_SW	T92619 make Wikidata browsertests non-flaky (tracking)
Resolved	• adrianheine	T126585 Sitelink browser test sometimes fails with firefox because of rate limit?

Event Timeline

• adrianheine created this task.Feb 11 2016, 9:18 AM

• adrianheine raised the priority of this task from to Medium.

• adrianheine updated the task description. (Show Details)

• adrianheine added projects: Browser-Tests, Wikidata.

• adrianheine subscribed.

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 11 2016, 9:18 AM

• adrianheine renamed this task from Sitelink browser test fails with firefox to Sitelink browser test sometimes fails with firefox.Feb 11 2016, 9:24 AM

• adrianheine updated the task description. (Show Details)

• adrianheine set Security to None.

This is a rate limiting problem. Whenever the test fails, I have the following in my debug log:

[ratelimit] User '0:0:0:0:0:0:0:1' (IP ::1) tripped my_wiki:limiter:edit:ip:%3A%3A1 at 8 (limit 8 in 60s)

Probably something relating to https://www.mediawiki.org/wiki/Manual:$wgRateLimits ?

@Tobi_WMDE_SW @Addshore @JanZerebecki Rate limiting settings for beta need to be adapted, either by IP or in general.

Probably @aude can help with this?

I think https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/InitialiseSettings-labs.php#L555-L557 is where this can be set.

https://support.saucelabs.com/customer/portal/questions/13054963-whitelist-ip-or-ip-range-from-saucelabs tells us their ip range, current of at least December.

do we need more than saucelabs ip range for this? e.g. our office IP range if we run these also from the office?

Change 269944 had a related patch set uploaded (by Aude):
Exclude Sauce Labs IP ranges from rate limits on beta

https://gerrit.wikimedia.org/r/269944

gerritbot added a project: Patch-For-Review.Feb 11 2016, 11:04 AM

Change 269944 merged by jenkins-bot:
Exclude Sauce Labs IP ranges from rate limits on beta

https://gerrit.wikimedia.org/r/269944

Change deployed, please verify.

Not fixed: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-WikidataTests-linux-firefox-sauce/521/ :/

The failures seem unrelated to this. It seems that the some of the articles used for testing do not really exist on the corresponding test wikis. Either we need to change the test data or ideally make the test cases more robust.

Locally this failed because of rate limiting and stopped failing after I changed config. I'll try again locally.

Lydia_Pintscher moved this task from incoming to consider for next sprint on the Wikidata board.Feb 18 2016, 12:50 PM

I checked again: This works locally.

JanZerebecki renamed this task from Sitelink browser test sometimes fails with firefox to Sitelink browser test sometimes fails with firefox because of rate limit?.May 7 2016, 8:41 PM

JanZerebecki removed a project: Patch-For-Review.

JanZerebecki added a project: Wikidata-Sprint-2016-05-10.May 10 2016, 6:33 PM

JanZerebecki moved this task from Proposed to Backlog on the Wikidata-Sprint-2016-05-10 board.

Lydia_Pintscher added a project: Wikidata-Sprint-2016-05-24.May 24 2016, 12:48 PM

Addshore moved this task from Proposed to Backlog on the Wikidata-Sprint-2016-05-24 board.May 24 2016, 12:48 PM

Looking at the actual screenshots on Saucelabs, e.g. https://assets.saucelabs.com/jobs/e6cd0f782ac54c04bfaa75923bff8f00/0022screenshot.png suggested that it is a problem with the corresponding article on beta Wikipedia. I've just checked that, and indeed the article "Test" was not existing on German beta Wikipedia. I've re-created it, and I assume this test failure is fixed now.
However, as I've stated before, we need to make the test more robust, e.g. by ensuring the used test data does actually exist. We could do this in an extra Given-step.

Nice catch. When I tried it locally it probably ran against enwp, not beta.

• adrianheine moved this task from Backlog to Done on the Wikidata-Sprint-2016-05-24 board.Jun 14 2016, 7:19 AM

Sitelink browser test sometimes fails with firefox because of rate limit?Closed, ResolvedPublicActions

Description

Details

Related ObjectsSearch...

Event Timeline

Sitelink browser test sometimes fails with firefox because of rate limit?
Closed, ResolvedPublic
Actions

Related Objects
Search...