Page MenuHomePhabricator

Timeouts when trying to create mappings.
Closed, ResolvedPublic

Description

There is a default master_timeout of 30s, we are running over that at time preventing index creation. This has happened both when creating the indexes for a new wiki and again today creating indexes for the titlesuggest feature. We can override the timeout by setting master_timeout in the query string, this is not supported by the Elastica client library we use in all instances.

Ideally we could fix this by making the cluster respond in < 30s, but i fear we might need to make the timeout adjustable until then.

Event Timeline

EBernhardson updated the task description. (Show Details)
EBernhardson raised the priority of this task from to Needs Triage.
EBernhardson added a project: CirrusSearch.
EBernhardson added a subscriber: EBernhardson.
Restricted Application added a project: Discovery. · View Herald TranscriptJul 29 2015, 9:23 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Ironholds moved this task from Needs triage to Search on the Discovery board.Aug 4 2015, 8:16 AM

This requires adding adding master_timeout=60s to the query string which is not directly supported by the
Mapping class we use. Going to require minor hacks all around for temporary solution, or coming up with a higher level way of attaching master_timeout to the right kinds of requests.

EBernhardson renamed this task from Add an option to set timeouts from cli in updateSearchIndexConfig.php to Timeouts when trying to create mappings..Sep 1 2015, 12:27 AM
EBernhardson updated the task description. (Show Details)
EBernhardson set Security to None.

Change 258017 had a related patch set uploaded (by EBernhardson):
Add $wgCirrusSearchMasterTimeout to deal with master slowdowns

https://gerrit.wikimedia.org/r/258017

Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptDec 9 2015, 11:36 PM

I've also submitted a fix to the Elastica library which will let us change the master timeout in a less hacky way. This has been merged and will be in the next release: https://github.com/ruflin/Elastica/pull/1004

Change 258017 merged by jenkins-bot:
Add $wgCirrusSearchMasterTimeout to deal with master slowdowns

https://gerrit.wikimedia.org/r/258017

Change 258088 had a related patch set uploaded (by EBernhardson):
Add $wgCirrusSearchMasterTimeout to deal with master slowdowns

https://gerrit.wikimedia.org/r/258088

Change 258089 had a related patch set uploaded (by EBernhardson):
Add $wgCirrusSearchMasterTimeout to deal with master slowdowns

https://gerrit.wikimedia.org/r/258089

Change 258088 merged by jenkins-bot:
Add $wgCirrusSearchMasterTimeout to deal with master slowdowns

https://gerrit.wikimedia.org/r/258088

Change 258089 merged by jenkins-bot:
Add $wgCirrusSearchMasterTimeout to deal with master slowdowns

https://gerrit.wikimedia.org/r/258089

Change 258095 had a related patch set uploaded (by EBernhardson):
Utilize the master timeout for creation and deletion as well

https://gerrit.wikimedia.org/r/258095

Change 258095 merged by jenkins-bot:
Utilize the master timeout for creation and deletion as well

https://gerrit.wikimedia.org/r/258095

Deskana triaged this task as Normal priority.Dec 17 2015, 11:46 PM
Deskana moved this task from Search to On Sprint Board on the Discovery board.
Deskana added a subscriber: Deskana.
Deskana closed this task as Resolved.Dec 17 2015, 11:50 PM