Page MenuHomePhabricator

Prefixsearch requests time out on mobile testwiki
Closed, ResolvedPublic

Description

https://test.m.wikipedia.org/w/api.php?action=query&format=json&generator=prefixsearch&redirects=true&gpssearch=a&gpsnamespace=0&gpslimit=20&list=search&srsearch=a&srnamespace=0&srwhat=text&srinfo=suggestion&srprop=&sroffset=0&srlimit=1&prop=pageterms%7Cpageimages&wbptterms=description&piprop=thumbnail&pithumbsize=320&pilimit=20&continue=

yields PHP Warning: unable to connect to 10.64.0.179 [110]: Connection timed out coming from CirrusSearch/includes/Util.php#147. Full error:

2015-09-16 23:15:40 mw1017 testwiki error-json ERROR:
{"id":"5f67d025","type":"ErrorException","file":"/srv/mediawiki/php-1.26wmf23/extensions/PoolCounter/PoolCounterClient_body.php","line":71,"message":"PHP Warning: unable to connect to 10.64.0.179 [110]: Connection timed out","code":0,"url":"/w/api.php?action=query&format=json&generator=prefixsearch&redirects=true&gpssearch=a&gpsnamespace=0&gpslimit=20&list=search&srsearch=a&srnamespace=0&srwhat=text&srinfo=suggestion&srprop=&sroffset=0&srlimit=1&prop=pageterms%7Cpageimages&wbptterms=description&piprop=thumbnail&pithumbsize=320&pilimit=20&continue=","suppressed":true,"backtrace":[{"file":"/srv/mediawiki/php-1.26wmf23/includes/Hooks.php","line":253,"function":"handleError","class":"MWExceptionHandler","type":"::","args":["integer","string","string","integer","array","array"]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/PoolCounter/PoolCounterClient_body.php","line":71,"function":"hookErrorHandler","class":"Hooks","type":"::","args":["integer","string","string","integer","array","array"]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/PoolCounter/PoolCounterClient_body.php","line":39,"function":"open","class":"PoolCounter_ConnectionManager","type":"->","args":["string","integer","integer","string"]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/PoolCounter/PoolCounterClient_body.php","line":123,"function":"get","class":"PoolCounter_ConnectionManager","type":"->","args":["string"]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/PoolCounter/PoolCounterClient_body.php","line":143,"function":"getConn","class":"PoolCounter_Client","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/PoolCounter/PoolCounterClient_body.php","line":190,"function":"sendCommand","class":"PoolCounter_Client","type":"->","args":["string","string","integer","integer","integer"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/poolcounter/PoolCounterWork.php","line":108,"function":"acquireForMe","class":"PoolCounter_Client","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/CirrusSearch/includes/Util.php","line":147,"function":"execute","class":"PoolCounterWork","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/poolcounter/PoolCounterWorkViaCallback.php","line":69,"function":"Closure$CirrusSearch\\Util::doPoolCounterWork#4","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/poolcounter/PoolCounterWork.php","line":123,"function":"doWork","class":"PoolCounterWorkViaCallback","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/CirrusSearch/includes/Util.php","line":161,"function":"execute","class":"PoolCounterWork","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/CirrusSearch/includes/Searcher.php","line":1473,"function":"doPoolCounterWork","class":"CirrusSearch\\Util","type":"::","args":["string","User","Closure$CirrusSearch\\Searcher::search#2;1334410063","Closure$CirrusSearch\\Searcher::search#3;1334410063"]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/CirrusSearch/includes/Searcher.php","line":349,"function":"search","class":"CirrusSearch\\Searcher","type":"->","args":["string","string"]},{"file":"/srv/mediawiki/php-1.26wmf23/extensions/CirrusSearch/includes/Hooks.php","line":696,"function":"prefixSearch","class":"CirrusSearch\\Searcher","type":"->","args":["string"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/Hooks.php","line":204,"function":"prefixSearch","class":"CirrusSearch\\Hooks","type":"::","args":["array","string","integer","array","integer"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/PrefixSearch.php","line":161,"function":"run","class":"Hooks","type":"::","args":["string","array"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/PrefixSearch.php","line":70,"function":"searchBackend","class":"PrefixSearch","type":"->","args":["array","string","integer","integer"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/PrefixSearch.php","line":98,"function":"search","class":"PrefixSearch","type":"->","args":["string","integer","array","integer"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/api/ApiQueryPrefixSearch.php","line":49,"function":"searchWithVariants","class":"PrefixSearch","type":"->","args":["string","integer","array","integer"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/api/ApiQueryPrefixSearch.php","line":35,"function":"run","class":"ApiQueryPrefixSearch","type":"->","args":["ApiPageSet"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/api/ApiPageSet.php","line":172,"function":"executeGenerator","class":"ApiQueryPrefixSearch","type":"->","args":["ApiPageSet"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/api/ApiPageSet.php","line":136,"function":"executeInternal","class":"ApiPageSet","type":"->","args":["boolean"]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/api/ApiQuery.php","line":271,"function":"execute","class":"ApiPageSet","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/api/ApiMain.php","line":1270,"function":"execute","class":"ApiQuery","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/api/ApiMain.php","line":466,"function":"executeAction","class":"ApiMain","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/includes/api/ApiMain.php","line":438,"function":"executeActionWithErrorHandling","class":"ApiMain","type":"->","args":[]},{"file":"/srv/mediawiki/php-1.26wmf23/api.php","line":88,"function":"execute","class":"ApiMain","type":"->","args":[]},{"file":"/srv/mediawiki/w/api.php","line":3,"function":"include","args":["string"]}]}

Event Timeline

Tgr raised the priority of this task from to Needs Triage.
Tgr updated the task description. (Show Details)
Tgr added a project: Discovery-ARCHIVED.
Tgr added subscribers: Tgr, Niedzielski, Jdlrobson.

@EBernhardson @dcausse Is this just flakiness of our test infrastructure, or do you think this is indicative of a larger issue?

I think it's related to the test infrastructure, I asked on ops channel and they told me that poolcounter was added in test just yesterday so someone is probably aware of the issue?

@Deskana I am running an experiment on testwiki, recorded yesterday in the SAL; I purposedly dropped packets to one of the poolcounter servers in order to test the fast-reconnect feature for T105378

Since this is the first real-world error that we see, I'll stop the experiment. anyways it's a bit odd that it would fail just because one of the two poolcounter servers is unreachable, I'll inspect the code.

Deskana claimed this task.

I'm going to assume that this issue is now resolved. Please reopen with more information if that is not the case.