PHP Warning: {"type":"error","message":"cirrussearch-too-busy-error","params":[]}
[Called from Wikibase\Search\Elastic\EntitySearchElastic::getRankedSearchResults
in /srv/mediawiki/php-1.34.0-wmf.22/extensions/WikibaseCirrusSearch/src/EntitySearchElastic.php at line 318]I guess the ElasticSearch cluster is too busy?
That started before I promoted wikidatawiki to 1.34.0-wmf.22
#0 [internal function]: MWExceptionHandler::handleError(integer, string, string, integer, array)
#1 /srv/mediawiki/php-1.34.0-wmf.22/includes/debug/MWDebug.php(333): trigger_error(string, integer)
#2 /srv/mediawiki/php-1.34.0-wmf.22/includes/debug/MWDebug.php(188): MWDebug::sendMessage(string, array, string, integer)
#3 /srv/mediawiki/php-1.34.0-wmf.22/includes/GlobalFunctions.php(1079): MWDebug::warning(string, integer, integer, string)
#4 /srv/mediawiki/php-1.34.0-wmf.22/extensions/WikibaseCirrusSearch/src/EntitySearchElastic.php(318): wfLogWarning(string)
#5 /srv/mediawiki/php-1.34.0-wmf.22/extensions/Wikibase/repo/includes/Api/CombinedEntitySearchHelper.php(52): Wikibase\Search\Elastic\EntitySearchElastic->getRankedSearchResults(string, string, string, integer, boolean)
#6 /srv/mediawiki/php-1.34.0-wmf.22/extensions/Wikibase/repo/includes/Api/TypeDispatchingEntitySearchHelper.php(55): Wikibase\Repo\Api\CombinedEntitySearchHelper->getRankedSearchResults(string, string, string, integer, boolean)
#7 /srv/mediawiki/php-1.34.0-wmf.22/extensions/Wikibase/repo/includes/Api/SearchEntities.php(120): Wikibase\Repo\Api\TypeDispatchingEntitySearchHelper->getRankedSearchResults(string, string, string, integer, boolean)
#8 /srv/mediawiki/php-1.34.0-wmf.22/extensions/Wikibase/repo/includes/Api/SearchEntities.php(254): Wikibase\Repo\Api\SearchEntities->getSearchEntries(array)
#9 /srv/mediawiki/php-1.34.0-wmf.22/includes/api/ApiMain.php(1599): Wikibase\Repo\Api\SearchEntities->execute()
#10 /srv/mediawiki/php-1.34.0-wmf.22/includes/api/ApiMain.php(537): ApiMain->executeAction()
#11 /srv/mediawiki/php-1.34.0-wmf.22/includes/api/ApiMain.php(508): ApiMain->executeActionWithErrorHandling()
#12 /srv/mediawiki/php-1.34.0-wmf.22/api.php(87): ApiMain->execute()
#13 /srv/mediawiki/w/api.php(3): require(string)
#14 {main}| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | PRODUCTION ERROR | Gehel | T232612 WikibaseCirrusSearch emits cirrussearch-too-busy-error errors | ||
| Declined | None | T233351 Increase poolcounter for wikibase prefix search | |||
| Open | None | T233352 Better error handling for pool counter errors on cirrus search |
Most of the requests I see in logstash have &limit=5 in the URL, which is not usually set by the Wikibase editing UI as far as I can tell. I wonder if some large third party has started to hit wbsearchentities with their users’ searches? I would not expect this search term, for example, to come from a Wikidata editor.
The API usage of wbsearchentities is higher than usual (https://graphite.wikimedia.org/S/h). This causes the poolcounter to start rejecting queries.
I suppose we could also handle the error path a bit better :
Reading the code I see:
// FIXME: this is a hack, we need to return Status upstream instead
foreach ( $result->getErrors() as $error ) {
wfLogWarning( json_encode( $error ) );
}Should we pass an exception upstream so that we have a chance to warn the user that something is not right?
The errors themselves are expected behaviour. Subtasks have been created to track improvements we can do.