Seen in fatalmonitor
80 Undefined index: _score in /srv/mediawiki/php-1.31.0-wmf.7/extensions/CirrusSearch/in
cludes/Query/CompSuggestQueryBuilder.php on line 224
80 Undefined index: _id in /srv/mediawiki/php-1.31.0-wmf.7/extensions/CirrusSearch/inclu
des/Query/CompSuggestQueryBuilder.php on line 223It was just a brief spike of 300 errors today around 15:00 UTC, looking at the code I see no obvious reasons why it could happen, except a broken response from elastic.
I'll assume that this error was due to the rolling restart and won't try to hide it by calling isset on the result sets.
Still seen on mediawiki-errors in Logstash: The second and third most frequent PHP error from MediaWiki, in the last 7 days.
| 2 | [{exception_id}] {exception_url} ErrorException from line 226 of /srv/mediawiki/php-1.32.0-wmf.12/extensions/CirrusSearch/includes/Query/CompSuggestQueryBuilder.php: PHP Notice: Undefined index: _id | 8,860 hits |
|---|---|---|
| 3 | [{exception_id}] {exception_url} ErrorException from line 227 of /srv/mediawiki/php-1.32.0-wmf.12/extensions/CirrusSearch/includes/Query/CompSuggestQueryBuilder.php: PHP Notice: Undefined index: _score | 8,280 hits |
Change 447567 had a related patch set uploaded (by EBernhardson; owner: EBernhardson):
[operations/puppet@production] Split per-cluster config out of elasticsearch::curator
Change 447567 merged by Gehel:
[operations/puppet@production] Split per-cluster config out of elasticsearch::curator
Found a possible reason for this to happen and filed a bug upstream: https://github.com/elastic/elasticsearch/issues/32467
In short the completion suggester code does not handle gracefully shard failures during the fetch phase causing the response received by cirrus to not contain the necessary information.
I'll add a workaround in Cirrus with a note to this bug upstream.
Change 449200 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/CirrusSearch@master] Workaround bad response returned by the completion suggester
Change 449200 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Workaround bad response returned by the completion suggester