Page MenuHomePhabricator

CompletionSuggester produces IndexCreationException in CirrusSearch
Closed, ResolvedPublic

Description

When I run extensions/CirrusSearch/maintenance/updateSuggesterIndex.php for the first time under MediaWiki 1.27.1, I get this error (which includes the sentence, "This is always a bug in CirrusSearch." hence this ticket).

$ php updateSuggesterIndex.php
Scanning available plugins...
        kopf
Picking analyzer...english
Fetching Elasticsearch version...1.7.4...ok
Inferring index identifier...wikidb-vpw__titlesuggest_first
Index does not exist yet cannot recycle.
Inferring index identifier...wikidb-vpw__titlesuggest_first
Setting index identifier...wikidb-vpw__titlesuggest_1479240582
2016-11-15 15:09:42 
Unexpected Elasticsearch failure.
Elasticsearch failed in an unexpected way.  This is always a bug in CirrusSearch.
Error type: Elastica\Exception\ResponseException
Message: IndexCreationException[[wikidb-vpw__titlesuggest_1479240582] failed to create index]; nested: ElasticsearchIllegalArgumentException[failed to find token filter type [icu_normalizer] for [icu_normalizer]]; nested: NoClassSettingsException[Failed to load class setting [type] with value [icu_normalizer]]; nested: ClassNotFoundException[org.elasticsearch.index.analysis.icunormalizer.IcuNormalizerTokenFilterFactory]; 
Trace:
#0 /home/wiki/wiki/wiki/vendor/ruflin/elastica/lib/Elastica/Request.php(171): Elastica\Transport\Http->exec(Object(Elastica\Request), Array)
#1 /home/wiki/wiki/wiki/vendor/ruflin/elastica/lib/Elastica/Client.php(621): Elastica\Request->send()
#2 /home/wiki/wiki/wiki/vendor/ruflin/elastica/lib/Elastica/Index.php(496): Elastica\Client->request('wikidb-vpw__tit...', 'PUT', Array, Array)
#3 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(706): Elastica\Index->request('wikidb-vpw__tit...', 'PUT', Array, Array)
#4 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(301): CirrusSearch\Maintenance\UpdateSuggesterIndex->createIndex()
#5 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(231): CirrusSearch\Maintenance\UpdateSuggesterIndex->rebuild()
#6 /home/wiki/wiki/wiki/maintenance/doMaintenance.php(103): CirrusSearch\Maintenance\UpdateSuggesterIndex->execute()
#7 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(809): require_once('/home/wiki/wiki...')
#8 {main}

I ran composer update but it didn't make a difference.

Event Timeline

Restricted Application added projects: Discovery, Discovery-Search. · View Herald TranscriptNov 15 2016, 8:50 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

This looks to be attempting to use the icu plugin[1], but it's not installed to your cluster. Can you verify it's not available? We should be auto-detecting that and only using icu when available but perhaps we got something wrong that needs to be fixed.

[1] https://www.elastic.co/guide/en/elasticsearch/plugins/2.4/analysis-icu.html

Here you go:

$ /usr/share/elasticsearch/bin/plugin --list
Installed plugins:
    - experimental-highlighter-elasticsearch-plugin
    - kopf

@maiden_taiwan sorry about that, yes it's a bug in the code that creates the completion suggester index.
It's fixed in https://gerrit.wikimedia.org/r/#/c/310359 but this patch is not merged yet and I'm not sure that we will backport it to 1.27.
The workaround would be to install analysis-icu plugin on your elasticsearch cluster.

Change 310359 had a related patch set uploaded (by DCausse):
Add support for ICU folding

https://gerrit.wikimedia.org/r/310359

dcausse claimed this task.
dcausse triaged this task as Normal priority.
dcausse moved this task from in progress to Needs review on the Discovery-Search (Current work) board.

Thank you very much. Is it possible to install ICU on our old version of elasticsearch (1.7.4)? From looking at the docs, I see ICU only for versions 2.0 and up.

Change 310359 merged by jenkins-bot:
Add support for ICU folding

https://gerrit.wikimedia.org/r/310359

Deskana closed this task as Resolved.Dec 9 2016, 3:25 PM
Deskana added a project: Discovery.