Page MenuHomePhabricator

CompletionSuggester produces IndexCreationException in CirrusSearch
Closed, ResolvedPublic

Description

When I run extensions/CirrusSearch/maintenance/updateSuggesterIndex.php for the first time under MediaWiki 1.27.1, I get this error (which includes the sentence, "This is always a bug in CirrusSearch." hence this ticket).

$ php updateSuggesterIndex.php
Scanning available plugins...
        kopf
Picking analyzer...english
Fetching Elasticsearch version...1.7.4...ok
Inferring index identifier...wikidb-vpw__titlesuggest_first
Index does not exist yet cannot recycle.
Inferring index identifier...wikidb-vpw__titlesuggest_first
Setting index identifier...wikidb-vpw__titlesuggest_1479240582
2016-11-15 15:09:42 
Unexpected Elasticsearch failure.
Elasticsearch failed in an unexpected way.  This is always a bug in CirrusSearch.
Error type: Elastica\Exception\ResponseException
Message: IndexCreationException[[wikidb-vpw__titlesuggest_1479240582] failed to create index]; nested: ElasticsearchIllegalArgumentException[failed to find token filter type [icu_normalizer] for [icu_normalizer]]; nested: NoClassSettingsException[Failed to load class setting [type] with value [icu_normalizer]]; nested: ClassNotFoundException[org.elasticsearch.index.analysis.icunormalizer.IcuNormalizerTokenFilterFactory]; 
Trace:
#0 /home/wiki/wiki/wiki/vendor/ruflin/elastica/lib/Elastica/Request.php(171): Elastica\Transport\Http->exec(Object(Elastica\Request), Array)
#1 /home/wiki/wiki/wiki/vendor/ruflin/elastica/lib/Elastica/Client.php(621): Elastica\Request->send()
#2 /home/wiki/wiki/wiki/vendor/ruflin/elastica/lib/Elastica/Index.php(496): Elastica\Client->request('wikidb-vpw__tit...', 'PUT', Array, Array)
#3 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(706): Elastica\Index->request('wikidb-vpw__tit...', 'PUT', Array, Array)
#4 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(301): CirrusSearch\Maintenance\UpdateSuggesterIndex->createIndex()
#5 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(231): CirrusSearch\Maintenance\UpdateSuggesterIndex->rebuild()
#6 /home/wiki/wiki/wiki/maintenance/doMaintenance.php(103): CirrusSearch\Maintenance\UpdateSuggesterIndex->execute()
#7 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(809): require_once('/home/wiki/wiki...')
#8 {main}

I ran composer update but it didn't make a difference.

Event Timeline

Restricted Application added projects: Discovery, Discovery-Search. · View Herald TranscriptNov 15 2016, 8:50 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

This looks to be attempting to use the icu plugin[1], but it's not installed to your cluster. Can you verify it's not available? We should be auto-detecting that and only using icu when available but perhaps we got something wrong that needs to be fixed.

[1] https://www.elastic.co/guide/en/elasticsearch/plugins/2.4/analysis-icu.html

Here you go:

$ /usr/share/elasticsearch/bin/plugin --list
Installed plugins:
    - experimental-highlighter-elasticsearch-plugin
    - kopf

@maiden_taiwan sorry about that, yes it's a bug in the code that creates the completion suggester index.
It's fixed in https://gerrit.wikimedia.org/r/#/c/310359 but this patch is not merged yet and I'm not sure that we will backport it to 1.27.
The workaround would be to install analysis-icu plugin on your elasticsearch cluster.

Change 310359 had a related patch set uploaded (by DCausse):
Add support for ICU folding

https://gerrit.wikimedia.org/r/310359

dcausse claimed this task.Nov 16 2016, 10:35 AM
dcausse triaged this task as Normal priority.
dcausse moved this task from Uncategorised to Technical on the CirrusSearch board.
dcausse moved this task from in progress to Needs review on the Discovery-Search (Current work) board.

Thank you very much. Is it possible to install ICU on our old version of elasticsearch (1.7.4)? From looking at the docs, I see ICU only for versions 2.0 and up.

Change 310359 merged by jenkins-bot:
Add support for ICU folding

https://gerrit.wikimedia.org/r/310359

Deskana closed this task as Resolved.Dec 9 2016, 3:25 PM
Deskana added a project: Discovery.