CompletionSuggester produces IndexCreationException in CirrusSearch
Closed, ResolvedPublic
Actions

Description

When I run extensions/CirrusSearch/maintenance/updateSuggesterIndex.php for the first time under MediaWiki 1.27.1, I get this error (which includes the sentence, "This is always a bug in CirrusSearch." hence this ticket).

$ php updateSuggesterIndex.php
Scanning available plugins...
        kopf
Picking analyzer...english
Fetching Elasticsearch version...1.7.4...ok
Inferring index identifier...wikidb-vpw__titlesuggest_first
Index does not exist yet cannot recycle.
Inferring index identifier...wikidb-vpw__titlesuggest_first
Setting index identifier...wikidb-vpw__titlesuggest_1479240582
2016-11-15 15:09:42 
Unexpected Elasticsearch failure.
Elasticsearch failed in an unexpected way.  This is always a bug in CirrusSearch.
Error type: Elastica\Exception\ResponseException
Message: IndexCreationException[[wikidb-vpw__titlesuggest_1479240582] failed to create index]; nested: ElasticsearchIllegalArgumentException[failed to find token filter type [icu_normalizer] for [icu_normalizer]]; nested: NoClassSettingsException[Failed to load class setting [type] with value [icu_normalizer]]; nested: ClassNotFoundException[org.elasticsearch.index.analysis.icunormalizer.IcuNormalizerTokenFilterFactory]; 
Trace:
#0 /home/wiki/wiki/wiki/vendor/ruflin/elastica/lib/Elastica/Request.php(171): Elastica\Transport\Http->exec(Object(Elastica\Request), Array)
#1 /home/wiki/wiki/wiki/vendor/ruflin/elastica/lib/Elastica/Client.php(621): Elastica\Request->send()
#2 /home/wiki/wiki/wiki/vendor/ruflin/elastica/lib/Elastica/Index.php(496): Elastica\Client->request('wikidb-vpw__tit...', 'PUT', Array, Array)
#3 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(706): Elastica\Index->request('wikidb-vpw__tit...', 'PUT', Array, Array)
#4 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(301): CirrusSearch\Maintenance\UpdateSuggesterIndex->createIndex()
#5 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(231): CirrusSearch\Maintenance\UpdateSuggesterIndex->rebuild()
#6 /home/wiki/wiki/wiki/maintenance/doMaintenance.php(103): CirrusSearch\Maintenance\UpdateSuggesterIndex->execute()
#7 /home/wiki/wiki/wiki/extensions/CirrusSearch/maintenance/updateSuggesterIndex.php(809): require_once('/home/wiki/wiki...')
#8 {main}

I ran composer update but it didn't make a difference.

Details

	Subject	Repo	Branch	Lines +/-
	Add support for ICU folding	mediawiki/extensions/CirrusSearch	master	+417 -20

Customize query in gerrit

Related Objects

Duplicates Merged Here: T158805: Traceback when running updateSuggesterIndex.php

Event Timeline

maiden_taiwan created this task.Nov 15 2016, 8:50 PM

Restricted Application added projects: Discovery-ARCHIVED, Discovery-Search. · View Herald TranscriptNov 15 2016, 8:50 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

This looks to be attempting to use the icu plugin[1], but it's not installed to your cluster. Can you verify it's not available? We should be auto-detecting that and only using icu when available but perhaps we got something wrong that needs to be fixed.

[1] https://www.elastic.co/guide/en/elasticsearch/plugins/2.4/analysis-icu.html

Here you go:

$ /usr/share/elasticsearch/bin/plugin --list
Installed plugins:
    - experimental-highlighter-elasticsearch-plugin
    - kopf

@maiden_taiwan sorry about that, yes it's a bug in the code that creates the completion suggester index.
It's fixed in https://gerrit.wikimedia.org/r/#/c/310359 but this patch is not merged yet and I'm not sure that we will backport it to 1.27.
The workaround would be to install analysis-icu plugin on your elasticsearch cluster.

Change 310359 had a related patch set uploaded (by DCausse):
Add support for ICU folding

https://gerrit.wikimedia.org/r/310359

gerritbot added a project: Patch-For-Review.Nov 16 2016, 10:32 AM

dcausse claimed this task.Nov 16 2016, 10:35 AM

dcausse triaged this task as Medium priority.

dcausse moved this task from Inbox to Technical on the CirrusSearch board.

dcausse edited projects, added Discovery-Search (Current work); removed Discovery-Search, Discovery-ARCHIVED.

dcausse moved this task from Incoming to Needs review on the Discovery-Search (Current work) board.

Thank you very much. Is it possible to install ICU on our old version of elasticsearch (1.7.4)? From looking at the docs, I see ICU only for versions 2.0 and up.

Should be available, see https://www.elastic.co/guide/en/elasticsearch/reference/1.7/analysis-icu-plugin.html

If the plugin script isn't finding the jars, the version for 1.7.x is http://search.maven.org/#artifactdetails%7Corg.elasticsearch%7Celasticsearch-analysis-icu%7C2.7.0%7Cjar

Change 310359 merged by jenkins-bot:
Add support for ICU folding

https://gerrit.wikimedia.org/r/310359

ReleaseTaggerBot added a project: MW-1.29-release (WMF-deploy-2016-12-13_(1.29.0-wmf.6)).Dec 6 2016, 10:00 PM

Smalyshev moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Dec 8 2016, 10:18 PM

• Deskana closed this task as Resolved.Dec 9 2016, 3:25 PM

• Deskana added a project: Discovery-ARCHIVED.

dcausse merged a task: T158805: Traceback when running updateSuggesterIndex.php.Feb 22 2017, 8:40 PM

dcausse added a subscriber: Eil.