Page MenuHomePhabricator

mapper_parsing_exception: Unknown Similarity type [default] for field [title]
Closed, ResolvedPublic

Description

I am trying CirrusSearch (es5 branch) with ElasticSearch 5.x and creating mappings fails with an error:

Validating mappings...
Validating mapping...different...failed!
Couldn't update existing mappings. You may need to reindex.
Here is elasticsearch's error message: mapper_parsing_exception: Unknown Similarity type [default] for field [title]

Appears that 'default' is no longer a valid similarity:

https://github.com/elastic/elasticsearch/commit/81fd2169cf9f394c7be2a853caae5a54c0c75b18

'default' has been renamed to 'classic'

Alternatively, if we want whatever really is the default for elasticsearch (BM25?) then maybe it would be used in cases where similarity is not expicitly defined in the mapping? (e.g. it's omitted)

https://www.elastic.co/guide/en/elasticsearch/reference/current/similarity.html

Or maybe we can add some default similarity in $wgCirrusSearchSimilarityProfiles instead of 'default' => []?

Looks like most Wikipedias use BM25 now (wmf_defaults), except some (e.g. Japanese) that still use 'default'

Event Timeline

aude created this task.Mar 12 2017, 8:48 PM
Restricted Application added projects: Discovery, Discovery-Search. · View Herald TranscriptMar 12 2017, 8:48 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
aude updated the task description. (Show Details)Mar 12 2017, 8:50 PM
aude updated the task description. (Show Details)Mar 12 2017, 8:57 PM
aude updated the task description. (Show Details)Mar 12 2017, 9:04 PM
EduRam added a subscriber: EduRam.Mar 24 2017, 4:28 PM

I have the same problem while configuring a wiki (1.28).
Did you manage to resolve it ?

Deskana closed this task as Resolved.Mar 30 2017, 5:16 PM
Deskana claimed this task.
Deskana added a subscriber: Deskana.

In general, this shouldn't happen with the latest upgrade to Elasticsearch 5 (which happened a little while after this task was filed), so this should work just fine if everything is updated to master. Please reopen if this problem persists.

@EduRam 1.28 does not support Elasticsearch 5. It is odd that you got this far into the installation process though; you should've seen an error much sooner. I would suggest updating everything to the latest versions, which should fix your problem.

EduRam added a comment.Apr 1 2017, 3:46 PM

Hi! Thanks for the reply. I upgraded mw + extensions to "wmf/1.29.0-wmf.18" (via git) ... and then updated db.

Unfortunally I somehow managed to have the error again.

[e@wiki2 maintenance]$ php updateSearchIndexConfig.php --reindexAndRemoveOk --indexIdentifier=now

PHP Notice:  Undefined index: SERVER_NAME in /var/www/html/includes/GlobalFunctions.php on line 1477
PHP Notice:  Undefined index: SERVER_NAME in /var/www/html/includes/GlobalFunctions.php on line 1477
content index...
        Fetching Elasticsearch version...5.1.2...ok
        Scanning available plugins...none
        Setting index identifier...wikidb-wiki__content_1491060934
        Picking analyzer...english
        Creating index...ok
                Validating number of shards...ok
                Validating replica range...ok
                Validating shard allocation settings...done
                Validating max shards per node...ok
        Validating analyzers...ok
        Validating mappings...
                Validating mapping...different...failed!
Couldn't update existing mappings. You may need to reindex.
Here is elasticsearch's error message: mapper_parsing_exception: Unknown Similarity type [default] for field [title]

Probably is something wrong on my side, but I decided to reply to your message after I upgraded to latest mw, just in case there could be further developments.

MediaWiki1.29.0-wmf.18
PHP7.1.2 (apache2handler
MariaDB5.5.52-MariaDB
ICU50.1.2
ElasticSearch5.1.2
dcausse reopened this task as Open.Apr 11 2017, 4:49 PM
dcausse added a subscriber: dcausse.

Reopening as there's probably something going wrong here and Trey had the same issue.

Change 347824 had a related patch set uploaded (by DCausse):
[mediawiki/extensions/CirrusSearch@master] Fix problem when using the default similarity

https://gerrit.wikimedia.org/r/347824

dcausse claimed this task.Apr 12 2017, 10:16 AM
dcausse moved this task from needs triage to Current work on the Discovery-Search board.
dcausse moved this task from in progress to Needs review on the Discovery-Search (Current work) board.

Change 347824 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Fix problem when using the default similarity

https://gerrit.wikimedia.org/r/347824

debt closed this task as Resolved.May 30 2017, 5:25 PM