Page MenuHomePhabricator

Add Nori (Korean) configuration to AnalysisConfigBuilder
Closed, ResolvedPublic

Description

The Nori Korean analyzer looks good to go, but since it is only available with ES 6.4.2+, we can't build it out yet.

  • Implement the configs in AnalysisConfigBuilder and add tests.
    • The command line version of the Elasticsearch config I used for testing so far is on MediaWiki.
  • Determine whether we need to change the config for plain field and the completion suggester.

For future reference: when it comes time to re-index, we will need to figure out how re-indexing Korean with a very different analyzer interacts with LTR.

Event Timeline

TJones created this task.Oct 12 2018, 6:52 PM
TJones triaged this task as Normal priority.
Restricted Application added a subscriber: revi. · View Herald TranscriptOct 12 2018, 6:52 PM
Ryuch added a subscriber: Ryuch.Oct 25 2018, 11:04 AM

Change 486266 had a related patch set uploaded (by DCausse; owner: DCausse):
[operations/software/elasticsearch/plugins@master] [WIP] Add nori korean analyzer

https://gerrit.wikimedia.org/r/486266

Change 486266 abandoned by DCausse:
[WIP] Add nori korean analyzer

Reason:
test build only

https://gerrit.wikimedia.org/r/486266

Change 491786 had a related patch set uploaded (by Tjones; owner: Tjones):
[mediawiki/extensions/CirrusSearch@es6] Configure Nori Korean analyzer for ES6

https://gerrit.wikimedia.org/r/491786

Change 491786 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@es6] Configure Nori Korean analyzer for ES6

https://gerrit.wikimedia.org/r/491786

We need to reindex, but not until after the ES6 upgrade is complete, and LTR has been disabled.

debt closed this task as Resolved.Feb 22 2019, 8:34 PM
debt added a subscriber: debt.

Thanks for adding this to the ongoing index ticket (T147505), @TJones !