Maniphest T192395

Create Croatian, Serbo-Croatian, and Bosnian Analysis Chains Using Serbian Morphological Libraries
Closed, ResolvedPublic
Actions

Description

It should be possible to deploy the Serbian analysis chain with the Serbian stemmer (or some variant) to the Serbo-Croatian (sh/442K Wikipedia articles), Croatian (hr/185K Wikipedia articles), and Bosnian (bs/77K Wikipedia articles) wikis. (See the English Wikipedia article on Serbo-Croatian; the standard varieties of the language differ, but have the same basic grammar, so the stemmer should do as good a job on them as on Serbian.)

The task here is to review the results of the analysis chain and make sure there aren't any surprises. Then, we can re-index wikis in these three languages.

Details

	Subject	Repo	Branch	Lines +/-
	Create BSC Analysis Chains using Serbian Stemmer	mediawiki/extensions/CirrusSearch	master	+832 -28

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Invalid	None	T174065 [FY 2017-18 Objective] Improve support for searching in multiple languages
Open	None	T154511 [Tracking] Research, test, and deploy new language analyzers
Resolved	TJones	T171652 Language Analysis Morphological Library Research Spike
Resolved	TJones	T178926 Review Serbian Morphological Libraries
Resolved	TJones	T183015 Create Serbian Elasticsearch Plugin/Analysis Chain Using Serbian Morphological Libraries
Resolved	TJones	T192395 Create Croatian, Serbo-Croatian, and Bosnian Analysis Chains Using Serbian Morphological Libraries
Resolved	TJones	T196658 Re-index Croatian, Serbo-Croatian, and Bosnian Wikis

Event Timeline

TJones triaged this task as Medium priority.Apr 17 2018, 6:29 PM

TJones created this task.

TJones moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.

Everything looks good to me with the Serbian analyzer and ICU folding enabled, but need speaker review. If everything looks good to them, then I'll deploy the analysis chain and then re-index the relevant wikis.

Full write up is on Mediawiki.

TJones moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Apr 24 2018, 5:23 PM

Waiting on speaker feedback.

zeljkofilipin added a project: User-zeljkofilipin.May 2 2018, 2:41 PM

zeljkofilipin moved this task from Backlog 🪒 to Q4 👔 on the User-zeljkofilipin board.

TJones mentioned this in T193734: Move Serbian language wikis from extra-analysis to extra-analysis-serbian plugin.May 3 2018, 1:28 PM

zeljkofilipin moved this task from Q4 👔 to Deep work 🌊 on the User-zeljkofilipin board.May 4 2018, 3:04 PM

zeljkofilipin moved this task from Deep work 🌊 to Waiting 🏹 on the User-zeljkofilipin board.May 8 2018, 2:34 PM

TJones moved this task from Waiting to not in use - please delete on the Discovery-Search (Current work) board.May 8 2018, 5:34 PM

Speaker feedback is positive, and we are good to go on that front. However, we're refactoring the plugin that provides the analysis, so we're going to finish that and get the Serbian wikis all sorted out (T193734) before enabling this for Croatian, Serbo-Croatian, and Bosnian.

And I just realized that we moved from waiting on speaker feedback to waiting on plugin refactoring, so back to Waiting status for now!

zeljkofilipin moved this task from Waiting 🏹 to Watching 📺 on the User-zeljkofilipin board.May 10 2018, 3:49 PM

TJones mentioned this in T171652: Language Analysis Morphological Library Research Spike.May 22 2018, 5:44 PM

TJones moved this task from Waiting to not in use - please delete on the Discovery-Search (Current work) board.Jun 5 2018, 3:10 PM

Restricted Application added a subscriber: • Petar.petkovic. · View Herald TranscriptJun 5 2018, 3:10 PM

Change 437510 had a related patch set uploaded (by Tjones; owner: Tjones):
[mediawiki/extensions/CirrusSearch@master] Create BSC Analysis Chains using Serbian Stemmer

https://gerrit.wikimedia.org/r/437510

gerritbot added a project: Patch-For-Review.Jun 5 2018, 5:06 PM

TJones moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Jun 5 2018, 5:18 PM

Change 437510 merged by jenkins-bot:
[mediawiki/extensions/CirrusSearch@master] Create BSC Analysis Chains using Serbian Stemmer

https://gerrit.wikimedia.org/r/437510

TJones moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Jun 7 2018, 5:08 PM

• Petar.petkovic removed a project: Patch-For-Review.Jun 7 2018, 5:19 PM

ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-06-12 (1.32.0-wmf.8)).Jun 7 2018, 6:00 PM

TJones mentioned this in T196780: Review Applying Indonesian Analysis Chain for Malay.Jun 8 2018, 8:23 PM

debt closed this task as Resolved.Jun 14 2018, 8:08 PM

debt closed subtask T196658: Re-index Croatian, Serbo-Croatian, and Bosnian Wikis as Resolved.Jul 9 2018, 11:57 PM