Update TextCat with wrong-keyboard models
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	TJones
	Jan 16 2019, 3:14 PM

Description

Add wrong-keyboard–transformed language models for "Cyrillic English" (en_cyr) and "Latin Russian" (ru_lat) to TextCat, both to the query-based (LM-query/) and wikitext-based (LM/) models. Also add Windows-1251 wrong-encoding model (ru_win1251) to the wikitext-based models.

Details

	Subject	Repo	Branch	Lines +/-
	Add Wrong-Keyboard and Wrong-Encoding Models to TextCat	wikimedia/textcat	master	+100 K -22

Customize query in gerrit

Related Objects
Search...

Status	Assigned	Task
Open	None	T118278 [EPIC] Improve Language Identification for use in Cirrus Search
Open	None	T138958 Detect "wrong keyboard" queries for Russian/American keyboards on EN/RU Wikipedias
Resolved	TJones	T213931 Update TextCat with wrong-keyboard models
Declined	TJones	T213935 Revert changes to TextCat that add dependency on autoload.php

Event Timeline

TJones triaged this task as High priority.Jan 16 2019, 3:14 PM

TJones created this task.

TJones mentioned this in T213936: Deploy new version of TextCat.Jan 16 2019, 3:39 PM

TJones moved this task from Incoming to not in use - please delete on the Discovery-Search (Current work) board.

Change 484752 had a related patch set uploaded (by Tjones; owner: Tjones):
[wikimedia/textcat@master] Add Wrong-Keyboard and Wrong-Encoding Models to TextCat

https://gerrit.wikimedia.org/r/484752

gerritbot added a project: Patch-For-Review.Jan 16 2019, 7:43 PM

TJones moved this task from not in use - please delete to Needs review on the Discovery-Search (Current work) board.Jan 16 2019, 7:49 PM

TJones closed subtask T213935: Revert changes to TextCat that add dependency on autoload.php as Declined.Jan 17 2019, 2:27 PM

Updated Perl models on GitHub. PHP models are awaiting review in the patch above.

Change 484752 merged by jenkins-bot:
[wikimedia/textcat@master] Add Wrong-Keyboard and Wrong-Encoding Models to TextCat

https://gerrit.wikimedia.org/r/484752

TJones moved this task from Needs review to Needs Reporting on the Discovery-Search (Current work) board.Jan 18 2019, 3:49 PM

debt closed this task as Resolved.Jan 18 2019, 7:06 PM

TJones mentioned this in T245677: Reader searches with romanized version of non-Latin script.Feb 26 2020, 8:32 PM