Page MenuHomePhabricator

Improve Basque language processing for search
Open, Needs TriagePublic

Description

After talking to @Inaki-LL, we identified some minor improvements for Basque language processing for search:

  • Unpack the Basque language analyzer and enable ICU normalization and test ICU folding (with an exception for ñ).

(The previously mentioned issue with the completion suggester folding ñ to n is okay. It happens in the stop analyzer, which is much more aggressive—and which I hadn't taken into account. My test didn't encounter enough exact matches to push the folded matches off the list, which I wasn't expecting, so I misinterpreted the results.)

Event Timeline

TJones created this task.Aug 29 2019, 5:30 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 29 2019, 5:30 PM
TJones updated the task description. (Show Details)Aug 30 2019, 6:55 PM