Wikimedia sites do not use MediaWiki's default search backend (MediaWiki-Search), hence setting CirrusSearch.

Probably see also: T77967 (for Chinese?)

It looks like a potential solution to this is Elastic's "serbian_normalization" plugin. It has been in "not-released-yet" status since v2.0 (the current version is v5.4), but it is in the Elastic code and already available in my development installation. I don't know if it is unstable or just improperly documented, but it hasn't been updated in a long time.

It is built on the Lucene analyzer, but it does not have the "haircut" option ("bald" Latin vs Latin with diacritics). The haircut option also seems to have been removed from the Lucene code.

So, if we wanted to implement both a Cyrillic-to-Latin conversion and the bald-Latin conversion (from T138858), we could do that by enabling this plugin for Serbian-language wikis.

As the Lucene link from the Description points out, this does end up conflating some words with diacritics in Latin, like strašan/strasan, and teža/teza. This conflation would map back into the Cyrillic, too, because the Cyrillic would be mapped to the bald Latin for indexing and search.

The mapping is:

a б в г д ђ  е ж з и ј к л љ  м н њ  о п р с т ћ у ф х ц ч џ  ш / č ć đ  š ž
a b v g d dj e z z i j k l lj m n nj o p r s t c u f h c c dz s / c c dj s z

If that doesn't cover everything, let me know and I can try out other characters, too.

Some pros and cons:

• Pro++: the conversion only affects the text that is indexed, so the article text is unchanged. If "teža" or "тежа" is in an article, it would be indexed as "teza", but unchanged in the article.
• Pro+: We also search an unchanged version of the text, so a query of "teža" would match "teža" somewhat better than "teza". It would not get any boost matching "тежа", though.
• Con-: Words with non-Serbian Cyrillic would only be partially converted. So Russian "чёрная дыра" ("black hole") would be indexed as "cёrnaя dыra". I don't think that's a problem, but there could be weird collisions. The Faux Cyrillic band name "LIИKIИ PARK" would get indexed as "liikii park". ("KoЯn" comes out relatively unscathed.)
• Con-: There is a chance there is some difference between the conversion done by the LanguageConverter (for display) and this plugin (or custom version of it, see below) that could cause confusion.
• Con--: This is not clearly an official release from Elastic, so it could disappear or change.
  • Pro+: We don't want to wait forever, though.
  • Pro++: On the other hand, if all we need is a very simple mapping as above, then it would be very easy to write this as custom filter, and we could choose either the bald or not-bald version. (I think doing supporting both bald and not-bald would be hard and would require a level of customization for Serbian-language wikis that we aren't able to support.)

If the community supports using bald Latin for searching and matching—meaning that some distinctions could be lost—then this is straightforward. Of course I'd do the usual analysis and set up an instance in labs for testing before making it live, so we could be sure it works as intended.

We would still need deal with the bald Latin search (T138858), but the upcoming Serbian analysis chain (T183015) will take care of the Cyrillic-vs-Latin search, in addition to doing some basic stemming.

There will still be a slight preference for exact matches in the same alphabet, just as there is a slight preference for exact matches that are stemmed. As an example of the stemming match, if you search for dogs then dog is a good match, but dogs is a slightly better match, via the plain field. The plain field also allows you to match exact forms with quotes, so that searching for "dogs" does not match dog.

*hot dog* :-P