Page MenuHomePhabricator

Translation memory not working for anything longer
Closed, ResolvedPublic

Description

Tech News translators are experiencing the translation memory not working for anything but the very shortest of sentences.

See discussion here:
https://meta.wikimedia.org/wiki/Talk:Tech/News/2017/41

Details

Event Timeline

Johan created this task.Oct 8 2017, 10:46 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 8 2017, 10:46 PM
Stryn added a subscriber: Stryn.Oct 9 2017, 4:40 AM
Nikerabbit triaged this task as High priority.Oct 9 2017, 6:47 AM
Nikerabbit moved this task from Backlog to TTMServer on the MediaWiki-extensions-Translate board.
Nikerabbit added subscribers: dcausse, Nikerabbit.

CC @dcausse for ba7373e05385fe982d75efc537f38cf4aa158c24

Another report I got last night: https://meta.wikimedia.org/wiki/User_talk:Nikerabbit#K.C3.A4.C3.A4nn.C3.B6smuisti_ei_anna_en.C3.A4.C3.A4_ehdotuksia_pidemmille_viesteille

Based on the reports, it doesn't seem to work "without significant degradation" as I hoped.

While revert seems appropriate for an interim solution, there is now also https://github.com/codelibs/elasticsearch-minhash LSH plugin that could tried out to see if it can replace the fuzzy query.

Change 383075 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/Translate@master] Revert "[tech-debt] Remove usage of FuzzyLikeThis in favor of simple fuzzy match"

https://gerrit.wikimedia.org/r/383075

@Nikerabbit I think it'd be interesting to try this plugin, but given that it'll be hard to test such changes without breaking current usage I'd suggest keeping the FuzzyLikeThis method now and start to add some new logic in TTM that we could activate with a URL param.

I'll swat the revert today, and add this plugin to the list of plugins to install to production.

Change 383079 had a related patch set uploaded (by DCausse; owner: DCausse):
[mediawiki/extensions/Translate@wmf/1.31.0-wmf.2] Revert "[tech-debt] Remove usage of FuzzyLikeThis in favor of simple fuzzy match"

https://gerrit.wikimedia.org/r/383079

@dcausse Your suggested approach makes perfect sense.

Change 383075 merged by jenkins-bot:
[mediawiki/extensions/Translate@master] Revert "[tech-debt] Remove usage of FuzzyLikeThis in favor of simple fuzzy match"

https://gerrit.wikimedia.org/r/383075

Change 383084 had a related patch set uploaded (by DCausse; owner: DCausse):
[search/extra@master] Revert "Stop maintaining FuzzyLikeThis"

https://gerrit.wikimedia.org/r/383084

Change 383084 merged by Gehel:
[search/extra@master] Revert "Stop maintaining FuzzyLikeThis"

https://gerrit.wikimedia.org/r/383084

Johan renamed this task from Translation memory now working for anything longer to Translation memory not working for anything longer.Oct 9 2017, 10:33 AM

Change 383079 merged by jenkins-bot:
[mediawiki/extensions/Translate@wmf/1.31.0-wmf.2] Revert "[tech-debt] Remove usage of FuzzyLikeThis in favor of simple fuzzy match"

https://gerrit.wikimedia.org/r/383079

Mentioned in SAL (#wikimedia-operations) [2017-10-09T13:53:01Z] <zfilipin@tin> Synchronized php-1.31.0-wmf.2/extensions/Translate/: SWAT: [[gerrit:383079|Revert "[tech-debt] Remove usage of FuzzyLikeThis in favor of simple fuzzy match" (T177727)]] (duration: 00m 57s)

@Johan we reverted the change that caused the issue, I tested few french translations and it seemed to work again on long text, could you double-check on your side and close the task if the issue is resolved? Sorry for the inconvenience, next time I'll keep the current implementation in place so that we can experiment with new techniques without breaking production usage.

Johan closed this task as Resolved.Oct 9 2017, 2:00 PM
Johan claimed this task.

Yes, this is working as expected again for me. Thanks.

Restricted Application added a project: User-Johan. · View Herald TranscriptOct 9 2017, 2:00 PM
Johan added a comment.Oct 9 2017, 2:01 PM

(Sorry, had accidentally assigned this to me; didn't have that intention.)

matej_suchanek removed Johan as the assignee of this task.Oct 9 2017, 4:37 PM