Page MenuHomePhabricator

Server-side caching for MinT
Closed, ResolvedPublic

Description

MinT supports machine translation for different products. In the initial integrations of MinT to support Content Translation and the Localization infrastructure, the requested contents to translate are quite unique. That is, an editor creating a new translation for a Wikipedia article will generate requests for MinT to translate the different sentences of such article. which will be very different from those translated by another editor working on a different article. In this context, there was no need for caching.

However, as MinT is applied in contexts where multiple users may be reading the same translated content, there may be a bigger benefit in applying caching to reduce waiting times and the general workload for the service. This may be relevant for context such as the translation of wishes (T363306) for the new Wishlist process and MinT for Wikipedia readers (T359072).

This ticket proposes to consider ways in which the MinT service could use caching to provide a better performance in the described contexts. Given a certain request text, language pairs and models, the result should come from the caching system if the same exact request was made previously/recently.
We need to consider that (a) language models are not updated frequently, but we also plan to allow users to provide community-verified translations (T351748) and the caching system should not get in the way of users being able to correct a bad translation (i.e. taking time to see changes reflected).

Event Timeline

Pginer-WMF added a project: MinT.

CXServer may be a better place for caching to keep MinT itself simple(r). One could even go one step further and make a caching proxy API in MediaWiki, given MediaWiki has easy to use caching options.

Simple LRU cache with a maximum storage limit could be a good starting point, with monitoring to monitor cache hit rate to see if the cache is effective at all.

It may also be beneficial to support client provided cache hints to determine which content should be cached and for how long.

Thanks for surfacing potential approaches @Nikerabbit!

I'd defer to the engineers to identify the most promising directions. From my perspective, I think starting simple and reusing existing capabilities from the platform sounds great to me.

In this way, we can avoid premature optimization since this is a case where we anticipate that caching may be beneficial, but we still have not a sense of how often this potential overlap among requests will be happening in practice.

Change #1056869 had a related patch set uploaded (by Santhosh; author: Santhosh):

[mediawiki/services/machinetranslation@master] Introduce optional caching for translations

https://gerrit.wikimedia.org/r/1056869

Change #1056869 merged by jenkins-bot:

[mediawiki/services/machinetranslation@master] Introduce optional caching for translations

https://gerrit.wikimedia.org/r/1056869

Change #1057421 had a related patch set uploaded (by KartikMistry; author: KartikMistry):

[operations/deployment-charts@master] Update MinT to 2024-08-05-062247-production

https://gerrit.wikimedia.org/r/1057421

Change #1057421 merged by jenkins-bot:

[operations/deployment-charts@master] Update MinT to 2024-08-05-062247-production

https://gerrit.wikimedia.org/r/1057421

@santhosh Is this QA testable and if so, can you please provide me the steps? Thanks!

@GMikesell-WMF It is a backend feature and verifiable by developers. You may skip it. We had verified it.

This ticket completes the basic fallback caching. For a more reliable caching using the Mediawiki standard systems for that purposes you can check T370755: Caching service request for MinT