Page MenuHomePhabricator

Server-side caching for MinT
Open, MediumPublic

Description

MinT supports machine translation for different products. In the initial integrations of MinT to support Content Translation and the Localization infrastructure, the requested contents to translate are quite unique. That is, an editor creating a new translation for a Wikipedia article will generate requests for MinT to translate the different sentences of such article. which will be very different from those translated by another editor working on a different article. In this context, there was no need for caching.

However, as MinT is applied in contexts where multiple users may be reading the same translated content, there may be a bigger benefit in applying caching to reduce waiting times and the general workload for the service. This may be relevant for context such as the translation of wishes (T363306) for the new Wishlist process and MinT for Wikipedia readers (T359072).

This ticket proposes to consider ways in which the MinT service could use caching to provide a better performance in the described contexts. Given a certain request text, language pairs and models, the result should come from the caching system if the same exact request was made previously/recently.
We need to consider that (a) language models are not updated frequently, but we also plan to allow users to provide community-verified translations (T351748) and the caching system should not get in the way of users being able to correct a bad translation (i.e. taking time to see changes reflected).

Event Timeline

Pginer-WMF added a project: MinT.

CXServer may be a better place for caching to keep MinT itself simple(r). One could even go one step further and make a caching proxy API in MediaWiki, given MediaWiki has easy to use caching options.

Simple LRU cache with a maximum storage limit could be a good starting point, with monitoring to monitor cache hit rate to see if the cache is effective at all.

It may also be beneficial to support client provided cache hints to determine which content should be cached and for how long.

Thanks for surfacing potential approaches @Nikerabbit!

I'd defer to the engineers to identify the most promising directions. From my perspective, I think starting simple and reusing existing capabilities from the platform sounds great to me.

In this way, we can avoid premature optimization since this is a case where we anticipate that caching may be beneficial, but we still have not a sense of how often this potential overlap among requests will be happening in practice.