Page MenuHomePhabricator

[Tech spike] Action API integration with TWN for dynamic content translations
Open, MediumPublic

Description

Can I dream of a somewhat more API based real-time integration?

Sure! Dreams are what make us find better solutions right? I looked at the docs on TWN a bit and did not see an obvious page about using an API other than the git import/export system.

On the TWN side (as far as I understand it), the messages end up being normal pages like https://translatewiki.net/wiki/Wikimedia:Wsa-404-header/en, https://translatewiki.net/wiki/Wikimedia:Wsa-404-header/qqq, and https://translatewiki.net/wiki/Wikimedia:Wsa-404-header/pt. Could integration at an API level be as direct as using the Action API to create or update a message to be localized? That seems to be functionally what the import process for a git based source file does. I also see action=managemessagegroups in the API list which looks like it could be used to create new messages, but it is marked as "internal". Getting translated things back out of TWN looks to be as direct as calling action=query&meta=messagetranslations?

Can I dream of a somewhat more API based real-time integration?

Translatewiki.net is not suitable for on-demand fetching of translations through an API: Our setup is not high-availability nor designed with that kind of performance considerations in mind.

I certainly would not want to put undue stress on the TWN servers, but assuming we put reasonable caching in place from the Toolhub side I wonder how long the product would have to grow before performance actually became a concrete issue? It is hard to make educated guesses about page views per language or even total page views, but Toolhub is most definitely going to have access patterns more like a tool than a project wiki. By this I mean that page views are going to be on the order of N thousands per day rather than N thousands per second. I think it would be reasonable to think about caching lookups for up to 24 hours on the Toolhub side. I don't think it makes sense for the stability of either system to think about real time lookup of all messages.

Event Timeline

bd808 triaged this task as High priority.Oct 6 2020, 10:49 PM

I chatted with @Nikerabbit a bit on IRC today. He suggested that we try building a proof of concept for the VCS sync method discussed in T259838: Explore translation options for Toolhub dynamic content before trying this API driven approach. The reasoning being that the API approach is likely to end up needing some work on the TWN side to complete which adds risk. This all sounded reasonable to me. Documenting here as we do not currently have a higher level task about dynamic content translation support in general.

bd808 lowered the priority of this task from High to Medium.Apr 12 2022, 8:33 PM