Page MenuHomePhabricator

Investigation: i18n framework for Tool Labs tools
Closed, ResolvedPublic

Description

Answer the following questions:

  • What solutions already exist?
    • What programming languages do they support?
    • Are they old and crusty or actively maintained?
    • Are there improvements/bug fixes/features that are needed?
  • If a new framework is needed for a particular language (like Python), would it make more sense to port a framework from another language or to create something from scratch?

Timebox: 1 day

Event Timeline

kaldari raised the priority of this task from to Needs Triage.
kaldari updated the task description. (Show Details)
kaldari added a project: Community-Tech.
kaldari subscribed.

Looks promising :) Is there a framework for Python tools too?

Not that I'm aware of. You might be able to re-use pywikibot's i18n module though.

I thought of extracting i18n code from my Node-based rewrite of the Deep Wiki Inspector, using a banana-like format, but I am not able to reach a decent implementation.
There's also ongoing work on nodejs/Intl

Hi @Krinkle, I was trying to catch you on IRC but couldn't. I was interested in knowing how much time/effort you think it'd take to add support for Python tools to Intuition? Other solutions exist, but none seems as sophisticated and easy to use as Intuition. There are many tools that could benefit from this.

I'd love for Intuition to be used more widely, such as in Python tools.

This will fit in nicely with the existing range of tools using Intuition for localisation. And it also benefits from the established relationship I have set up with translatewiki.net for this.

I've recently released a new version of Intuition that encourages wider re-usability of the framework by removing the requirement that text domains are registered and shipped in the core software and main git repository. Instead, tools are now encouraged to host the i18n directory in their own repository (thus keeping it close to the actual code, and making it easier for developers to add new messages in atomic changes).

The migration guide (https://github.com/Krinkle/intuition/wiki/Migrate) should help people with their repo set up (it's agnostic about whether you use the original Intuition or a python port).

As for what a Python port would need to contain, here are the required features for a compatible Intuition API:

  • Register domains by name and directory path.
  • Retrieve messages by key and language from the JSON files.
  • Automatically fallback to English for keys not yet translated.

This is currently implemented in the PHP interface (php-source, example use in OrphanTalk), and in the JavaScript interface for on-wiki gadgets (js-source, example use in RTRC).

Optional features that I'd strongly recommend:

  • Implement parsemag MessageFunctions (PLURAL and GENDER). Coordinate with the Language Engineering team to see if there's a way we can auto-generate the relevant Language classes for these in Python. (@siebrand, @Amire80, @Nikerabbit)
  • Support the tool-labs wide TsIntuition_userlang cookie to initialise the user language.
  • Provide a method to output HTML that points to the Intuition dashboard to manage the user preference with returnto query to where the user was.
  • Embed the full fallback chain (not just from non-English to English).
  • Provide a method to determine whether the current language is LTR or RTL.

Aside from the presence of these features, I'd also recommend we standardise the method names and parameters (function signatures) to provide users with a consistent API for Intuition across Python and PHP.

Answer the following questions:

  • What solutions already exist?
    • What programming languages do they support?
    • Are they old and crusty or actively maintained?
    • Are there improvements/bug fixes/features that are needed?
  • If a new framework is needed for a particular language (like Python), would it make more sense to port a framework from another language or to create something from scratch?

Intuition

  • Actively maintained
  • Compatible with TranslateWiki
  • Works with PHP and JS as of now
  • Good chance of getting support from volunteer contributors for this task

Features required in a Python port:

  • Register domains by name and directory path.
  • Retrieve messages by key and language from the JSON files.
  • Automatically fallback to English for keys not yet translated.

Pros: This would be a one-size fits-all solution, which we can use for Python and PHP projects alike. It seems like the best way forward here would be to create subtasks for completing the basic Python port for Intuition. We can then adding the optional features that Krinkle mentioned in his comment above:

  • Implement parsemag MessageFunctions (PLURAL and GENDER). Coordinate with the Language Engineering team to see if there's a way we can auto-generate the relevant Language classes for these in Python.
  • Support the tool-labs wide TsIntuition_userlang cookie to initialise the user language.
  • Provide a method to output HTML that points to the Intuition dashboard to manage the user preference with returnto query to where the user was.
  • Embed the full fallback chain (not just from non-English to English).
  • Provide a method to determine whether the current language is LTR or RTL.

Pywikibot's i18n module

  • Also relies on TranslateWiki
  • Seems a lot more basic, compared to Intuition.
  • Going by this: https://www.mediawiki.org/wiki/Manual:Pywikibot/i18n_conversion it looks like to make the i18n module usable for getting tool translations, we'd have to write a few scripts to automate some of the tasks, such as adding translation keys and fetching translated messages from TWN periodically etc.

Conclusion: I vote that we aim at adding a Python port to Intuition and create the recommended subtasks above.

Outcome of this investigation was T116190.