Page MenuHomePhabricator

Wish form internationalization
Closed, ResolvedPublic

Description

Background

The new Wishlist aims to increase participation among users across the globe and our projects. We can measure this by 1) diversity of submissions and 2) diversity of votes/edits by language/IP addresses.
To create a new wish, users will enter a valid form submission. It requires project, type (bug/new idea), title, description, and users impacted, as well as optional fields for Phabricator tickets.

Goal: localize the form so that any user can view the form in their native language (i.e. their current interface language as set in their preferences).

User story

Given that I am logged in and have set my default language on wiki to non-english, or I am visiting from an IP address where the default language is not english, when I visit the wishlist intake form, then all form fields and labels should be visible in my native language.

Requirements

  • The user should be able to view all form fields, labels, and helper text in their native language
  • The user should view all form validation or error states in their native language
  • All buttons should be in the user's native language

QA

  • QA can be done on patchdemo.
  • The form's interface language should follow the currently selected user language (in preferences, or via ?uselang=xx in the URL).
  • Currently there is only en available, but qqx should also work to display the message names. Don't worry about testing for languages other than those two.

Feature: Localization of Wish Form

BDD

Feature: Localization of Wish Form

Scenario: Ensure all form elements are localized correctly
  Given the user has their language settings set to a non-default language
  When the user navigates to the Wish form
  Then all form elements should display text in the user's language

Scenario: Ensure error messages are localized correctly
  Given the user has their language settings set to a non-default language
  When the user submits the Wish form with invalid inputs
  Then the error messages should be displayed in the user's language
Test Steps

Test Case 1: Ensure Form Elements are Localized Correctly

  1. Navigate to the Wish form. https://patchdemo.wmflabs.org/wikis/daa3db7f2e/wiki/Community_Requests/Intake
  2. Set the language by adding ?uselang=en after the URL.
  3. Verify that all form elements display text in the selected language.
  4. AC1: Confirm that all labels, placeholders, and instructions are correctly localized.

Test Case 2: Ensure Form Elements are Localized Correctly

  1. Navigate to the Wish form. https://patchdemo.wmflabs.org/wikis/daa3db7f2e/wiki/Community_Requests/Intake
  2. Set the language by adding ?uselang=qqx after the URL.
  3. Verify that all form elements display text in the selected language.
  4. AC2: Confirm that all labels, placeholders, and instructions are correctly localized.

Task Description:

QA Results - PatchDemo

ACStatusDetails
1T361512#9819479
2T361512#9819479

Details

TitleReferenceAuthorSource BranchDest Branch
Add banana checker for i18n messagesrepos/commtech/wishlist-intake!24samwilsonbanana-checkermain
Add remaining i18n messagesrepos/commtech/wishlist-intake!12samwilsoni18n-msgsmain
Add i18n system and start l10nrepos/commtech/wishlist-intake!5samwilsoni18n-setupmain
Customize query in GitLab

Event Timeline

Translate allows translating pages, not forms. If you want to have a localized form, I think chances are better with VisualEditor and TemplateData, as TemplateData natively supports translating parameter labels and descriptions (of which a form is composed). It also allows dropdown lists for parameter values, although the list items are not localizable yet (T292969).

Thanks - this is something we'll look into. For clarity, I am discussing localizing and/or translating the form fields so that the form is accessible in a user's native language. Example image of a form field below.

image.png (187×1 px, 32 KB)

For clarity, I am discussing localizing and/or translating the form fields so that the form is accessible in a user's native language. Example image of a form field below.

image.png (187×1 px, 32 KB)

This is not exactly what I had in mind, but neither something I remember having seen in previous iterations of CWS. What shows this form? A gadget? A Toolforge tool? A special page that’s being built? Where are the form fields defined? TemplateData may or may not be a good fit depending on these questions.

The plan is to use a gadget for the survey intake (the form). Until T238386 is a reality, we can rely on the translatable-wikitext-becomes-JSON system we've used for other gadgets such as AddMe https://meta.wikimedia.org/wiki/MediaWiki:Gadget-addMe-messages (there are proper safeguards in place to make sure things get parsed correctly, FYI). Longer-term, we may migrate to using translatewiki.net.

This task is about internationalizing the intake form isn't it? Making the labels etc. available in the user's interface language. I wonder if being able to "submit their response in their native language" is a separate matter and should have its own task? That seems like we'd be having to figure out how to save data (T362761) and put it in the right format and location for the Translate extension to work. (E.g. with a base language other than English? Or by inserting the provided text as the other language and adding the English version ourselves? The latter is how we've done it for previous surveys.) But yeah, should this task stay scoped on the i18n of the form?

I wonder if we can start using TranslateWiki.net now for this, as in have the intake-form build process also produce a i18n.json (or whatever name) wiki page (or bundle it in one of the existing pages) that can be read in and set with mw.messages.set(). That way, we can build and run the intake form locally and it won't depend on any on-wiki pages. (Not that we'd have to register it now on TranslateWiki.net; but we could start using a in-repo i18n/ directory now.)

This task is about internationalizing the intake form isn't it? … should this task stay scoped on the i18n of the form?

Yes, I think so.

I wonder if we can start using TranslateWiki.net now for this, as in have the intake-form build process also produce a i18n.json (or whatever name) wiki page (or bundle it in one of the existing pages) that can be read in and set with mw.messages.set(). That way, we can build and run the intake form locally and it won't depend on any on-wiki pages. (Not that we'd have to register it now on TranslateWiki.net; but we could start using a in-repo i18n/ directory now.)

There won't be a lot of messages initially, but over time this will definitely grow. I think bundling all translations with the gadget may not scale well, but we can work around that:

  • Have the deploy script save to a separate JSON page for each language, so we only have to fetch 1 or 2 (depending on translations exist and/or if fallbacks are needed)
  • Fetch and cache messages in session storage or something, so we don't repeatedly download them over and over
  • Go the old-school hacky route and just use a wikitext-based JSON page (like https://meta.wikimedia.org/wiki/MediaWiki:Gadget-addMe-messages )

I haven't looked into it yet, but the another possible solution and alternative to translatewiki.net are message bundles, which (sort of) allow translatable JSON on the wiki, but comes with a few caveats. It may or may not meet our needs.

Bundling all languages together is probably the easiest, but you're right it's not very efficient. A couple of things that seem useful to me:

  1. Use TranslateWiki rather than on-wiki translation. This would mean that it's possible to also add/change messages during development, and I think ties in well with what you were saying the other day about making the intake form deployable to any wiki (for dev and testing).
  2. Have it be possible to use the messages both while loading directly from init.js and when the code is deployed as a gadget. Bundling all message files together into one works in both these cases, but it should also be possible to load them as separate pages/files.

There are gadgets that do the all-in-one thing, and others that have separate pages, e.g. Gadget-Cat-a-lot.js/fr.

I think if we're only loading one or two languages then caching in browser storage is not necessary; the language data won't be much bigger than any of the various Vue components that we're loading, I'd imagine.

I've got the beginnings of a patch for dynamic loading of per-lang subpages/files.

I don’t like when messages used on-wiki are translated on translatewiki.net:

  • It’s slow. Messages translated until early European Monday morning appear on Wednesday on Meta, meaning a 2 (translated on Monday morning) to 9 day (translated on Monday afternoon) waiting time until the translation appears. In contrast, a translation done on Meta appears almost immediately (or, if a bot is used to bundle the translations, with a probably much lower waiting time – especially in the best-case scenario, which is almost immediately rather than two days later).
  • One needs to leave the wiki to do the translations: personal preferences like UI language and skin and other wiki-bound things like recent changes, watchlists and contributions lists change/remain separate.
  • Even worse, one needs to leave for a non-SUL wiki, where they need to register separately (unlike Meta, no anonymous editing is allowed) and even needs to wait for confirmation until they can actually start translating. Of course, it’s just a one-time thing, but it’s a pretty big onboarding obstacle.

On the other side, the only real advantage of translatewiki.net is that the translations are available on multiple wikis – but this is not a real advantage in this case, since they will be used only on Meta anyway.

@Tacsipacsi You're right about these things, but one of the design goals we've got for the new intake form (and other components of the Wishlist) is that it could at some point be moved to become and extension. See T363374 for some more details about that.

Also, having the messages located within the repository is in keeping with how other MediaWiki components work, and it makes for a much better development/deployment process (i.e. we'll not be restricted to only having messages on Meta but can work on things locally and in demonstration environments such as patchdemo).

It does make it slower to get the translations out to the wiki, but these parts of the form are not likely to be changing that fast (no faster than other parts of the MediaWiki interface anyway). I think it'll work okay.

@Tacsipacsi You're right about these things, but one of the design goals we've got for the new intake form (and other components of the Wishlist) is that it could at some point be moved to become and extension. See T363374 for some more details about that.

Could and at some point are not very convincing… If and when it happens, the translations can be exported from Meta and imported into the extension. Before that, it’s a gadget, and it’s not conventional to translate gadgets on translatewiki.net.

Also, having the messages located within the repository is in keeping with how other MediaWiki components work, and it makes for a much better development/deployment process (i.e. we'll not be restricted to only having messages on Meta but can work on things locally and in demonstration environments such as patchdemo).

The English messages (the message bundle JSON) could be part of the repo and compiled into the JS code. This would also mean less network transfer for the most likely case of people not having been changed their Meta UI language.

@Samwilson Is there a way to test this intake form right now for QA? If so, how? I'll move this back to In Development for now. I can move it back to QA if there is a way, thanks!

@Samwilson Is there a way to test this intake form right now for QA? If so, how? I'll move this back to In Development for now. I can move it back to QA if there is a way, thanks!

You could test it locally by pulling the latest code and then running npm run dev, but @MusikAnimal what do you think, should we change the dev/build commands to always output both the dev and prod builds, to make it easier for people who just want to run it locally?

@MusikAnimal what do you think, should we change the dev/build commands to always output both the dev and prod builds, to make it easier for people who just want to run it locally?

Done as part of https://gitlab.wikimedia.org/repos/commtech/wishlist-intake/-/merge_requests/17

I've added banana validation to CI; MR ready for review.

I think the last part of this is to register with TranslateWiki.

@Tacsipacsi
Could and at some point are not very convincing… If and when it happens, the translations can be exported from Meta and imported into the extension. Before that, it’s a gadget, and it’s not conventional to translate gadgets on translatewiki.net.

I understand what you're saying, but I also think that gadgets are designed to be edited directly on-wiki, and we're not managing this code like that, so it's okay to do things differently for i18n messages as well. Having the messages on-wiki makes it hard to work on, and I really don't think the delay from TranslateWiki will make much difference once things are established.

I also think that gadgets are designed to be edited directly on-wiki, and we're not managing this code like that

There are other gadgets as well that are developed on Git. A few are listed on https://en.wikipedia.org/wiki/Wikipedia:Gadget/Repositories.

I've started T365471 talk about the next steps here. I think this task can be resolved as the form uses i18n messages and can be localized (how that's done is not quite clear yet).

@Samwilson All form fields, labels, and helper text in their native language for "en" and "qqx". Thanks for all your work and I will move this to Done.

Test Result - PatchDemo

Status: ✅ PASS
Environment: PatchDemo
OS: macOS Sonoma 14.5
Browser: Chrome 125, Safari 17.5, Firefox 125
Device: MBA
Emulated Device:NA

Test Steps

✅ AC1: Visiting any URL with uselang=en
Test Link:
https://patchdemo.wmflabs.org/wikis/daa3db7f2e/w/index.php?title=Community_Requests/Intake&uselang=en

Logged inLogged Out
2024-05-21_13-41-28 (1).gif (1×2 px, 1 MB)
2024-05-21_13-40-07.png (1×2 px, 229 KB)

✅ AC2: Visiting any URL with uselang=qqx
Test Link:
https://patchdemo.wmflabs.org/wikis/daa3db7f2e/w/index.php?title=Community_Requests/Intake&uselang=qqx

Logged inLogged Out
2024-05-21_13-42-50.png (1×2 px, 301 KB)
2024-05-21_13-39-50.png (1×2 px, 232 KB)
Samwilson renamed this task from Wish form localization to Wish form internationalization.Wed, May 22, 8:07 AM

Shouldn’t the project names also be translatable? For example, Wikipedia is Wikipédia in French, Википедия in Russian etc.