Page MenuHomePhabricator

Split most used messages to separate i18n file
Closed, DeclinedPublic

Description

The "most used" or "most important" list of messages is useful for languages that start their journey in the Wikimedia world. It is also used by the Language committee as one of the criteria for checking that a language is ready for "graduating" from the Incubator. It includes messages, mostly from core MediaWiki and also from some skins and extensions, which are most frequently seen by users, for the most central wiki reading and editing tasks: page editing, logging in, categories, talk pages, language switching, etc.

Before 2015, it was generated by gathering statistics. However, some of those statistics were bringing up messages that are frequently loaded for technical reasons, but aren't actually seen by a lot of users. In addition, they were shown in a random order, which is not great for translators. So since 2015, that list was curated manually, mostly by @Amire80, @jhsoby, and @Nemo_bis.

The translatewiki message group for it uses a special format—a plain list of message keys, which duplicates the keys that already appear in json files. This makes maintenance somewhat inconvenient, because message keys sometimes get changed or deleted, and this has to be done in more than one place. It is proposed to move away from this text file format and store the most important messages as JSON files mapped to message groups. The actual steps to doing it will be discussed in the comments to this task.

A proposed transition plan by @Amire80 (other suggestions are welcome):

  • Create a group of the most important messages within core. The name is flexible: "Main", "Most important", etc.
  • Create similar groups in the other extensions that are currently included in "Most important". (I've already discussed this with @Lydia_Pintscher about Wikibase, for example, and she supported it. I'm not sure that she remembers because it was three or four years ago.)
  • Create an aggregate group that includes all the "Most used" groups in core and extensions.
  • Delete the "Most important" group from translatewiki.

Event Timeline

Genuine question: would it not be potentially slightly confusing if the messages get separated by component, but then some of the messages were also separated by being 'most-used'?
E.g., if the messages for the MediaWiki-Blocks component were separated, and I was looking to find one of those messages, I might instinctively look for them in the (e.g.) i18n/blocks/en.json file; however, some of the messages relating to the MediaWiki-Blocks component might then actually live in i18n/main/en.json instead, which seems like it might have the potential to confuse things a bit.

I am not sure I think this is a good idea. The benefit of the current setup of the most used message group is that we can include messages from many different sources. There are messages there from many more sources than just core MediaWiki. Off the top of my head, we also have messages from the Vector skin, the Minerva skin, the MobileFrontend extension, the VisualEditor extension, the WikiEditor extension, the Wikibase extension and more. Separating out the parts that come from core wouldn't really solve any problems, since it would have to be done on a per-component basis, and that might be confusing, like @A_smart_kitten points out.

It would also remove a lot of messages from the context they're in when translating MediaWiki core proper; when using Special:Translate/mediawiki-core (and choosing to display all messages, not just the untranslated ones), the order the messages appear in is the order they appear in the i18n JSON file. If we remove the most used messages from the main and component-based files, translators will lose that context which may be useful for them.

I think it would be better to try to think of other ways of reducing the maintenance of the list of most used messages, like perhaps a jenkins job to check if any renamed or removed message keys also appear in the list of most used messages.

I’m pretty sure it’s not a good idea. As I wrote in T167762#11223704, context helps a lot to create quality translations. (As a developer, I personally don’t care much: I just grep for the message keys, which finds those wherever they are. Although other developers may work differently.)

MaryMunyoki subscribed.

Hi. Some people expressed objections to this idea. It's my fault—I should have written more details earlier. I've updated the task description now. I'll reply in detail here:

Genuine question: would it not be potentially slightly confusing if the messages get separated by component, but then some of the messages were also separated by being 'most-used'?

Till now, we've mostly separated components that are relatively obscure, like Interwiki and Bot Passwords. The most notable components that were separated so far are probably Preferences and DateTime.

I don't think that it's really a very big deal. Even within relatively notable components, there can be separation to "Basic" and "Advanced".

E.g., if the messages for the MediaWiki-Blocks component were separated, and I was looking to find one of those messages, I might instinctively look for them in the (e.g.) i18n/blocks/en.json file; however, some of the messages relating to the MediaWiki-Blocks component might then actually live in i18n/main/en.json instead, which seems like it might have the potential to confuse things a bit.

There could be "Blocks - Basic" and "Blocks - Advanced". There are currently more than 200 messages in core related to blocking. As a first step, those of them that currently appear in "Most important" can go to "Blocks - Basic", and the rest can go to "Blocks - Advanced". It can be adjusted later.

Searching for a message's definition is not a big deal anyway. When I handle messages, I mostly grep and don't search by directories (and @Tacsipacsi wrote the same). The division into files is most important to the translators. We currently have about 4000 messages in core, and it's obviously going to grow further, so we need to think of some kind of division now.

I am not sure I think this is a good idea. The benefit of the current setup of the most used message group is that we can include messages from many different sources. There are messages there from many more sources than just core MediaWiki. Off the top of my head, we also have messages from the Vector skin, the Minerva skin, the MobileFrontend extension, the VisualEditor extension, the WikiEditor extension, the Wikibase extension and more. Separating out the parts that come from core wouldn't really solve any problems, since it would have to be done on a per-component basis, and that might be confusing, like @A_smart_kitten points out.

See my proposed plan in the task description. I'm quite sure that it can scale. What doesn't scale well is having ~4000 messages in one group. That's already way too much. Some big extensions like Wikibase, Content Translation, AbuseFilter, etc., should also be split in any case (in fact, some of them have some division already, and it should go further).

I think it would be better to try to think of other ways of reducing the maintenance of the list of most used messages, like perhaps a jenkins job to check if any renamed or removed message keys also appear in the list of most used messages.

This will perhaps solve the problem of maintaining the keys, but it will not solve the problem of a ~4000 messages in one group, which is inevitably going to grow further.

I’m pretty sure it’s not a good idea. As I wrote in T167762#11223704, context helps a lot to create quality translations. (As a developer, I personally don’t care much: I just grep for the message keys, which finds those wherever they are. Although other developers may work differently.)

I don't think that it will ruin the context. Messages that appear in the same screen should appear in the same group, too. If currently some messages from a given screen appear in "Most important", and some don't, it's probably a mistake in the first place.

I’m pretty sure it’s not a good idea. As I wrote in T167762#11223704, context helps a lot to create quality translations. (As a developer, I personally don’t care much: I just grep for the message keys, which finds those wherever they are. Although other developers may work differently.)

I don't think that it will ruin the context. Messages that appear in the same screen should appear in the same group, too. If currently some messages from a given screen appear in "Most important", and some don't, it's probably a mistake in the first place.

I don't think that it's really a very big deal. Even within relatively notable components, there can be separation to "Basic" and "Advanced".

…and it may very well be possible that “basic” and “advanced” appear on the same screen. I feel you contradict yourself here.

Also, “context” is not only about messages that appear on the same screen. It’s about messages that describe the same feature. Maybe a button text Protect $1 isn’t that important, because it’s used only by admins, but the error message $1 is protected is important because it’s shown everyone if they try to edit the main page. Yet, the word “protect” should be translated the same. (For example, the canonical German translation of “protect” is “sperren”, literally “bar entry”, rather than the literal translation of “protect”, which would be “schützen”. Without context, a translator could get it wrong.)