Page MenuHomePhabricator

[BUG] Some translations not recognised
Closed, ResolvedPublic5 Story PointsBug

Description

What is the problem?

Some translations in SVGs are not recognised by the tool. They don't appear in the "From" translation dropdown and they are not in the SVG if you download it from the tool (like it has been stripped out).

For example, this image, if you download the SVG from commons has languages fr and zh-my. But, the tool only picks up fr.

This also means that, if you upload a new translation to Commons it will override all the translations it did not recognised. So in the above example, that SVG also had a zh-hk translation, but that has been overridden when I uploaded the fr translation.

Speculation

So far, all the missing translations have codes of the form "xx-xx" (e.g. zh-hk). I have not seen it happen for languages such as "en", "simple", etc. I have not tested this exhaustively.

On the other hand, the tool does recognise all the languages (including things like zh-hans) for this SVG.

Steps to reproduce problem
  1. Find a translatable SVG image
  2. Enter a translation for a language with a dash, e.g. es-419, zh-my
  3. Upload this file to Commons

Expected behavior: After the upload succeeds, you will see your new translation in the "From" dropdown
Observed behavior: Your new translation is not in the dropdown. The SVG you download from the tool will not have the translation.

Screenshots (if applicable):

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 11 2019, 11:24 AM
Niharika set the point value for this task to 5.Apr 16 2019, 11:07 PM
Niharika triaged this task as Normal priority.
Samwilson moved this task from Ready to In Development on the Community-Tech-Sprint board.
Samwilson added a subscriber: Samwilson.

I'm getting this error with the above file:

Warning: DOMNode::insertBefore(): Couldn't add newnode as the previous sibling of refnode, in src/Model/Svg/SvgFile.php (line 820)

Which I think means it's failing to correctly reorder those text elements with a sublocale (i.e. any language code with an underscore).

PR: https://github.com/wikimedia/svgtranslate/pull/105

This is failing in the code that reorders the text elements in a switch statement so that sublocales are at the top and fallback entries are at the bottom. Why do we do this? It seems unnecessary, but it might have something to do with errors in rsvg's handling of variants/fallbacks and so higher ones get preferred. Anyway, we weren't testing for language codes given with underscores vs hyphens — the latter is I think the standard, so I've added support for them here (and left in the underscore checking because they're also used sometimes). Ideally we'd normalize these values, but I think the above patch makes sense anyway.

dom_walden moved this task from QA to Product sign-off on the Community-Tech-Sprint board.EditedWed, May 1, 2:52 PM

I can no longer reproduce the bug in the description, https://tools.wmflabs.org/svgtranslate-test/File:USCG_Sentinel_class_cutter_poster.svg returns the appropriate translations.

Using some preexisting SVGs on beta commons, I compared what text I could find from parsing the SVG with what appears in the tool (by inspecting the appConfig.translations JS variable).

The only translations that appear to be missing from the tool are preexisting bugs (e.g. T221382, we don't support nested tspans) or where the SVG format is somewhat questionable. For example, this file is missing the translation for "Eustachian tube" because that <switch> statement has no fallback language.

@Niharika How forgiving should the tool be?

I tried to test uploading translations to commons and seeing if these were recognised by the tool. I could not get very far as there appears to be some kind of caching happening and you might have to wait some time before the tool picks up the latest version of the SVG.

I can no longer reproduce the bug in the description, https://tools.wmflabs.org/svgtranslate-test/File:USCG_Sentinel_class_cutter_poster.svg returns the appropriate translations.

Using some preexisting SVGs on beta commons, I compared what text I could find from parsing the SVG with what appears in the tool (by inspecting the appConfig.translations JS variable).

The only translations that appear to be missing from the tool are preexisting bugs (e.g. T221382, we don't support nested tspans) or where the SVG format is somewhat questionable. For example, this file is missing the translation for "Eustachian tube" because that <switch> statement has no fallback language.

@Niharika How forgiving should the tool be?

I tried to test uploading translations to commons and seeing if these were recognised by the tool. I could not get very far as there appears to be some kind of caching happening and you might have to wait some time before the tool picks up the latest version of the SVG.

Hmm, I think we'll have to be satisfied with what we have for now. I wish it were less forgiving but I don't think we have more time to spend on this, unfortunately. Thanks for the thorough review, Dom!

Niharika closed this task as Resolved.Wed, May 1, 8:59 PM
Niharika moved this task from Product sign-off to Done on the Community-Tech-Sprint board.

Commits linked to this task have been released in svgtranslate 0.9.0.