Page MenuHomePhabricator

Adapt red links on the translation - part 1
Open, MediumPublic

Assigned To
None
Authored By
Arrbee
Dec 10 2014, 11:52 AM
Referenced Files
F23638: link_flagged_lightgray.png
Dec 23 2014, 7:46 AM
F23632: flag_gray.png
Dec 23 2014, 7:46 AM
F23631: flag_gray.svg
Dec 23 2014, 7:46 AM
F23637: link_flagged_lightgray.svg
Dec 23 2014, 7:46 AM
F23634: question-mark.svg
Dec 23 2014, 7:46 AM
F23635: question-mark.png
Dec 23 2014, 7:46 AM
F20493: cx_translate-redlink-marked.png
Dec 10 2014, 11:58 AM
F20495: cx_translate-redlink-warnings.png
Dec 10 2014, 11:58 AM

Description

Context
Links on the source that do not exist on the target language are just added to the translation as plain text. We could better indicate such circumstance and allow users to decide whether to keep them as red-links, or plain text.

Narrative
As a user, I can keep source links as red links, so that they will become blue links when the lacking article is created.

Acceptance Criteria

  1. Links that cannot be adapted will be represented in gray with a dashed underline.
    1. The dashed underline will only appear for the links on the current paragraph. Links from other paragraphs will not show the dashed underline to avoid too much visual noise.
  2. Users selecting them will have an option to turn them into red links from the link card ("mark as missing" option).
    1. Links marked as missing will be represented as red links.
    2. The link card will present the red link, by clicking on it, Content Translation will be opened in a new window to create the corresponding translation for the missing link.
      1. A tooltip "translate (in new window)" will be shown on hover.
    3. Once a link is marked as missing, the link card will provide an option to "remove link" in order to revert the action.

Design Notes
The UI suggests the user to mark as missing links those that could not be adapted:

cx_translate-redlink.png (517×819 px, 211 KB)

Once a red link is created, an option is provided to revert it. In addition, it can be used as an entry point to create a new translation:

cx_translate-redlink-marked.png (517×819 px, 211 KB)

Warning indicators at the bottom, provides awareness of different issues and options to deal with them:

cx_translate-redlink-warnings.png (517×819 px, 266 KB)

Event Timeline

Arrbee assigned this task to Jsahleen.
Arrbee raised the priority of this task from to Needs Triage.
Arrbee updated the task description. (Show Details)
Arrbee changed Security from none to None.
Arrbee subscribed.
Arrbee triaged this task as Medium priority.Dec 10 2014, 3:50 PM

Change 181374 had a related patch set uploaded (by Jsahleen):
Red Links: Adding red link functionality

https://gerrit.wikimedia.org/r/181374

Patch-For-Review

Timeboxed: 23rd December 2014

Here are the needed graphic assets:

Link placeholder with flag:


link_flagged_lightgray.png (647×500 px, 19 KB)

Question mark placeholder:


question-mark.png (647×500 px, 7 KB)

Small flag action:


flag_gray.png (11×11 px, 315 B)

I just uploaded the revised version of my red links patch. I implemented the functionality Pau discussed in the meeting on Monday and in general cleaned things up and added the graphics. Everything is working quite well now. :-) There may still be a few cases where links that actually have corresponding articles are not adapted automatically after machine translation, but these issues have to do with what is returned from wikidata so there’s not much we can do. At any rate, they’re fairly rare and can still be manually adapted by the user.

I did not break the file up into link and red link files, so it’s still pretty huge. I think we should leave it in its current state for now since I couldn’t find an easy way to split things up without making it not pass code validation. A lot of the size is actually comments since I wanted the code to be understandable to others who look at it. If anyone wants to take a crack at splitting up the code while I am out for the holidays please go ahead. However, as I said everything is working quite well right now so we need to be careful to avoid any regressions caused by splitting up the code.

@Pginer-WMF,
"Once a link is marked as missing, the link card will provide an option to "remove link" in order to revert the action."

Here when you click the "remove link" button in the link card, which is the expected result?

(a) Consistent with the removeLink behavior in blue link cards, remove link make the link in the translation column a plain text. There is no way you can make it again a red link. One can edit the text and select and see if there is a target link for that.
(b) remove link make the red link goes to grayed out. There is no way one can make that text just plain text other than deleting and typing same.

https://wikimedia.mingle.thoughtworks.com/projects/language_engineering/cards/4373 is the related story about handling red links in source article.
As per https://gerrit.wikimedia.org/r/#/c/163100/ we make the links in the source article red only when the translator click on them. This was considering the performance trade off of checking every link for its existence.

Note that with option (b) in Santhosh's comment above any grayed out links in the translation column are removed when publishing and left as plain text. The grayed out links also only show when the section being translated has focus, per Pau's design. However, I am fine with removing the link entirely if that is what Pau suggests.

I will refactor my patch set on Monday so that it incorporates Santhosh's comments. Most of the comments have to do with parameter documentation or method naming, but there are some places where the code could be optimized and the file could be shortened by combining similar methods and (possibly) moving them into the selection utility class.

I will submit the updated patch set by EOD on my Monday. If I have time on Sunday I may do it then.

Joel, we discussed this in todays daily meeting, and we would like pause this for a while - till deployment is over. This is a new feature and doing feature development with many unrelated code changes is not a good idea while there a few days before production deployment. Unfortunately this got delayed this much. We can resume this once we are done with deployment. We need your help in focusing bug fixing, code review and testing in coming days. Will discuss with you.

Santhosh, That is fine if you want to pause this until after the release. If we do that, however, you should know that there are several link-related bugs that will need to be fixed now that would have been resolved by my patch. I did not set out to fix these bugs, but since making red links work required making link adaptation in general work, they were fixed as a by-product.

Can you list the bugs? I see only one bug linked to that commit https://gerrit.wikimedia.org/r/#/c/181374/ as per commit message.

The are currently several issues with link adaptation in master. I will go through and compile a list today, making sure everything is in phabricator. I believe most of the issues are currently undocumented. I just came across them in the course of doing red links.

Here is the promised list of current issues with link adaptation. These are fixed in the red links patch, but since we are pulling red links from the release they will need to be addressed in some other way.

https://phabricator.wikimedia.org/T85932
https://phabricator.wikimedia.org/T85931
https://phabricator.wikimedia.org/T85930
https://phabricator.wikimedia.org/T85928
https://phabricator.wikimedia.org/T85927
https://phabricator.wikimedia.org/T74265

Thanks for the list. As you can see, some of them need discussion and decision whether we need to do now or later. There can be fresh ideas about doing them in a completely different way. So please report the bugs as and when you see them and don't make assumptions and hide inside other feature commits, resulting unexpected scope change, resulting delayed code review and completion of task.

I understand about not hiding bug fixes inside feature commits but that is not what I did. IMHO, the root cause of the problems listed above is architectural. In order to make the red links feature work I had to change some aspects of the architecture to accommodate the new feature. Making these architectural changes for red links also resolved the issues listed above. So the bugs were fixed as a result of doing what was necessary to make red links work properly.

Making some assumptions in a case like this is unavoidable. I outlined my assumptions about how things should work in the etherpad I provided (http://etherpad.wikimedia.org/p/red-links).

Change 181374 abandoned by Jsahleen:
Red Links: Adding red link functionality

Reason:
Red links has been postponed until after the release. Due to other code changes it makes no sense to keep this patch.

https://gerrit.wikimedia.org/r/181374

Arrbee raised the priority of this task from Medium to High.Feb 2 2015, 8:08 AM
Arrbee moved this task from Needs Triage to Long term on the ContentTranslation board.
Arrbee lowered the priority of this task from High to Medium.Feb 2 2015, 9:28 AM

We have addressed most of this as part of Link feature rewrite. I am proposing this to Sprint 88 to assess the UX and create tickets for further improvements if any and then close this ticket

(Before I forget) Currently gray links turn blue when focus is elsewhere. I wasted good amount of time because I thought the links which were supposed to be missing were actually blue links. Avoiding visual noise is good, but changing color is distracting and misleading (and not what was intended by my reading of the acceptance criteria).

I made a quick review and found some areas to improve:

  • As Niklas mentioned above, links appear as blue when they are not in the active paragraph. This is confusing since those will not become links in the final result. I see two options to make it more consistent:
    1. Show them as plain text when the user is not in the active paragraph. That will be consistent with the final result, and provide the information when needed the most.
    2. Show them as grey links when they are not in the active paragraph. This keeps missing links consistently represented but can also crowd some translations becoming distracting.In that case we may want to have a way to remove it (e.g., an option to clear the grey marks next to "mark as missing link"). I'd start with option (1) above, and think on more details based on user behaviour.
  • When a grey link is marked as missing, the link turns red but the link card is still showing the missing link card (with the option to turn it into a red link) which no longer makes sense. I'd expect the link card to be updated to reflect the current status (red link with an option to remove it).
  • When text is selected in the translation paragraph and it is turned into a link by clicking on a link from the source paragraph, if the source link cannot be adapted, the the result is a blue link. It should be a grey link, in the same way that it happens with the initial adaptation.
  • When clicking a red link on the link card, a new translation is started. In the new translation dialog, the target title appears prefilled with the former article you were translating, which is not going to match the new article you are starting now. It should be empty for the users to fill instead.

For first bullet, feel free to improve https://gerrit.wikimedia.org/r/#/c/218609/, which I created so that I can test properly.

This is confusing since those will not become links in the final result.

[...]

Show them as plain text when the user is not in the active paragraph. That will be consistent with the final result,

As per the current behavior, they will become red links once published. They are still <a> tags with a title and goes to publishing. This is suboptimal because the title(link target) is in source language. At the same time, "Mark as missing" also does not allow to correct the link target and keep it as red link.

Pau, can we simplify this workflow? Can we remove the "mark as missing" feature and provide "remove link" feature alone? As per the current code, Mark as missing just removes the uncertainity and adds a 'new' css class to the link.

At the same time, the upcoming "add any link" feature allow you to give a proper link target in target language and keep it red.

Getting the correct href for redlinks is tricky because the article in the target language is missing (as it is the information about it on Wikidata).

Currently the "Add missing link" uses the source href (the only available) as the href for the red link in the translation. This approach has low chances of succeeding in anticipating the correct article name (it may work for some terms such as "Internet" in some languages).
Since the most common step would be to edit the target of such link we can anticipate that by opening the link inspector where the user can edit the link target in order to link to correct the redlink href.

There is also another edge case that can be solved in a similar way. Let's imagine that the source article has not an equivalent in the target language, but the source href happens to be a totally unrelated article that does exist in the target wiki. In that case, clicking on "add as missing link" will open the editing link inspector empty so that the user writes the correct name for the target article. For example, Pie (a dessert in English) should not become accidentally a link to Pie ("foot" in Spanish). the user should be able to make the href to be "Pastel" (the Spanish equivalent of the English "pie").

Change 219312 had a related patch set uploaded (by Santhosh):
Convert unadapted links to plain text while publishing

https://gerrit.wikimedia.org/r/219312

Change 219312 merged by jenkins-bot:
Convert unadapted links to plain text while publishing

https://gerrit.wikimedia.org/r/219312

Amire80 subscribed.

If I understand correctly, the missing parts here are:

  • T105915
  • Warning indicators at the bottom, according to Pau's last screenshot. I believe that it warrants a separate task.

Some scattered suggestions, wishes, and questions.

  • Grey links, adapted or not, should get a little reference of the source article in the form of a comment <!-- [[:en:Foobar|translated text]] --> after the link. Doing so would improve the lives of subsequent editors by allowing them to find what the link is really referring to and help them wikify. This helps even more for sites with templates such as https://enwp.org/Template:ill & https://zhwp.org/T:tsl.
  • The target-language community should get a say in whether the red links are "auto-adapted" when possible. That should help with discovering missing articles. Or at least allow an "adapt all" button (just warn users of the potential waiting time).
  • What does it do when an grey link's translated page title happens to be a blue link in the target language? It would likely be a homophone or something else that needs a disambig, so we gotta ask the user about that, right? (Pginer already mentioned how the "no translation" approach we got here breaks cross-language homonyms.)