Some languages are really close, and with good ml engines, and thus will produce a lot of translated text that doesn't need to be fixed. Such translations might give a large number of false positives. One example is the Bokmål-Nynorsk engine from Apertium. This engine often generates flawless translations.

Should this be implemented then the limit must be set for each language pair from each ml engine.

It should also depend on amount of unknown words, at least from translations by Apertium, as those words gives a different limit compared to the translated text.

Note that if a paragraph triggers a warning then the paragraph should somehow be highlighted. It can be somewhat tedious to try to track down which paragraph triggers a warning… ;)

In T190279#4114812, @jeblad wrote:

Some languages are really close, and with good ml engines, and thus will produce a lot of translated text that doesn't need to be fixed...

The quality level of machine translation is different from language to language. In order to avoid issues, we want to be able to (a) adjust the thresholds to each case, and (b) allow users to "mark as resolved" as a way to confirm that the warning is not actually an issue for such case.

In T190279#4114814, @jeblad wrote:

Note that if a paragraph triggers a warning then the paragraph should somehow be highlighted. It can be somewhat tedious to try to track down which paragraph triggers a warning… ;)

The yellow marks next to each paragraph will indicate which paragraphs need review (unless the user explicitly marked them as resolved).

In T190279#4115737, @Pginer-WMF wrote:

The quality level of machine translation is different from language to language. In order to avoid issues, we want to be able to (a) adjust the thresholds to each case,

Have you considered to have it defined by the community?
Some communities may think that 80% of automated is acceptable for publication, while there are a lot of people ready to review, or if they prefer to have articles more than nothing.
Some other would ask for less than 40%.

That's a bit tricky, because giving that right to the communities may block some publications. Or be used to block those publications purposefully.

and (b) allow users to "mark as resolved" as a way to confirm that the warning is not actually an issue for such case.

Should it be authorized via a role?
A beginner shouldn't be able to mark it as resolved if it is a way to dismiss the message with no further consequences. When that beginner has made let's say 5 translations with enough quality, they should get the right to dismiss the message and publish.

Have you considered a way to block the publication if the threshold is not reached?

In T190279#4152745, @Trizek-WMF wrote:

Have you considered to have it defined by the community?

Yes. We expect thresholds to be different for different language pairs based on different factors, and people working on those wikis are in the best position to provide such information. What is not completely clear is how to best support that process. One simple initial approach is to start with a default and adjust it based on feedback. For the longer term we may want to explore how to expose the impact of those thresholds for communities to make informed decisions and adjust them.

and (b) allow users to "mark as resolved" as a way to confirm that the warning is not actually an issue for such case.

Should it be authorized via a role?
A beginner shouldn't be able to mark it as resolved if it is a way to dismiss the message with no further consequences. When that beginner has made let's say 5 translations with enough quality, they should get the right to dismiss the message and publish.

We are defining issues of two kinds: errors prevent publishing and warnings let you publish but recommend improving. Issues are of one or the other kind depending on how severe is the issue and how sure we can be about it actually happening. Discarding a warning means that the user considers the content is good enough already. If the user choses to ignore one recommendation, I think it is better to let them focus on other recommendations. Otherwise the warnings that they ignore but cannot discard may hide other warnings they may like to consider.

For the case of MT not being edited enough, translations published with too much MT will get an edit tag (T190798) to make them easier to review.

Have you considered a way to block the publication if the threshold is not reached?

It's always open to debate how strict we need to be in these cases. for example, at which point would be ok to prevent an article to be created (with regular editor starting from scratch or as a translation) because spellchecking finds issues in the content?

For the case of translations I don't think we need to restrict publishing much more than what you'd get in other editors. For cases where the unmodified content is really really high, we defined an error to prevent publishing them (T190283), but that is intended to prevent the extreme cases. Otherwise it can lead to false positives.

@Pginer-WMF While I was working on this ticket, I noticed that the warning will get triggered immediately after user click on a placeholder to start translation. Because it is 100% unmodified content. Please confirm if this is your expectation as well or not

In T190279#4414650, @santhosh wrote:

@Pginer-WMF While I was working on this ticket, I noticed that the warning will get triggered immediately after user click on a placeholder to start translation. Because it is 100% unmodified content. Please confirm if this is your expectation as well or not

Good catch. That's distraction we want to avoid, since the user may already have the intention to edit the paragraph. It would be better to defer the checking for a later moment when we have given the user enough time to review the content. Checking a paragraph when the input focus moves out, or when the user starts working on another paragraph (e.g., adding it to the translation or editing it) could be some options. As part of the publishing process we may want to make a final check too.

Change 444208 had a related patch set uploaded (by Santhosh; owner: Santhosh):
[mediawiki/extensions/ContentTranslation@master] Translation progress calculation

https://gerrit.wikimedia.org/r/444208

Change 444208 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] Translation progress calculation

https://gerrit.wikimedia.org/r/444208

@Pginer-WMF do we need to display warnings about crossing MT threshold when translation is loaded from in-progress? We currently lose that info when user goes back to dashboard and returns to the translation later.

We do have that section level information, but we delay it showing till some changes happen in the content. It is possible to identify whether a section is restored or not. Based on that we can avoid delaying the issue display.

In T190279#4453941, @Petar.petkovic wrote:

@Pginer-WMF do we need to display warnings about crossing MT threshold when translation is loaded from in-progress? We currently lose that info when user goes back to dashboard and returns to the translation later.

Thanks for pointing to this, @Petar.petkovic. As Santhosh indicated in T190279#4455212, this seems to be produced by the delay strategy we use for not getting in the way as the user edits the content. For the case where an in-progress translation is loaded from the dashboard, it is better to skip the "delay" and load the warnings immediately.

In T190279#4455212, @santhosh wrote:

We do have that section level information, but we delay it showing till some changes happen in the content. It is possible to identify whether a section is restored or not. Based on that we can avoid delaying the issue display.

I have tried to load 100% unmodified content from in progress and changing some section. That way only edited section gets "100% unmodified MT" warning and others don't, even though other sections have 100% unmodified content as well. That happens because sections are added to validation queue, that we have in progress tracker, only when section is changed. We need to display warnings when 100% MT is recognized during initialization of progress tracker.

@Petar.petkovic - I do not see any warnings for machine-translated content in cx2 - not when using MT (Apertium), not when I loaded saved translation with 100% MT translated content.

In T190279#4458266, @Etonkovidova wrote:

@Petar.petkovic - I do not see any warnings for machine-translated content in cx2 - not when using MT (Apertium), not when I loaded saved translation with 100% MT translated content.

Warnings are only displayed after you make an edit to avoid the situation where you get the warning immediately after adding new paragraph, if you use MT. Try adding some number of paragraphs by clicking on "Add translation". Then, edit any and you'll see the warnings for all of them.

In T190279#4459122, @Petar.petkovic wrote:

In T190279#4458266, @Etonkovidova wrote:

@Petar.petkovic - I do not see any warnings for machine-translated content in cx2 - not when using MT (Apertium), not when I loaded saved translation with 100% MT translated content.

Warnings are only displayed after you make an edit to avoid the situation where you get the warning immediately after adding new paragraph, if you use MT. Try adding some number of paragraphs by clicking on "Add translation". Then, edit any and you'll see the warnings for all of them.

The current approach to decide when to show warnings can still be polished further. I created a separate ticket to discuss a proposal: T200683: CX2: Adjust when to check warnings for a paragraph

@Pginer - this task seems to me as a tracking ticket (sort of) since it has subtasks (T200683 and T190798) that actually are part of the specs. So, please review the following summary - especially #2 and #3 to decide if some additional work needs to be done.

Summary of the issues that not done yet for this warning:

(1) The triggering point for the warning is not super intuitive; as per @Petar.petkovic comment - you need to actually edit some of the content to see the warning. But the issue will be addressed in T200683: CX2: Adjust when to check warnings for a paragraph

(2) There is no 'Mark as resolved' option. So, you cannot permanently dismiss the warning (which might be useful, actually). And there is no calculation how many issues are present. In the screenshots below there are two issues - the title exists already and too much of unmodified content, but there is no counter for the issues.

(3) The layout of the warning is different - the icon is black and the wording is different from the mockup and from the task's specs.

Machine translation is provided only as a starting point. You need to make sure that the content is accurate and reads naturally in your language.

(4)

The "Learn more" link will open a related documentation page on a new window/tab.

Currently, the link directs to https://en.wikipedia.org/wiki/Wikipedia:Translation#How_to_translate Would it be modified for other wikis? Don't we have more relevant information for editors-translators?

(5) T190798: CX2: Add to a maintenance category when translation is published with too much unmodified text is still not done.

In T190279#4463289, @Etonkovidova wrote:

@Pginer - this task seems to me as a tracking ticket (sort of) since it has subtasks (T200683 and T190798) that actually are part of the specs. So, please review the following summary - especially #2 and #3 to decide if some additional work needs to be done.

Thanks for the review. Here the infrastructure for supporting issues is being developed in parallel with some of the specific issues that such infrastructure supports. So there are missing parts that will be built in iterations. I think all the pending work is now captured in other tickets, so we can close this one.

(2) There is no 'Mark as resolved' option. So, you cannot permanently dismiss the warning (which might be useful, actually). And there is no calculation how many issues are present. In the screenshots below there are two issues - the title exists already and too much of unmodified content, but there is no counter for the issues.

There are two relevant tickets about adding the possibility to "mark as resolved" (T198188) which requires some storage to track the issues you marked that is not available yet, and the logic on when to show the issue cards (T198187).

(3) The layout of the warning is different - the icon is black and the wording is different from the mockup and from the task's specs.

Good catch. I created a separate ticket to adjust the message (T200768).

For the icon there is a technical limitation on how the icons are used, but I created also a ticket to help moving that forward (T200773).

(4)

The "Learn more" link will open a related documentation page on a new window/tab.

Currently, the link directs to https://en.wikipedia.org/wiki/Wikipedia:Translation#How_to_translate Would it be modified for other wikis? Don't we have more relevant information for editors-translators?

I'd expect the equivalent links (based on Wikidata) to open. Base on user research we can consider having a custom help page.