Page MenuHomePhabricator

CX2: Highlight references with a template that is missing mandatory parameters after being adapted
Closed, ResolvedPublic

Description

References and the templates often used to structure their information are important elements that cannot always be transferred appropriately across languages due to the diversity and fragmentation of templates across wikis.

There is a proposal (T203160) to highlight references with templates that could not be adapted at all resulting in empty references. The current ticket proposes to highlight in a similar way those references with a template that after the adaptation process they have empty parameters that are marked as mandatory.

Highlighting those could bring the attention to important missing information, that may go unnoticed by the user otherwise. For example. when translating "Börek" from Spanish to Catalan the third reference was added to the translation but only the Publisher parameter ("editorial") was mapped from the original Cita libro template in Spanish into the corresponding Ref-llibre template in Catalan:

SourceTranslation

Proposed solution

When a reference is added to the translation, but after the adaptation of its template it still has empty parameters that are marked as mandatory, the tool will show a warning for users to complete the missing information:

Incomplete reference
A reference was added without some required information.
The missing details could not be obtained automatically from the original reference since their structure is a bit different.
Please, edit the reference in the translation to fill the missing information.

  • References with a template with missing mandatory parameters will be rendered in grey.
  • "Learn more" link will point to the appropriate documentation page.
  • "Edit reference" action will edit the affected reference.
  • "Mark as resolved" will discard the issue.


Related tickets:

Details

Related Gerrit Patches:
operations/deployment-charts : masterUpdate cxserver to 2019-09-04-065911-production
mediawiki/services/cxserver : masterReferences: Pass additional data about adaptation status
mediawiki/extensions/ContentTranslation : masterDisplay warning for partially adapted references
mediawiki/extensions/ContentTranslation : masterSkip empty references when publishing translation
mediawiki/extensions/ContentTranslation : masterFix cx-tools-linter-template-missing-mandatory according to the design

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Pginer-WMF updated the task description. (Show Details)Nov 14 2018, 11:42 AM

Screenshot from above patch

Remaining:

"Edit reference" action will edit the affected reference.

This patch https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ContentTranslation/+/472925 just shows the adaptation status - turns the reference to gray.

Missing:

  1. handling these references while publishing
  2. Editing the reference and turning that back into blue
santhosh removed santhosh as the assignee of this task.Apr 2 2019, 8:35 PM
santhosh added a subscriber: santhosh.

I am unassigning this from me for the time being. The remaining items are listed above. @Pginer-WMF please note.

Petar.petkovic added a subscriber: Petar.petkovic.

There is a patch associated with this ticket that is still in review.

There is a patch associated with this ticket that is still in review.

@santhosh, I see there are comments for your patchset. Are you planning to reply to those and/or update the patchset accordingly? Should the ticket remain assigned to you until the code is merged?

Change 472925 had a related patch set uploaded (by Petar.petkovic; owner: Santhosh):
[mediawiki/extensions/ContentTranslation@master] Show adaptation status for references

https://gerrit.wikimedia.org/r/472925

Petar.petkovic removed a project: Patch-For-Review.

This ticket was added on 472925, but that patch only covers parts of T203160. Returning to priority backlog, where it previously was - T206310#5080915.

Pginer-WMF raised the priority of this task from Normal to High.May 21 2019, 7:31 AM

Change 521243 had a related patch set uploaded (by Amire80; owner: Amire80):
[mediawiki/extensions/ContentTranslation@master] Fix cx-tools-linter-template-missing-mandatory according to the design

https://gerrit.wikimedia.org/r/521243

Change 521243 abandoned by Petar.petkovic:
Fix cx-tools-linter-template-missing-mandatory according to the design

Reason:
Ibb00e5c8d22f1a30ddaf00217b762e97bca500c0 should be reviewed instead

https://gerrit.wikimedia.org/r/521243

Change 522010 had a related patch set uploaded (by Petar.petkovic; owner: Petar.petkovic):
[mediawiki/extensions/ContentTranslation@master] Skip empty references when publishing translation

https://gerrit.wikimedia.org/r/522010

Change 522010 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] Skip empty references when publishing translation

https://gerrit.wikimedia.org/r/522010

Moving this to QA to check the parts from T206310#4990683, but checking full specs would be good.

The main functionality that this ticket requires is not worked on. Some general parts for unadapted references are there, but feature to check for missing mandatory params inside reference templates was never added.
I was following instructions on missing parts from T206310#4990683 when I was moving this ticket to QA, but "Incomplete reference" feature is still not there yet.

I did find one nice example where we could test this case once implemented. It is Cite journal template on Bengali Wikipedia.
It has two required parameters, of which one has Latin script (title) alias, while other is only available in Bengali script (সাময়িকী). Also, there is no TemplateData for this template. Since first required parameter is available with title alias, that could almost always be mapped, plus some non-mandatory params get mapped. So, we end up with one required parameter that user needs to fill, which means warning from this ticket should appear.

This can be tested while translating en:Incumbent to Bengali.
By translating paragraph that begins with "A 2017 study in the", we get one mandatory param for first reference mapped (with 7 non-mandatory params mapped) and one mandatory param (সাময়িকী) not mapped.

Jpita added a subscriber: Jpita.Aug 16 2019, 11:56 AM

Moving to Pending waiting for @Pginer-WMF to decide what needs to be done here

Moving to Pending waiting for @Pginer-WMF to decide what needs to be done here

Based on @Petar.petkovic comment in T206310#5416676 it seems that the main functionality is not supported yet and the changes made are more specific to T203160 rather than the current ticket. So I'll move the ticket to the priority backlog for the development to continue.

@Petar.petkovic, please let me know if there are parts of the ticket description that needs to be adjusted to distinguish what is completed and what is still pending.

@Petar.petkovic, please let me know if there are parts of the ticket description that needs to be adjusted to distinguish what is completed and what is still pending.

No, ticket description does not need adjustment. We just got lost in long cycle of development and reviews on what's completed.
Many comments, like T206310#4819398 are suggesting that main parts of this ticket are completed long time ago, but it was wrong message being used as patch was dealing with T203160.
Functionality that this ticket requires was never actually worked on and will require some changes on cxserver as well. We did similar thing for regular templates, used outside of references.

No, ticket description does not need adjustment.

Thanks for the confirmation, and the additional details!

Change 532553 had a related patch set uploaded (by Petar.petkovic; owner: Petar.petkovic):
[mediawiki/services/cxserver@master] References: Pass additional data about adaptation status

https://gerrit.wikimedia.org/r/532553

Change 532554 had a related patch set uploaded (by Petar.petkovic; owner: Petar.petkovic):
[mediawiki/extensions/ContentTranslation@master] Display warning for partially adapted references

https://gerrit.wikimedia.org/r/532554

Change 532554 merged by jenkins-bot:
[mediawiki/extensions/ContentTranslation@master] Display warning for partially adapted references

https://gerrit.wikimedia.org/r/532554

Change 532553 merged by jenkins-bot:
[mediawiki/services/cxserver@master] References: Pass additional data about adaptation status

https://gerrit.wikimedia.org/r/532553

Change 534427 had a related patch set uploaded (by KartikMistry; owner: KartikMistry):
[operations/deployment-charts@master] Update cxserver to 2019-09-04-065911-production

https://gerrit.wikimedia.org/r/534427

Change 534427 merged by KartikMistry:
[operations/deployment-charts@master] Update cxserver to 2019-09-04-065911-production

https://gerrit.wikimedia.org/r/534427

@Jpita The patch is now deployed, you may proceed with testing.

Jpita closed this task as Resolved.Sep 9 2019, 3:46 PM
Jpita moved this task from Needs QA to Done on the Language-Team (Language-2019-July-September) board.
Jpita reopened this task as Open.Sep 10 2019, 4:02 AM
Jpita moved this task from Done to Needs QA on the Language-Team (Language-2019-July-September) board.
Jpita added a comment.Sep 10 2019, 5:21 AM

@Pginer-WMF @Petar.petkovic can't find any example for this, do you have any?

@Pginer-WMF @Petar.petkovic can't find any example for this, do you have any?

Yes, I wrote an example in T206310#5416676.

&page=Incumbent&from=en&to=bn


side by side comparison of both templates

&page=Incumbent&from=en&to=bn


side by side comparison of both templates

I see target template on the left and source on the right. Looks like Chrome's "Translate to English" is applied. Since it can be seen that mandatory param is missing and warning is shown, I assume your comment is positive.
You could try some more cases with Cite journal template which I described in T206310#5416676 or find some other example. I've seen Esperanto Wikipedia also having some templates with non-English param names.

Pginer-WMF added a comment.EditedSep 10 2019, 11:22 AM

So it seems that in the example, the "Journal" parameter from the source (on the right side of the screenshot) was not copied over, causing a mandatory parameter in the target template (in the screenshot labelled as "periodicals" due to a browser translation of the Bengali "সাময়িকী " parameter) to be empty. As a result the warning is shown, as expected, for the user to fill the mandatory parameter. Is my interpretation correct, @Jpita, @Petar.petkovic?

So it seems that in the example, the "Periodicals" parameter from the source was not copied over, causing a mandatory parameter in the target template (labelled as "periodicals") to be empty. As a result the warning is shown, as expected, for the user to fill the mandatory parameter. Is my interpretation correct, @Jpita, @Petar.petkovic?

On screenshots @Jpita posted, source is on the right and target is on the left. Target is in Bengali, but translated to English. Source does not have "Periodicals" param. That is translation of সাময়িকী param in Bengali Wikipedia's Cite journal. Since nothing can be mapped to সাময়িকী from source due to lack of English aliases, and that param is mandatory, warning is shown.

So it seems that in the example, the "Periodicals" parameter from the source was not copied over, causing a mandatory parameter in the target template (labelled as "periodicals") to be empty. As a result the warning is shown, as expected, for the user to fill the mandatory parameter. Is my interpretation correct, @Jpita, @Petar.petkovic?

On screenshots @Jpita posted, source is on the right and target is on the left. Target is in Bengali, but translated to English. Source does not have "Periodicals" param. That is translation of সাময়িকী param in Bengali Wikipedia's Cite journal. Since nothing can be mapped to সাময়িকী from source due to lack of English aliases, and that param is mandatory, warning is shown.

Thanks for the correction. I updated T206310#5478553 based on that.

I want to point out that warning is shown because সাময়িকী (periodicals in target template) is empty, not because "Journal" is not mapped. Even if target template had equivalent field named exactly the same (Journal), সাময়িকী would still not be mapped and warning would be shown.

I want to point out that warning is shown because সাময়িকী (periodicals in target template) is empty, not because "Journal" is not mapped. Even if target template had equivalent field named exactly the same (Journal), সাময়িকী would still not be mapped and warning would be shown.

Yes, the warning is shown because সাময়িকী is empty, but if CX were able to copy the info ("The Journal of Politics") from Journal into সাময়িকী, then সাময়িকী won't be empty and the warning won't be shown. Is that correct? Here I'm assuming that সাময়িকী and "Journal" are equivalent parameters since both of them seem to be about the name of the publication in each template.

I want to point out that warning is shown because সাময়িকী (periodicals in target template) is empty, not because "Journal" is not mapped. Even if target template had equivalent field named exactly the same (Journal), সাময়িকী would still not be mapped and warning would be shown.

Yes, the warning is shown because সাময়িকী is empty, but if CX were able to copy the info ("The Journal of Politics") from Journal into সাময়িকী, then সাময়িকী won't be empty and the warning won't be shown. Is that correct? Here I'm assuming that সাময়িকী and "Journal" are equivalent parameters since both of them seem to be about the name of the publication in each template.

Yes, if "Journal" were to be mapped to সাময়িকী, the warning would not be shown. I just don't know if those two are equivalent.

I want to point out that warning is shown because সাময়িকী (periodicals in target template) is empty, not because "Journal" is not mapped. Even if target template had equivalent field named exactly the same (Journal), সাময়িকী would still not be mapped and warning would be shown.

Yes, the warning is shown because সাময়িকী is empty, but if CX were able to copy the info ("The Journal of Politics") from Journal into সাময়িকী, then সাময়িকী won't be empty and the warning won't be shown. Is that correct? Here I'm assuming that সাময়িকী and "Journal" are equivalent parameters since both of them seem to be about the name of the publication in each template.

Yes, if "Journal" were to be mapped to সাময়িকী, the warning would not be shown. I just don't know if those two are equivalent.

Ok. Then, everything seems to work as expected. Thanks!

Jpita closed this task as Resolved.Sep 11 2019, 9:13 PM
Jpita moved this task from Needs QA to Done on the Language-Team (Language-2019-July-September) board.