Page MenuHomePhabricator
Create Task
Maniphest T194132

Parse the file info wikitext to forbid files with a wrong licence
Closed, ResolvedPublic
Actions

Description

Motivation
The CommonsHelper's config files tell us which categories and templates should prohibit a file move. We want to find out if one of the forbidden categories and templates exist in the most current version of the file page, to be able to disallow the move.

Acceptance Criteria

  • If the most current version of the file move candidate contains one of the categories or templates as defined in T194129, disallow the file move.
  • The user should be redirected to the error page. The message should be as defined in T189570 under "wrong licence":

The file cannot be imported, because it is marked as <template name/ category name>. <The target wiki name> does not allow such files.

  • If there is no forbidden template or category, continue with the file move without any further comment.
  • This happens before we enter the file preview page.

Notes

  • Investigation and proof of concept was done in T193622. We should get the lists of templates and categories from the API of the source wiki. The above ticket contains a proof of concept patch

Details

SubjectRepoBranchLines +/-
Make WikiTextConversions comparisons case-insensitivemediawiki/extensions/FileImportermaster+83 -17
Add test for WikiTextContentValidatormediawiki/extensions/FileImportermaster+44 -0
Forbid the import of a file that contains a blocked Category/Templatemediawiki/extensions/FileImportermaster+170 -13
Customize query in gerrit

Related Objects
Search...

Event Timeline

Lea_WMDE triaged this task as Medium priority.May 8 2018, 8:56 AM
Lea_WMDE created this task.
Lea_WMDE mentioned this in T189570: Show meaningful error messages if import fails.
Lea_WMDE updated the task description. (Show Details)May 8 2018, 12:54 PM
Lea_WMDE set the point value for this task to 3.
Lea_WMDE moved this task from Backlog to Tickets ready for pickup on the Move-Files-To-Commons board.May 11 2018, 12:15 PM
thiemowmde moved this task from Incoming to Move files to Commons on the TCB-Team (now WMDE-TechWish) board.May 11 2018, 5:07 PM
Lea_WMDE added a project: WMDE-QWERTY-Sprint-2018-05-23.May 23 2018, 1:40 PM
Lea_WMDE moved this task from Tickets ready for pickup to Tickets in sprint on the Move-Files-To-Commons board.
Andrew-WMDE claimed this task.May 30 2018, 10:59 AM
Andrew-WMDE moved this task from Sprint Backlog to Doing on the WMDE-QWERTY-Sprint-2018-05-23 board.
gerritbot subscribed.May 30 2018, 2:08 PM

Change 436291 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[mediawiki/extensions/FileImporter@master] Forbid the import of a file that contains a blocked Category/Template

https://gerrit.wikimedia.org/r/436291

gerritbot added a project: Patch-For-Review.May 30 2018, 2:08 PM
Andrew-WMDE mentioned this in rEFLI5dd38dfcd1ae: Forbid the import of a file that contains a blocked Category/Template.May 30 2018, 2:10 PM
Andrew-WMDE mentioned this in rEFLI3e4e0c7267fd: Forbid the import of a file that contains a blocked Category/Template.May 30 2018, 2:50 PM
Andrew-WMDE mentioned this in rEFLI6a2f222a050d: Forbid the import of a file that contains a blocked Category/Template.May 30 2018, 2:56 PM
Andrew-WMDE moved this task from Doing to Review on the WMDE-QWERTY-Sprint-2018-05-23 board.May 30 2018, 3:01 PM
thiemowmde mentioned this in rEFLI2670b930761a: Forbid the import of a file that contains a blocked Category/Template.Jun 4 2018, 10:00 AM
gerritbot added a comment.Jun 4 2018, 2:00 PM

Change 437248 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Add test for WikiTextContentValidator

https://gerrit.wikimedia.org/r/437248

gerritbot added a comment.Jun 4 2018, 2:01 PM

Change 437246 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Make WikiTextConversions comparisons case-insensitive

https://gerrit.wikimedia.org/r/437246

thiemowmde mentioned this in rEFLIedb3b52bc57e: Make WikiTextConversions comparisons case-insensitive.Jun 4 2018, 2:02 PM
thiemowmde mentioned this in rEFLIe6d3e2f6f9b2: Add test for WikiTextContentValidator.
gerritbot added a comment.Jun 5 2018, 9:24 AM

Change 436291 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Forbid the import of a file that contains a blocked Category/Template

https://gerrit.wikimedia.org/r/436291

thiemowmde mentioned this in rEFLI70cd7f5056ae: Make WikiTextConversions comparisons case-insensitive.Jun 5 2018, 9:28 AM
ReleaseTaggerBot added a project: MW-1.32-notes (WMF-deploy-2018-06-05 (1.32.0-wmf.7)).Jun 5 2018, 10:00 AM
WMDE-Fisch mentioned this in rEFLIb8f3a6a329af: Add test for WikiTextContentValidator.Jun 5 2018, 10:02 AM
WMDE-Fisch mentioned this in rEFLI684884b85101: Add test for WikiTextContentValidator.
gerritbot added a comment.Jun 5 2018, 10:13 AM

Change 437248 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Add test for WikiTextContentValidator

https://gerrit.wikimedia.org/r/437248

gerritbot added a comment.Jun 5 2018, 10:13 AM

Change 437246 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Make WikiTextConversions comparisons case-insensitive

https://gerrit.wikimedia.org/r/437246

WMDE-Fisch moved this task from Review to Demo on the WMDE-QWERTY-Sprint-2018-05-23 board.Jun 5 2018, 11:34 AM
WMDE-Fisch added subscribers: thiemowmde, WMDE-Fisch.Jun 11 2018, 12:01 PM

When I use https://mwfileimport.wmflabs.org/wiki/index.php/Special:ImportFile with https://en.wikipedia.org/wiki/File:2018_FIBA_Under-18_Americas_Championship.png I get

The configuration file for your wiki (https://meta.wikimedia.org/wiki/CommonsHelper2/Data_en.wikipedia) does not contain enough info about <Templates> to know if this file can be put on Commons.

So https://meta.wikimedia.org/wiki/CommonsHelper2/Data_en.wikipedia is missing a headline for templates so we stop the process and show that error. I just want to make sure if this is what we want. Enforcing the rough structure even when no Templates are needed/wanted to be configured.

@thiemowmde @Lea_WMDE

WMDE-Fisch added a comment.Jun 11 2018, 12:05 PM

Ok I just realized that this error is due to T194502 - we might want to change that, since en.wp seems to only rely on good categories not templates.

thiemowmde added a comment.Jun 11 2018, 12:47 PM

The file https://meta.wikimedia.org/wiki/CommonsHelper2/Data_en.wikipedia was incomplete because of an edit by an anonymous user that almost blanked the page. I restored the previous version.

I think we should decide if we want some of the sections we currently expect in each config file to be optional. For example, I think the two "Bad" sections need to be mandatory, while the "Good" sections can be optional. That would be convenient. On the other hand, it might be inconsistent and confusing. Maybe it's better to have all sections mandatory.

Lea_WMDE added a comment.Jun 12 2018, 7:31 AM

Thanks for fixing the page @thiemowmde! For now, let's stick to the decided behavior, and in the process of people using the file we will see if it makes sense to loosen the restrictions.

Lea_WMDE added a comment.Jun 12 2018, 8:22 AM

I got a nice error message, only the wording is different from what it says in the ticket, instead it says The file cannot be imported, because it is not marked with one of the required licences. Please add one of these templates, if appropriate.

WMDE-Fisch added a comment.Jun 12 2018, 8:26 AM
In T194132#4274667, @Lea_WMDE wrote:

I got a nice error message, only the wording is different from what it says in the ticket, instead it says The file cannot be imported, because it is not marked with one of the required licences. Please add one of these templates, if appropriate.

I assume when you get this error message the error is not due to this ticket but due to T194502 :-)

Lea_WMDE added a comment.Jun 12 2018, 8:27 AM

good point :) Can you point me to a file that actually triggers this error?

WMDE-Fisch added a comment.Jun 12 2018, 8:31 AM

That could be a bit tricky, because it means I have to find a file, that has a template marking it as "good" but also one marking it as "bad" ... if I get how these checks work now ... ^^

• Vvjjkkii renamed this task from Parse the file info wikitext to forbid files with a wrong licence to redaaaaaaa.Jul 1 2018, 1:11 AM
• Vvjjkkii removed Andrew-WMDE as the assignee of this task.
• Vvjjkkii raised the priority of this task from Medium to High.
• Vvjjkkii added projects: CheckUser, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), Tamil-Sites, Gamepress, Hashtags, Jade, KartoEditor, Language-2018-Apr-June, New-Editor-Experiences, Mail.
• Vvjjkkii updated the task description. (Show Details)
• Vvjjkkii removed the point value for this task.
• Vvjjkkii edited subscribers, added: Andrew-WMDE; removed: gerritbot, Aklapper.
Bodhisattwa renamed this task from redaaaaaaa to Parse the file info wikitext to forbid files with a wrong licence.Jul 1 2018, 2:00 PM
Bodhisattwa assigned this task to Andrew-WMDE.
Bodhisattwa lowered the priority of this task from High to Medium.
Bodhisattwa removed projects: Mail, New-Editor-Experiences, Language-2018-Apr-June, KartoEditor, Jade, Hashtags, Gamepress, Tamil-Sites, Connected-Open-Heritage-Batch-uploads (RAÄ-KMB_1_2017-02), CheckUser.
Bodhisattwa updated the task description. (Show Details)
Bodhisattwa added subscribers: gerritbot, Aklapper.
WMDE-Fisch added a comment.Mar 26 2019, 1:56 PM

@Lea_WMDE Can you please recheck if this is done from your perspective?

WMDE-Fisch closed this task as Resolved.Mar 26 2019, 2:10 PM

Lea approved ;-).

WMDE-Fisch mentioned this in T198372: Make sure people understand which template prevents them to moving a file to Commons.Nov 12 2020, 11:19 AM
Maintenance_bot removed a project: Patch-For-Review.Nov 12 2020, 12:10 PM
thiemowmde removed a project: TCB-Team (now WMDE-TechWish).Jan 7 2022, 2:59 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL