Page MenuHomePhabricator

Parse the config files for bad templates and categories
Closed, ResolvedPublic

Description

Motivation
The FileImporter is reusing the config files of the CommonsHelper2. In order to disallow file moves with unsuitable licences, we need to parse the bad template and bad category sections.

Acceptance Criteria

  • Given the correct config file for a file move candidate, parse out the names of templates and categories that should forbid the file move
  • If the configfile does not contain the section (such as category or template) or the subsection bad, return an error message: The configuration file for your wiki (linking to the config file) does not contain enough info about <problem e.g. Category or Category/bad> to know if this file can be put on Commons.

Notes

  • The names can be found in Categories/Bad and Templates/Bad in the config file
  • Investigation with proof of concept was done in T193620. This should be done using the raw wikitext, an d parsing it with regex
  • Needs trimming
  • The end result is an array with a list of all the names

Details

Related Gerrit Patches:
mediawiki/extensions/FileImporter : masterDon't fail CommonsHelper config parsing when sections are empty
mediawiki/extensions/FileImporter : masterRelax CommonsHelperConfigParser to allow "packed" wikitext
mediawiki/extensions/FileImporter : masterAdd test for CommonsHelperConfigParser
mediawiki/extensions/FileImporter : masterExtract category and template information from CommonsHelper's config files

Related Objects

Event Timeline

Lea_WMDE triaged this task as Medium priority.May 8 2018, 8:50 AM
Lea_WMDE created this task.
Lea_WMDE updated the task description. (Show Details)May 8 2018, 8:53 AM
Lea_WMDE updated the task description. (Show Details)May 8 2018, 12:42 PM
Lea_WMDE set the point value for this task to 5.

Change 434677 had a related patch set uploaded (by Andrew-WMDE; owner: Andrew-WMDE):
[mediawiki/extensions/FileImporter@master] Extract category and template information from CommonsHelper's config files

https://gerrit.wikimedia.org/r/434677

@Lea_WMDE Isn't it assumed that if a section is missing, such as Category or Category/Bad, there are no bad categories? So the following situation should never occur:

"If the configfile does not contain the section (such as category or template) or the subsection bad, return an error message: The configuration file for your wiki (linking to the config file) does not contain enough info about <problem e.g. Category or Category/bad> to know if this file can be put on Commons."

@Andrew-WMDE that is a nearly philosophical question. It might also mean that users misspelled the section title when touching the configfile. To prevent that, we are currently rather harsh (and see where it brings us in the long run). Small beta wikis will all have Category/Bad, though.

Change 437253 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] [WIP] Add test for CommonsHelperConfigParser

https://gerrit.wikimedia.org/r/437253

Change 437284 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Relax CommonsHelperConfigParser to allow "packed" wikitext

https://gerrit.wikimedia.org/r/437284

Change 437287 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Don't fail CommonsHelper config parsing when sections are empty

https://gerrit.wikimedia.org/r/437287

Change 434677 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Extract category and template information from CommonsHelper's config files

https://gerrit.wikimedia.org/r/434677

Change 437253 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Add test for CommonsHelperConfigParser

https://gerrit.wikimedia.org/r/437253

Change 437284 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Relax CommonsHelperConfigParser to allow "packed" wikitext

https://gerrit.wikimedia.org/r/437284

Change 437287 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Don't fail CommonsHelper config parsing when sections are empty

https://gerrit.wikimedia.org/r/437287

To tests: I configured https://mwfileimport.wmflabs.org/wiki/index.php/Special:ImportFile so it can be used with the CommonsHelper2 config files.

Vvjjkkii renamed this task from Parse the config files for bad templates and categories to uedaaaaaaa.Jul 1 2018, 1:11 AM
Vvjjkkii removed Andrew-WMDE as the assignee of this task.
Vvjjkkii raised the priority of this task from Medium to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed the point value for this task.
Vvjjkkii edited subscribers, added: Andrew-WMDE; removed: gerritbot, Aklapper.
Bodhisattwa renamed this task from uedaaaaaaa to Parse the config files for bad templates and categories.Jul 1 2018, 1:45 PM
Bodhisattwa assigned this task to Andrew-WMDE.
Bodhisattwa lowered the priority of this task from High to Medium.
Bodhisattwa updated the task description. (Show Details)
Bodhisattwa added subscribers: gerritbot, Aklapper.
thiemowmde closed this task as Resolved.Nov 27 2018, 2:21 PM
thiemowmde removed a project: Patch-For-Review.