Page MenuHomePhabricator

Remove templates as indicated by the config files
Closed, ResolvedPublic1 Story Points

Description

Acceptance Criteria

  • Remove all templates listed in the config files under Template/Remove
  • When a change (be it removal, or replacement) was made by us because of the config files, fill the edit summary line with <x> automatic changes to templates have been made

Note

  • For finding the brackets at the end, we can reuse the parser logic that skips nested templates

Details

Related Gerrit Patches:
mediawiki/extensions/FileImporter : wmf/1.32.0-wmf.13Fix flipped array indexes in template removal code
mediawiki/extensions/FileImporter : masterFix flipped array indexes in template removal code
mediawiki/extensions/FileImporter : masterRemove templates as indicated by the config files
mediawiki/extensions/FileImporter : masterCount template removals and add to summary line
mediawiki/extensions/FileImporter : masterRelax template parser return format

Related Objects

Mentioned In
T200837: PHP Notice: Undefined index from extensions/FileImporter/src/Services/WikiTextContentCleaner.php:83
rEFLI4568fda4221c: Fix flipped array indexes in template removal code
rEFLIaad713385b94: Fix flipped array indexes in template removal code
rEFLI0657f37e22ce: Fix flipped array indexes in template removal code
rEFLI9c484dfd476b: Fix flipped array indexes in template removal code
rEFLI7ed9b3a30eaf: Fix flipped array indexes in template removal code
T200406: Template removal removes too much
rEFLI565b83a5a77a: Count template removals and add to summary line
rEFLI5813d4bf034d: Count template removals and add to summary line
rEFLIecd3275e1d74: Count template removals and add to summary line
rEFLI828c4f2a9a3b: Remove templates as indicated by the config files
rEFLI155d13cf97b4: Remove templates as indicated by the config files
rEFLI9d45787a2e3f: Remove templates as indicated by the config files
rEFLI4ed132bfa410: Remove templates as indicated by the config files
rEFLI29a117d6840a: Remove templates as indicated by the config files
rEFLI92932f8aae1a: Remove templates as indicated by the config files
rEFLI285c80105690: Clean-up and fix bad validation in CommonsHelperConfigParser
rEFLI477b1acfa043: Relax template parser return format
rEFLI5c8862a7e36f: Remove templates as indicated by the config files
rEFLI50d1e1c0e49e: Clean-up and fix bad validation in CommonsHelperConfigParser
rEFLIf1f7c2a74645: Relax template parser return format
Mentioned Here
T196908: Beta test Move Files to Commons in Arabic Wikipedia
T198594: request to enable Move Files to Commons feature in sourceswiki

Event Timeline

Restricted Application added a project: TCB-Team. · View Herald TranscriptJul 3 2018, 1:51 PM
Lea_WMDE set the point value for this task to 5.Jul 3 2018, 1:51 PM

Change 443656 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Relax template parser return format

https://gerrit.wikimedia.org/r/443656

thiemowmde triaged this task as Low priority.
thiemowmde moved this task from Sprint Backlog to Doing on the WMDE-QWERTY-Sprint-2018-07-03 board.

Change 443814 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Remove templates as indicated by the config files

https://gerrit.wikimedia.org/r/443814

Open question: Should removals also appear in the summary line? Currently only replacements are counted.

  • We could count removals as replacements as well, and let the summary line say something like "2 automatic changes to templates have been made".
  • The summary could also list removals separately, and say something like "0 removals and 2 replacements have been made".

Change 443656 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Relax template parser return format

https://gerrit.wikimedia.org/r/443656

Open question: Should removals also appear in the summary line? Currently only replacements are counted.

  • We could count removals as replacements as well, and let the summary line say something like "2 automatic changes to templates have been made".
  • The summary could also list removals separately, and say something like "0 removals and 2 replacements have been made".

@Lea_WMDE

Change 443814 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Remove templates as indicated by the config files

https://gerrit.wikimedia.org/r/443814

Lea_WMDE updated the task description. (Show Details)Jul 13 2018, 9:09 AM

@thiemowmde @WMDE-Fisch good point!
Let's user 2 automatic changes to templates have been made for now - I edited the ticket description

Change 446251 had a related patch set uploaded (by WMDE-Fisch; owner: WMDE-Fisch):
[mediawiki/extensions/FileImporter@master] Count template removals and add to summary line

https://gerrit.wikimedia.org/r/446251

@thiemowmde @WMDE-Fisch good point!
Let's user 2 automatic changes to templates have been made for now - I edited the ticket description

While doing this, I wonder if we also should also write something, when we changed the section headings. Currently we only count template changes and removals. We could also add the changes of headlines to that number. The text then would more be like 1 automatic changes to the wikitext have been made ....

WMDE-Fisch changed the point value for this task from 5 to 1.
WMDE-Fisch moved this task from Sprint Backlog to Review on the WMDE-QWERTY-Sprint-2018-07-17 board.

Change 446251 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Count template removals and add to summary line

https://gerrit.wikimedia.org/r/446251

@Fisch-WMDE do you have a sample file? I'm struggling to find something to test it with

@Fisch-WMDE do you have a sample file? I'm struggling to find something to test it with

Sorry, my bad. - But let's use this awesome search interface I've heard of to find a fitting file :-D

List with templates to remove:
https://www.mediawiki.org/wiki/Extension:FileImporter/Data/de.wikipedia

Template to remove:
Commonsfähig

Searching for files with that template:
https://de.wikipedia.org/w/index.php?search=hastemplate%3ACommonsf%C3%A4hig+filetype%3Abitmap&title=Spezial:Suche&profile=advanced&fulltext=1&advancedSearch-current=%7B%22options%22%3A%7B%22filetype%22%3A%22bitmap%22%2C%22hastemplate%22%3A%5B%22Commonsf%C3%A4hig%22%5D%7D%2C%22namespaces%22%3A%5B6%5D%7D&ns6=1

File with template that should be removed:
https://de.wikipedia.org/wiki/Datei:K%C3%B6nigsforst_Waldlehrpfad_Rath.png

\o/ :-P

oh män, sorry, the interface rumor should have reached me too ;)

So thanks and it looks great!

Pikne added a subscriber: Pikne.Jul 24 2018, 12:10 PM

It seems that visible file info that comes before template set for removal is entirely or almost entirely removed along with this template. If I try to import a file (source) then the only text preserved is "Self" (name for another template replaced). At the end original file info has "viiCommonsisse" template which is set for removal in config file.

It seems that visible file info that comes before template set for removal is entirely or almost entirely removed along with this template. If I try to import a file (source) then the only text preserved is "Self" (name for another template replaced). At the end original file info has "viiCommonsisse" template which is set for removal in config file.

Thanks for the hint ... seems like to be something we have to look into. I will create a separate ticket for it.

Change 448565 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Fix flipped array indexes in template removal code

https://gerrit.wikimedia.org/r/448565

Change 448801 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@wmf/1.32.0-wmf.14] Fix flipped array indexes in template removal code

https://gerrit.wikimedia.org/r/448801

Change 448802 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@wmf/1.32.0-wmf.13] Fix flipped array indexes in template removal code

https://gerrit.wikimedia.org/r/448802

Change 448565 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Fix flipped array indexes in template removal code

https://gerrit.wikimedia.org/r/448565

Change 448802 abandoned by WMDE-Fisch:
Fix flipped array indexes in template removal code

Reason:
Since commons already uses the .14 branch we do not need this backport for .13

https://gerrit.wikimedia.org/r/448802

It seems that visible file info that comes before template set for removal is entirely or almost entirely removed along with this template. If I try to import a file (source) then the only text preserved is "Self" (name for another template replaced). At the end original file info has "viiCommonsisse" template which is set for removal in config file.

@Pikne: This should be fixed and deployed now. Thanks again for your report!

P.S.: @Pikne if you want to have the FileExporter[1] extension enabled as beta feature for the Estonian wiki. Just get an approval via community consensus, create a ticket and link the consensus there. :-)

See also T196908 or T198594

[1] You can then just use a convenient link on the file info pages to initiate the moving process.

Pikne added a comment.Jul 30 2018, 4:45 PM

P.S.: @Pikne if you want to have the FileExporter[1] extension enabled as beta feature for the Estonian wiki.

I don't know. I expect you'd later enable it anyway and so I haven't bothered to bring this up on wiki. Based on file moves done in the past there's probably little interest for this among other users. And for my own convenience I already did set up this export link browser-side. :)

Anyway, thanks for fixing this removal issue and work done on developing the tool altogether.

Lea_WMDE closed this task as Resolved.Aug 3 2018, 6:48 AM