Page MenuHomePhabricator
Create Task
Maniphest T213955

Add support for aliased parameter names in source-wiki templates
Closed, ResolvedPublic3 Estimated Story Points
Actions

Description

Some templates in the source wikis allow different spellings for the same parameter. For example, the Russian information template can say |Описание= or |описание= (note the different capitalization, which is significant). Both should become |Description=. The CommonsHelper2 format we use currently does not allow this, but can, in theory. See https://www.mediawiki.org/wiki/Special:Diff/3056489 for an example that would work if the code would support it.

Some of the code is already prepared for this, but this is neither fully implemented nor carefully tested.

Requested via:

Note this is very close to T198582 (this ticket here will make the other one easier), but not the same.

Details

SubjectRepoBranchLines +/-
Add WikitextContentCleaner test for multiple source parametersmediawiki/extensions/FileImportermaster+18 -0
Allow multiple source parameters map to same target parametermediawiki/extensions/FileImportermaster+48 -14
Customize query in gerrit

Related Objects

Event Timeline

thiemowmde created this task.Jan 16 2019, 7:21 PM
gerritbot subscribed.Jan 16 2019, 7:22 PM

Change 484747 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Allow multiple source parameters map to same target parameter

https://gerrit.wikimedia.org/r/484747

gerritbot added a project: Patch-For-Review.Jan 16 2019, 7:22 PM
thiemowmde mentioned this in rEFLIce54a499c794: Allow multiple source parameters map to same target parameter.Jan 16 2019, 7:24 PM
thiemowmde mentioned this in rEFLI20e5df6d2004: Allow multiple source parameters map to same target parameter.Jan 16 2019, 7:37 PM
stjn subscribed.Jan 16 2019, 8:08 PM
JStrodt_WMDE awarded a token.Jan 17 2019, 1:13 PM
jijiki mentioned this in rEFLIefb000cfae55: Allow multiple source parameters map to same target parameter.Mar 18 2019, 4:37 PM
gerritbot added a comment.Apr 2 2019, 6:28 PM

Change 484747 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Allow multiple source parameters map to same target parameter

https://gerrit.wikimedia.org/r/484747

thiemowmde mentioned this in rEFLI35b730d3b3c8: Allow multiple source parameters map to same target parameter.Apr 2 2019, 6:29 PM
thiemowmde triaged this task as Medium priority.Apr 26 2019, 11:26 AM
thiemowmde updated the task description. (Show Details)
• 4nn1l2 subscribed.Apr 26 2019, 12:03 PM
thiemowmde merged a task: T221959: ZWNJ and Whitespace not recognized in parameters.Apr 26 2019, 12:22 PM
thiemowmde added a subscriber: • MichaelSchoenitzer_WMDE.
thiemowmde mentioned this in rEFLIfc4cee733c24: Allow multiple source parameters map to same target parameter.May 5 2019, 8:21 AM
thiemowmde mentioned this in rEFLIa87b63682f43: Allow multiple source parameters map to same target parameter.May 7 2019, 4:42 PM
gerritbot added a comment.May 8 2019, 8:11 AM

Change 484747 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Allow multiple source parameters map to same target parameter

https://gerrit.wikimedia.org/r/484747

ReleaseTaggerBot added a project: MW-1.34-notes (1.34.0-wmf.5; 2019-05-14).May 8 2019, 9:00 AM
thiemowmde edited projects, added WMDE-TechWish, WMDE-QWERTY-Sprint-2019-04-30; removed Patch-For-Review.May 8 2019, 9:29 AM
thiemowmde moved this task from Backlog to Tickets in sprint on the Move-Files-To-Commons board.
thiemowmde moved this task from Incoming to In QWERTY sprint on the TCB-Team (now WMDE-TechWish) board.
thiemowmde claimed this task.May 8 2019, 10:04 AM
thiemowmde set the point value for this task to 3.
thiemowmde moved this task from Sprint Backlog to Doing on the WMDE-QWERTY-Sprint-2019-04-30 board.
thiemowmde added a subscriber: awight.

I noticed this feature is currently only tested at the data source (CommonsHelperConfigParser), not at the sink that uses the data (WikitextContentCleaner). I will add a unit test for the later.

This is also not covered by any integration test (the …Cleaner actually using the output from the …ConfigParser). But I feel this is acceptable and not strictly needed.

gerritbot added a comment.May 8 2019, 10:18 AM

Change 508797 had a related patch set uploaded (by Thiemo Kreuz (WMDE); owner: Thiemo Kreuz (WMDE)):
[mediawiki/extensions/FileImporter@master] Add WikitextContentCleaner test for multiple source parameters

https://gerrit.wikimedia.org/r/508797

gerritbot added a project: Patch-For-Review.May 8 2019, 10:18 AM
thiemowmde mentioned this in rEFLI2e676cff3331: Add WikitextContentCleaner test for multiple source parameters.May 8 2019, 10:20 AM
thiemowmde mentioned this in rEFLI2d52a53b0656: Add WikitextContentCleaner test for multiple source parameters.May 8 2019, 10:34 AM
thiemowmde moved this task from Doing to Review on the WMDE-QWERTY-Sprint-2019-04-30 board.May 8 2019, 11:01 AM
WMDE-Fisch moved this task from Review to Demo on the WMDE-QWERTY-Sprint-2019-04-30 board.May 8 2019, 5:02 PM
gerritbot added a comment.May 8 2019, 5:08 PM

Change 508797 merged by jenkins-bot:
[mediawiki/extensions/FileImporter@master] Add WikitextContentCleaner test for multiple source parameters

https://gerrit.wikimedia.org/r/508797

Lea_WMDE added a comment.May 9 2019, 11:18 AM

Could you just quickly remind me how I can test this outside of production?

thiemowmde added a comment.May 9 2019, 11:45 AM

Not super easy, I'm afraid, but possible.

  • You need a source wiki with a file page that contains a template with one or more parameter aliases, e.g. {{Information|desc=1|also desc=2}}. Note this is only an example. In reality, only one of the aliases would be present. But it's easier to test when they are all listed.
  • The config page for this source wiki must mention all aliases in the "Transfer" section, e.g. Information|Description=desc|Description=also desc.
  • After import, the template on the file page should now read {{Information|Description=1|Description=2}}.
awight added a comment.May 9 2019, 1:14 PM

I'm not sure how to get into the file importer on commonswiki to test this, but I did find an example file on dewiki which uses a template with parameter mapping:
https://de.wikipedia.org/wiki/Datei:GermanyDemRepPFX1-50Pfennig-1979-donatedmjd_b.jpg

According to this mapping, the Quelle parameter should become @Source, for example.

stjn added a comment.May 9 2019, 2:10 PM

If you need an example of a file for which conversion currently doesn’t work in production and should work with the patch, here:

https://ru.wikipedia.org/wiki/Файл:Портал_коми_news_inta.png (mapping for ruwiki)

(don’t upload the file on Commons, though, it’s useless)

Lea_WMDE closed this task as Resolved.May 15 2019, 12:50 PM
Lea_WMDE moved this task from Demo to Done on the WMDE-QWERTY-Sprint-2019-04-30 board.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL