Page MenuHomePhabricator
Create Task
Maniphest T140462

Correctly move files from Wikipedia to Commons (2013)
Closed, ResolvedPublic
Actions

Description

Survey: 2013

Original wish (German): https://de.wikipedia.org/wiki/Wikipedia:Umfragen/Technische_W%C3%BCnsche/Dateien
Update sites on-wiki:

Insights:

  • There are files that should stay on Wikipedia and not be moved to Commons, e.g. files under the fair use licence
  • Currently, the commonshelper tool is mainly used to move files to Commons
  • It is not about writing a bot to move many files quickly, but about writing a correct method for moving a single file

Plan:

  • Implement a Commons extension that moves files to Commons, keeping all data intact. This includes file revisions and text revisions as well as info about who moved the file. It also mentions the move in the version history. T146666
  • Link the new extension from Wikipedia file pages where appropriate T144480
  • Disable the link if moving is not encouraged T146751
  • all other open stories :)

Out of scope
For the wish, we will implement a hard coded mapping of well known matches of wikipedia and commons templates. Ideally, this should be configurable. Thoughts about how to do it are collected here: T143415

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedNoneT140462 Correctly move files from Wikipedia to Commons (2013)
 Resolved• Lea_WMDET140465 Research- what needs to be done to correctly move files form Wikipedia to Commons?
 InvalidNoneT140926 Investigate the technical requirements for adding the wished-for features to commonshelper
 ResolvedAddshoreT141867 Investigate how to move files with history from wikipedia to commons
Resolved• Lea_WMDET143098 Investigate how users should interact with the Commons-Mover-Tool
 DeclinedNoneT143414 Investigate how to undo moving a file to commons
 DuplicateNoneT143415 Investigate how to correctly move the file template info
 ResolvedNoneT144480 Move files from Wikipedia to Commons
 ResolvedAddshoreT160676 Open a new tab when users click on "Move to Commons"
 ResolvedAddshoreT160891 Only show move-link for autoconfirmed users
 ResolvedthiemowmdeT146666 Build a commons extension that moves files to Commons
 ResolvedAddshoreT156783 Create boilerplate for MoveToCommons extension
 ResolvedthiemowmdeT156792 Implement SpecialPage for FileImporter extension
 ResolvedAddshoreT160160 Handle duplicate titles gracefully
 ResolvedAddshoreT160166 Handle duplicate files gracefully
 ResolvedawightT156795 Mockup for FileImporter SpecialPage
 DeclinedNoneT160179 Preview option for changes made on SpecialPage
 ResolvedAndrew-WMDET160184 Allow editing the edit summary when changes have been made
 ResolvedAndrew-WMDET193004 Don't show edit summary if only the title was changed
 ResolvedAddshoreT161010 Implement NOJS SpecialPage based on mock
 ResolvedWMDE-FischT161197 Correct position & style of ImportFile edit title and edit file info buttons
 ResolvedAndrew-WMDET161615 Buttons on import preview page should have same size and font size as standard OOjs-UI
 ResolvedthiemowmdeT156797 Mechanism for switching templates
 ResolvedthiemowmdeT171605 Analyze template substitution mechanism of CommonsHelper and ForTheCommonGood tools
DuplicateNoneT156798 Implement the file-copy functionality
 ResolvedAddshoreT161011 Implement HTTP based file-copy functionality
 DeclinedAddshoreT161012 Implement http-less file-copy functionality
 ResolvedAddshoreT185671 Test file importer on beta
 ResolvedAddshoreT161013 Check in CentralAuth to see if users can be maintained on the import
 InvalidAddshoreT163499 Import user IDs for file revisions
 InvalidNoneT160180 Check if moving prerequisites are met
 ResolvedAndrew-WMDET160183 Add move to version history
 ResolvedAddshoreT171467 Make file info editable
 ResolvedNoneT161300 Investigate using the wikitetx editor for editing info from an import candidate file
 DeclinedNoneT161299 Investigate using the visual editor for editing info from an import candidate file
 DeclinedNoneT161301 Investigate using the new wikitext editor for editing file import candidates
 ResolvedAddshoreT160538 Investigate prerequisite checks of Commons File Upload page and add them to the File Importer, too
 ResolvedWMDE-FischT160540 When the target wiki has the abuse filter enabled, use it
 ResolvedWMDE-FischT190162 Install AbuseFilter on FileImporter test wiki and add a few meaningful rules
 ResolvedTobi_WMDE_SWT160543 Don't allow a file import if the user is less than autoconfirmed
 InvalidNoneT160544 Don't allow a file import if licence is Commons incompatible
 ResolvedAndrew-WMDET161014 Support continuation in ApiDetailRetriever
 ResolvedAddshoreT161016 Account for m.wikipedia.org domain name file sources
 ResolvedAddshoreT161126 Decide & Implement the behaviour for FileImporter & FileExporter when they come across files on another wiki
 ResolvedAndrew-WMDET162255 Decide & implement FileImporter behaviour when suppressed text revision data needs to be imported
 ResolvedAndrew-WMDET168406 On WMF wikis, call the FileExporter link "Import to Commons"
 ResolvedNoneT187470 Error handling in Move-Files-To-Commons
 InvalidNoneT180117 When an exception occurs, asks users to report the issue
 ResolvedAddshoreT180491 Have extensive logging for the import process
 DuplicateNoneT185735 When attempting to import a file for a file type not included in $wgFileExtensions the error is unclear
 InvalidNoneT185734 Improve error message when importing a file that is actually on a remote repo
 InvalidNoneT179564 Tracking how often files were (un)successfully imported and other stuff
 ResolvedWMDE-FischT189439 Error situations (wrong file format, abuse filter failure...) lead to an error page
 ResolvedNoneT189570 Show meaningful error messages if import fails
 ResolvedWMDE-FischT192295 Can't upload SVG files?
 ResolvedthiemowmdeT193614 Reuse Commonhelper2's config files for category and template matching
 ResolvedAndrew-WMDET193615 When a file move is started, find the correct config file for template and category mapping
 DeclinedNoneT198610 Cache API requests pulling CommonsHelper2 configuration files
 ResolvedthiemowmdeT196461 Make CommonsHelper2 source wiki address configurable
 ResolvedthiemowmdeT146751 Don't allow the moving of files with a wrong licence
 ResolvedWMDE-FischT193620 Investigate the parsing of config files for bad templates and categories
 ResolvedAndrew-WMDET193622 Investigate the parsing of the file info wikitext to forbid files with a wrong licence
 ResolvedAndrew-WMDET194129 Parse the config files for bad templates and categories
 ResolvedAndrew-WMDET194132 Parse the file info wikitext to forbid files with a wrong licence
 ResolvedAndrew-WMDET194502 Only allow moves of files that have one of the required templates
 ResolvedthiemowmdeT194505 Map templates to Commons templates whenever possible
 ResolvedthiemowmdeT198606 Add template parameters when requested via `+` syntax in the configuration
 ResolvedNoneT198609 Investigate if magic words `%AUTHOR%` and `%TRANSFERUSER%` are needed
 OpenNoneT198582 Default value for parameter in transferred templates
 ResolvedNoneT198607 Add language templates when requested via `@` syntax in the configuration
 InvalidNoneT198613 Template:Information not replaced on Commons
 DuplicateNoneT198617 Summary should be in the target wikis content language, not interface language
 ResolvedAndrew-WMDET198681 Template replacements are made 1 revision too early
 DeclinedNoneT198611 Provide a preview of the parsed CommonsHelper2 configuration
 Resolved• Lea_WMDET198315 Investigate what the "Template" part of the "Information" section in the config files is supposed to do
 ResolvedthiemowmdeT207502 Utilize Extension:FileExporter on Governance Wiki
 DuplicateNoneT147447 Delete moved file from home wiki
 ResolvedAddshoreT147451 Gracefully handle transfer failures
 DuplicateNoneT147452 Investigate the "name already exists on Commons" problem
 ResolvedAddshoreT156791 Create MoveToCommonsClient extension
 InvalidNoneT160169 Things to do when deploying to other wikis than de-wiki
 DeclinedWMDE-FischT160181 Display new location in source wiki
ResolvedAndrew-WMDET160182 Display original location in Commons
 Resolved• Lea_WMDET187532 Analytics: Insights into file moves and the usage of our tools
 ResolvedGoranSMilovanovicT187537 Usage metrics
 ResolvedAddshoreT187538 Provide data for usage metrics
 DeclinedNoneT187540 Track how many files receive the {{Original Upload Log}} template per day
 ResolvedNoneT190716 Deploying FileExporter and FileImporter
 ResolvedAddshoreT181383 Deploy FileImporter & FileExporter to beta sites
 ResolvedReedyT160982 Security review for FileImporter extension
 DuplicateNoneT179643 FileImporter import file URL entry field example url should include file extension
 ResolvedNoneT179645 FileImporter, importing https://commons.wikimedia.org/wiki/File:30th_Street_at_5th_Avenue_-_Stra%C3%9Fe_in_New_York.jpg fails
 ResolvedAnomieT184445 Broken image URL in API response when an image is missing from a file revision
 ResolvedAndrew-WMDET179647 FileImporter, Make limit of importable number revisions (file/file page) into a $wg
 OpenNoneT179646 FileImporter, Better duplicate handling when a file exists locally, Allowing re import / addition of missing revisions
 ResolvedReedyT158661 Security review for FileExporter extension
 ResolvedTobi_WMDE_SWT158072 Create extension manual for FileExporter and FileImporter
 ResolvedWMDE-FischT195370 Deploy FileExporter and FileImporter to group0
 ResolvedWMDE-FischT196969 Deploy FileExporter and FileImporter to de-,fa- and ar-wiki as a beta feature
 ResolvedNoneT196540 Enable FileImporter/FileExporter beta feature in Persian Wikipedia
 DuplicateNoneT196908 Beta test Move Files to Commons in Arabic Wikipedia
 ResolvedWMDE-FischT197066 Allow moves from mediawiki.org to Commons
 ResolvedNoneT213425 Deploy the FileExporter as a beta feature on all Wikimedia wikis
 ResolvedNoneT220309 FileImporter: allow more licenses
 DuplicateNoneT215381 Allow move files from Commons to Wikipedia

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
• Lea_WMDE moved this task from Incoming to Community Wishlist on the TCB-Team (now WMDE-TechWish) board.Jul 15 2016, 1:39 PM
• Lea_WMDE mentioned this in T141867: Investigate how to move files with history from wikipedia to commons .Aug 3 2016, 10:24 AM
• Lea_WMDE mentioned this in T140926: Investigate the technical requirements for adding the wished-for features to commonshelper.Aug 3 2016, 10:40 AM
• Lea_WMDE added subtasks: T140926: Investigate the technical requirements for adding the wished-for features to commonshelper, T141867: Investigate how to move files with history from wikipedia to commons .
• Lea_WMDE closed subtask T141867: Investigate how to move files with history from wikipedia to commons as Resolved.Aug 16 2016, 11:49 AM
• Lea_WMDE closed subtask T140465: Research- what needs to be done to correctly move files form Wikipedia to Commons? as Resolved.Aug 16 2016, 12:42 PM
• Lea_WMDE closed subtask T140926: Investigate the technical requirements for adding the wished-for features to commonshelper as Invalid.Aug 16 2016, 12:45 PM
• Lea_WMDE created subtask T143098: Investigate how users should interact with the Commons-Mover-Tool.Aug 16 2016, 12:54 PM
• Lea_WMDE updated the task description. (Show Details)Aug 16 2016, 1:02 PM
• Lea_WMDE edited projects, added TCB-Team (now WMDE-TechWish) (Oct2016-March2017); removed TCB-Team (now WMDE-TechWish).Aug 18 2016, 10:48 AM
• Lea_WMDE moved this task from Backlog to Prio2 on the TCB-Team (now WMDE-TechWish) (Oct2016-March2017) board.
• Lea_WMDE updated the task description. (Show Details)Aug 18 2016, 12:23 PM
• Jan_Dittrich subscribed.Aug 18 2016, 2:49 PM

Some more information on moving files:
(German) https://de.wikipedia.org/wiki/Hilfe:Dateien_nach_Commons_verschieben
(English) https://en.wikipedia.org/wiki/Wikipedia:Moving_files_to_Commons

• Lea_WMDE moved this task from Backlog to In Research Phase on the German-Community-Wishlist-Main-Wishes board.Aug 18 2016, 4:05 PM
• Lea_WMDE updated the task description. (Show Details)Aug 19 2016, 1:42 PM
• Lea_WMDE added a subtask: T143414: Investigate how to undo moving a file to commons.Aug 19 2016, 1:55 PM
• Lea_WMDE created subtask T143415: Investigate how to correctly move the file template info.Aug 19 2016, 2:04 PM
• Lea_WMDE updated the task description. (Show Details)Aug 19 2016, 2:06 PM
• Lea_WMDE created subtask T144480: Move files from Wikipedia to Commons.Sep 1 2016, 10:14 AM
• Lea_WMDE moved this task from In Research Phase to Backlog on the German-Community-Wishlist-Main-Wishes board.Sep 23 2016, 3:21 PM
• Lea_WMDE moved this task from Backlog to In Research Phase on the German-Community-Wishlist-Main-Wishes board.
• Lea_WMDE closed subtask T143098: Investigate how users should interact with the Commons-Mover-Tool as Resolved.Sep 26 2016, 12:59 PM
• Lea_WMDE created subtask T146666: Build a commons extension that moves files to Commons.Sep 26 2016, 4:40 PM
• Lea_WMDE updated the task description. (Show Details)
• Lea_WMDE updated the task description. (Show Details)Sep 27 2016, 10:30 AM
• Lea_WMDE created subtask T146751: Don't allow the moving of files with a wrong licence.Sep 27 2016, 10:57 AM
• Lea_WMDE updated the task description. (Show Details)
• Lea_WMDE updated the task description. (Show Details)Sep 27 2016, 10:59 AM
• Lea_WMDE created subtask T147447: Delete moved file from home wiki.Oct 5 2016, 3:29 PM
• Lea_WMDE created subtask T147451: Gracefully handle transfer failures.Oct 5 2016, 3:40 PM
• Lea_WMDE created subtask T147452: Investigate the "name already exists on Commons" problem.Oct 5 2016, 3:47 PM
• Lea_WMDE closed subtask T143414: Investigate how to undo moving a file to commons as Declined.Oct 5 2016, 3:53 PM
• Lea_WMDE updated the task description. (Show Details)Oct 5 2016, 3:59 PM
• Lea_WMDE updated the task description. (Show Details)Oct 6 2016, 5:01 PM
• Lea_WMDE updated the task description. (Show Details)Oct 7 2016, 2:02 PM
Tobi_WMDE_SW moved this task from Oct2016-March2017 to Move files to Commons on the TCB-Team (now WMDE-TechWish) board.Jan 31 2017, 10:11 AM
Tobi_WMDE_SW edited projects, added TCB-Team (now WMDE-TechWish); removed TCB-Team (now WMDE-TechWish) (Oct2016-March2017).
Tobi_WMDE_SW created subtask T156791: Create MoveToCommonsClient extension.Jan 31 2017, 1:10 PM
Tobi_WMDE_SW moved this task from In Research Phase to Ready for Development on the German-Community-Wishlist-Main-Wishes board.Feb 7 2017, 10:42 AM
Addshore closed subtask T156791: Create MoveToCommonsClient extension as Resolved.Feb 9 2017, 1:18 PM
Fastily subscribed.Feb 11 2017, 9:57 AM
Tobi_WMDE_SW moved this task from Ready for Development to Feature Development on the German-Community-Wishlist-Main-Wishes board.Feb 20 2017, 2:30 PM
• Lea_WMDE updated the task description. (Show Details)Mar 8 2017, 3:57 PM
• Lea_WMDE updated the task description. (Show Details)
• Lea_WMDE removed a subtask: T146751: Don't allow the moving of files with a wrong licence.Mar 10 2017, 1:21 PM
• Lea_WMDE created subtask T160169: Things to do when deploying to other wikis than de-wiki.Mar 10 2017, 1:43 PM
• Lea_WMDE created subtask T160181: Display new location in source wiki .Mar 10 2017, 5:34 PM
• Lea_WMDE created subtask T160182: Display original location in Commons.Mar 10 2017, 5:36 PM
• Lea_WMDE changed the status of subtask T160182: Display original location in Commons from Open to Stalled.Mar 15 2017, 1:02 PM
• Lea_WMDE changed the status of subtask T160181: Display new location in source wiki from Open to Stalled.Mar 15 2017, 3:39 PM
Tobi_WMDE_SW mentioned this in Move-Files-To-Commons.Mar 15 2017, 3:56 PM
Tobi_WMDE_SW added a project: Move-Files-To-Commons.Mar 15 2017, 3:59 PM
matmarex subscribed.Mar 27 2017, 6:37 PM

It would be way cool to also allow moving files in the opposite direction, from Commons to a specific Wikipedia. For files which are uploaded to Commons, used in articles, but later discovered to be improperly licensed (but okay under US fair use, or okay under a specific country's law), or simply for reverting erroneous moves in the other direction.

Addshore subscribed.Mar 27 2017, 10:46 PM
In T140462#3134959, @matmarex wrote:

It would be way cool to also allow moving files in the opposite direction, from Commons to a specific Wikipedia. For files which are uploaded to Commons, used in articles, but later discovered to be improperly licensed (but okay under US fair use, or okay under a specific country's law), or simply for reverting erroneous moves in the other direction.

Interesting. I doubt this use case will make it into the first deployed version, but in theory the extensions can work in both ways if deployed on both / all sites.

Geagea subscribed.May 7 2017, 10:59 PM
Addshore closed subtask T147451: Gracefully handle transfer failures as Resolved.Jul 12 2017, 9:57 AM
• Lea_WMDE moved this task from Backlog to Watching on the Move-Files-To-Commons board.Jul 31 2017, 9:15 AM
• Lea_WMDE changed the status of subtask T160182: Display original location in Commons from Stalled to Open.Jul 31 2017, 9:37 AM
• Lea_WMDE changed the status of subtask T160181: Display new location in source wiki from Stalled to Open.
Tobi_WMDE_SW closed subtask T160182: Display original location in Commons as Resolved.Nov 1 2017, 7:38 AM
• Lea_WMDE updated the task description. (Show Details)Nov 2 2017, 11:31 AM
jeblad subscribed.Nov 22 2017, 11:06 AM
• Ramsey-WMF subscribed.Jan 12 2018, 10:54 PM
• Lea_WMDE closed subtask T160181: Display new location in source wiki as Declined.Mar 12 2018, 9:22 AM
• Lea_WMDE added a subtask: T190716: Deploying FileExporter and FileImporter.Mar 26 2018, 7:22 PM
ToBeFree awarded a token.Jun 12 2018, 8:50 PM
ToBeFree subscribed.
Liuxinyu970226 awarded a token.Jun 25 2018, 12:01 PM
Liuxinyu970226 subscribed.
Shizhao mentioned this in T44473: Allow upload-by-URL from upload.wikimedia.org.Jun 26 2018, 7:16 AM
eranroz subscribed.Aug 3 2018, 8:16 PM
In T140462#3134959, @matmarex wrote:

It would be way cool to also allow moving files in the opposite direction, from Commons to a specific Wikipedia. For files which are uploaded to Commons, used in articles, but later discovered to be improperly licensed (but okay under US fair use, or okay under a specific country's law), or simply for reverting erroneous moves in the other direction.

+1 for matmarex idea. I recently discussed about another useful usecase with @Geagea - (non simple) logos - which sometimes doesn't fit to commons, but can be used with fair use license in Wikipedia.

Geagea added a comment.Aug 4 2018, 10:57 AM

As a matter a fact allow to move between wikis might be also good idea. If en.wiki uploaded a logo it might be usefull also as a logo to othe wikis.

Hispano76 subscribed.Aug 27 2018, 9:26 PM

My question is similar to the previous ones: Is it going to be possible to change the destination wiki or will it be Wikimedia Commons by default?

• Lea_WMDE closed subtask T187532: Analytics: Insights into file moves and the usage of our tools as Resolved.Jan 25 2019, 1:00 PM
• Lea_WMDE moved this task from Feature Development to Done on the German-Community-Wishlist-Main-Wishes board.Feb 5 2019, 3:12 PM
• Lea_WMDE moved this task from Done to Beta Feature Status / Feature Freeze on the German-Community-Wishlist-Main-Wishes board.
eranroz mentioned this in T215381: Allow move files from Commons to Wikipedia.Feb 6 2019, 7:15 AM
eranroz added a comment.Feb 6 2019, 7:17 AM
In T140462#4536396, @Wiki-1776 wrote:

My question is similar to the previous ones: Is it going to be possible to change the destination wiki or will it be Wikimedia Commons by default?

I think the scope of this task is only about one direction (to Commons). However, the opposite direction is also useful (though with different priority) and I moved it as a separate task T215381 which can be tracked separately

eranroz reopened subtask T215381: Allow move files from Commons to Wikipedia as Open.Feb 6 2019, 7:18 AM
• Lea_WMDE added a comment.Feb 11 2019, 8:08 AM

Hi all,
this project is about moving files from other Wikimedia projects to Commons. However, the code is designed in a way that it would allow moves from any wiki to any other wiki as well. For now, we are working on bringing FileImporter/FileExporter fully to production. Once that is done, anybody who feels like extending the extension for other directionalities, is appreciated :)

Fastily unsubscribed.Feb 11 2019, 7:23 PM
thiemowmde closed subtask T146666: Build a commons extension that moves files to Commons as Resolved.May 17 2019, 9:11 AM
WMDE-Fisch closed subtask T160169: Things to do when deploying to other wikis than de-wiki as Invalid.Mar 6 2020, 8:56 AM
Lena_WMDE moved this task from Beta Feature Status / Feature Freeze to Done on the German-Community-Wishlist-Main-Wishes board.Oct 16 2020, 9:03 AM
Nintendofan885 subscribed.Jun 7 2021, 8:43 AM
thiemowmde closed subtask T144480: Move files from Wikipedia to Commons as Resolved.Jan 7 2022, 12:28 PM
thiemowmde closed subtask T190716: Deploying FileExporter and FileImporter as Resolved.Jan 7 2022, 12:46 PM
thiemowmde closed this task as Resolved.Jan 7 2022, 12:51 PM
thiemowmde edited projects, added German-Community-Wishlist; removed TCB-Team (now WMDE-TechWish), German-Community-Wishlist-Main-Wishes.Jan 7 2022, 12:58 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits