To avoid unneeded traffic in areas of the cluster we want to be able to copy the files directly without http.
|Open||None||T140462 Correctly move files from Wikipedia to Commons (2013)|
|Open||None||T146666 Build a commons extension that moves files to Commons|
|Duplicate||None||T156798 Implement the file-copy functionality|
|Stalled||Addshore||T161012 Implement http-less file-copy functionality|
So, my notes from the hackathon are:
Not 100% sure about the impact this will have on the current method of using the built in mediawiki importing stuff :/
@fgiunchedi so the bit of code to look at is the commit method of https://gerrit.wikimedia.org/r/#/c/364253/11/src/Operations/FileRevisionFromForeignDbFile.php
This calls LocalFile::upload() with the backend storage path of the file to copy
This ends up calling the following AFAIK. LocalFile::publish -> LocalFile::publishTo -> FileRepo::quickImport -> FileRepo::quickImportBatch (which is where the actual backend operations are created.
From the testing that I have done this should all work, although I never managed to get any sort of swift setup similar to that in production locally or on labs to test this out.