For several years I have uploaded high quality photographs from the UK Ministry of Defence database and would like to move to url uploads rather than local caches. The upload links after a header redirect look like www.defenceimagery.mod.uk/fotoweb/cmdrequest/rest/Download.fwx/45153802.jpg - along with an auto-generated session cookie parameter.
I'm guessing that white-listing www.defenceimagery.mod.uk would be sufficient.
Example past images: Here
| Status | Assigned | Task | ||
|---|---|---|---|---|
| Resolved | None | T64820 Allow copy upload files from WMF sites on Wikimedia Commons | ||
| Resolved | Steinsplitter | T75724 Whitelisting domain for GWToolset | ||
| Open | None | T60224 Add domains to $wgCopyUploadsDomains (tracking) | ||
| Resolved | Framawiki | T166271 Please add www.defenceimagery.mod.uk to $wgCopyUploadsDomains |
Of what I could test, the cookie is mandatory. I don't know if it'll works, but let's try.
Change 355594 had a related patch set uploaded (by Framawiki; owner: Framawiki):
[operations/mediawiki-config@master] Add www.defenceimagery.mod.uk to CopyUploadsDomains
@Fae would you have an URL example so we can test how the access token behaves?
We can test now with a token generateed by another user.
It's look difficult to having these files with curl and cookies jars. Perhaps we can just ask them to allow our ips ?
I'm unclear as to why we are worried about tokens. If url upload is allowed, then the URL with a token passed as a parameter looks like:
http://www.defenceimagery.mod.uk/fotoweb/cmdrequest/rest/Download.fwx/45153802.jpg?D=EFCC51FEE65DA414D18085DA188CAB45524FFC4F7A63A403C47E17A8BEF1E554B796D6EA4FD91784A04B36049843E1FB56B129047A099FD2448D5AA2FD3EBB84D49852E5EF22F9F1E9930FDF2671F90028F4747E4DAAD3BE496BC62277DF33E1BC24AB66E7B4B90225B163F54F224DFE65DFE22A5F65B6D1328840103D2F128F615EE150C8AA32E00FC8DA1E13BEA266&ForceSaveDialog=no
I believe the system can upload from this address, so long as the domain is whitelisted. Naturally, the Commons image page would give a generic link to the source, rather than include session specific tokens.
If system sees the URL (tokens are valid worldwide), then it should be able
to upload. For me, link works.
If creating URL list is possible, why not. If it isn't, they must upload
images to other location anyway, so notify and whitelist.
Anyway, why we actually whitelist? Is there a Dos attack possible? Only
from our infrastructure i think but this can be solved by throttling upload
by URL actions by user. In some time maybe autoblock them.
...
Anyway, why we actually whitelist? Is there a Dos attack possible? Only
from our infrastructure i think but this can be solved by throttling upload
by URL actions by user. In some time maybe autoblock them.
This is an interesting point. Would it be worth creating a task to look at creating a user-whitelist for url uploads, where users appropriately experienced in batch uploads can be granted the right to run upload projects for any source?
It may be a solution. But I think much better would be simply give the upload from any source right to all current holders of upload_by_url right (no matter if we create separate upload_by_any_url and give it to all holders of upload_by_url by WMF config or simply remove the whitelist by conf variable). Why it can be destructive?
Change 355594 merged by jenkins-bot:
[operations/mediawiki-config@master] Add www.defenceimagery.mod.uk to CopyUploadsDomains