Page MenuHomePhabricator

UploadWizard on Wikimedia Commons no longer able to import files from Flickr
Closed, ResolvedPublicBUG REPORT

Description

There are multiple reports on Commons of UploadWizard no longer being able to import files from Flickr:

I haven't done an in-depth debugging; but from attempting to use UploadWizard's Flickr import functionality myself just now, it seems like a call to https://api.flickr.com/services/rest/ may be getting blocked by the recently-implemented Content Security policy?

screenshot.png (188×1 px, 53 KB)

Event Timeline

Do we need anything beyond api.flickr.com? Ideally, we'd like to be as limiting as possible and avoid things like *.flickr.com whenever feasible.

(I defer to other folks on that question, I do not have enough familiarity with UploadWizard to feel confident answering it myself)

Another question : Can the CSP limit the requestable media-types, or reject 'incorrect' types, given in a response?

We are now supporting api.flickr.com within the enforcing CSP, which hopefully unblocks this issue. (relevant config patch)

Another question : Can the CSP limit the requestable media-types, or reject 'incorrect' types, given in a response?

Yes, ideally we should further restrict this where possible.

FWICS it seems to now work for me (I didn't fully upload a file, but I went some of the way through the UploadWizard Flickr journey).
I left a comment on Commons asking if the folks that reported this issue can retry their Flickr uploads -- if it now works for them as well, we can close this IMO :)

So for longer term: given that this functionality from an extension (UploadWizard), where the API url is a config setting, there should probably be a hook handler in UploadWizard adding the required terms to the CSP instead of having that rely on correct configuration option set by the site administrator?

I left a comment on Commons asking if the folks that reported this issue can retry their Flickr uploads -- if it now works for them as well, we can close this IMO :)

Update: it seems like it's half-solved the problem, thx @sbassett! But it turns out that, when an end-user gives UploadWizard the URL to a Flickr album (e.g. https://www.flickr.com/photos/lucasartoni/albums/72157711331436398/), there's also an issue with (at least) live.staticflickr.com.
It seems like UploadWizard might try to load thumbnails from Flickr in order to ask the user which of the files in the album they'd like to upload -- however, as the thumbnails seem like they're being blocked by CSP, all that's currently being displayed on this screen seem to be the checkboxes.

(Example URL: https://live.staticflickr.com/65535/48896090016_30935590af_s.jpg)

screenshot.png (384×1 px, 139 KB)
screenshot.png (602×1 px, 75 KB)

Change #1248981 had a related patch set uploaded (by TheDJ; author: TheDJ):

[mediawiki/extensions/UploadWizard@master] Flickr: Add flick url to CSP directives

https://gerrit.wikimedia.org/r/1248981

Change #1249348 had a related patch set uploaded (by SBassett; author: SBassett):

[operations/puppet@production] Allow-list some additional domains to the currently enforcing CSP

https://gerrit.wikimedia.org/r/1249348

Change #1249348 merged by Scott French:

[operations/puppet@production] Allow-list some additional domains to the currently enforcing CSP

https://gerrit.wikimedia.org/r/1249348

Hey all -

  • live.staticflickr.com

should now be allowed within Wikimedia project CSP.

A_smart_kitten assigned this task to sbassett.

Based on some brief testing I've just done, and based on this comment onwiki that someone else can also now see files' thumbnails displaying after giving UploadWizard the link to a Flickr album, I'm going to boldly call this resolved. Please feel free to reopen and/or leave a comment if it turns out that there's something else that is also not working in UploadWizard as a result of the recent changes :)

I can see thumbnails in the selection UI, but the following screen in which I would add titles and descriptions doesn't display.

Hmmm... Thanks for the comment, @TimSC. A similar experience was also reported onwiki. Testing just now, I can sometimes reproduce it and sometimes can't.

From what I can see it seems like the https://api.flickr.com/services/rest/ endpoint might sometimes not be returning a access-control-allow-origin: * header. This in turn then appears to trigger a browser error similar to:

Access to XMLHttpRequest at 'https://api.flickr.com/services/rest/?<params>' from origin 'https://commons.wikimedia.org' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.

I can't guarantee that I'm not missing anything; but - from testing locally - it seems like the access-control-allow-origin header might be missing when the response from the Flickr API is:

{"code":201,"stat":"fail","message":"Sorry, the Flickr API service is not currently available."}

So maybe this is a degradation-of-service of the Flickr API that just happens to slightly coincide with problems caused by Wikimedia's new Content-Security-Policy (and maybe the CORS failure itself is a slight red-herring for the Flickr API just having degraded service)?


If this continues, it strikes me that it might need a member of WMF staff to reach out to Flickr about it (I would assume, as it's Wikimedia's API key that's being used here). (But if I'm correct that this might be a separate issue to the CSP problems, maybe that might be better for a new task.)