bzimport set Reference to bz20512.
Nemo_bis created this task.Via LegacySep 5 2009, 7:41 AM
bzimport added a comment.Via ConduitNov 3 2009, 11:28 PM

mdale wrote:

Would like to bump this up now that its better supported and running successfully running on testing.wikimedia.org.

Nemo_bis added a comment.Via ConduitDec 17 2009, 7:38 AM

November 5, «15:13 logmsgbot: brion synchronized php-1.5/wmf-config/InitialiseSettings.php 'enabling upload by URL for sysops on all wikis'».
December 1, «15:20 apergos: temporarily turned off upload-by-url on test.wikipedia, it interferes with testing the external page retrieval extension for fundraising».
We still have upload_by_url right on all wikis, but actual upload by URL is not enabled.

bzimport added a comment.Via ConduitDec 21 2009, 7:38 AM

mdale wrote:

Any update on turning this back on?

Catrope added a comment.Via ConduitJan 4 2010, 8:53 PM

Enabled on test and commons; tested it and it works on both.

tstarling added a comment.Via ConduitJan 5 2010, 2:21 AM

Disabled again, breaks Lucene search.

bzimport added a comment.Via ConduitJan 7 2010, 10:16 PM

mdale wrote:

Fix committed in r60811 ... this re-merges new HttpFunctions.php from js2 back into trunk. Its much closer to what is currently in the deployment branch, and is needed to support async copy-by-url uploads.

Tested locally with copy-by-url uploads. Hard to test the Lucene setup outside of the wikimedia.

Catrope added a comment.Via ConduitJan 7 2010, 10:30 PM

Reopening, the request itself (enable upload by URL) hasn't been fulfilled.

bzimport added a comment.Via ConduitMar 31 2010, 6:45 PM

mdale wrote:

Could we at least enable $wgAllowCopyUploads without background uploading?

bzimport added a comment.Via ConduitOct 27 2010, 7:48 PM

mdale wrote:

Bryon has been working on a jobQueue implementation.

Bryon do you have any updates on how well its supported in trunk presently and if there are any outstanding issues of this feature that your aware of?

bzimport added a comment.Via ConduitOct 27 2010, 7:53 PM

Bryan.TongMinh wrote:

As far as I'm concerned all outstanding issues have been fixes.

Also, there is both a Brion and a Bryan. There is no such thing as a Bryon though.

bzimport added a comment.Via ConduitOct 27 2010, 9:58 PM

mdale wrote:

Sorry "Bryan". Great to hear the outstanding issues have been fixed :) Can we add this to the list of feature to deploy soon after we review and deploy trunk? Is there a bug or road map page tracking that somewhere Roan, Rob ?

bzimport added a comment.Via ConduitMar 10 2011, 7:31 PM

mdale wrote:

Flagging this bug for additional comment, now that the dust has settled from 1_17 it would be great if we could review setting $wgAllowCopyUploads to true for commons.

bzimport added a comment.Via ConduitApr 12 2011, 6:53 PM

mdale wrote:

Any update on enabling this?

bzimport added a comment.Via ConduitMay 14 2011, 1:40 PM

mdale wrote:

Any update on enabling this?

Reedy added a comment.Via ConduitAug 11 2011, 2:04 PM

(In reply to comment #14)

Any update on enabling this?

Seems we've got some infrastructure issues as the apaches don't have direct external internet access...

Need to discuss with Tim

TTO added a comment.Via ConduitOct 30 2011, 10:02 AM

Please see bug 14919 comment 5 - I didn't notice this followup bug.

tstarling added a comment.Via ConduitDec 20 2011, 5:25 AM

Needs review.

Peachey88 added a comment.Via ConduitJan 16 2012, 1:27 PM
  • Bug 14919 has been marked as a duplicate of this bug. ***
Reedy added a comment.Via ConduitFeb 9 2012, 10:05 PM
  • Bug 1552 has been marked as a duplicate of this bug. ***
Reedy added a comment.Via ConduitFeb 9 2012, 10:21 PM
  • Bug 5283 has been marked as a duplicate of this bug. ***
Reedy added a comment.Via ConduitFeb 9 2012, 10:55 PM
  • Bug 1552 has been marked as a duplicate of this bug. ***
Reedy added a comment.Via ConduitMar 16 2012, 4:12 PM
  • Bug 35263 has been marked as a duplicate of this bug. ***
bzimport added a comment.Via ConduitMar 16 2012, 8:18 PM

M8R-udfkkf wrote:

Any progress/updates?

Reedy added a comment.Via ConduitJun 15 2012, 11:37 PM
  • Bug 37636 has been marked as a duplicate of this bug. ***
Yann added a comment.Via ConduitAug 7 2012, 7:24 AM

Any progress/updates?

kaldari added a comment.Via ConduitAug 30 2012, 11:32 PM

We now have a custom Flickr uploading interface that was built as a GSoC project: https://gerrit.wikimedia.org/r/#/c/12269/

But unfortunately it depends on $wgAllowCopyUploads. It would be really nice to have this available during Wiki Loves Monuments.

Any update on getting internet access to the apaches?

kaldari added a comment.Via ConduitSep 3 2012, 7:39 AM

According to Tim, there is already a proxy set up called url-downloader.wikimedia.org which should allow the apaches to access the outside world. He also said that UploadFromUrl.php could use a good review, especially for error reporting. Other than that, I don't think there are any blockers for turning this back on.

bzimport added a comment.Via ConduitSep 20 2012, 10:04 PM

sumanah wrote:

Ryan, maybe you could file an RT ticket to get this higher on ops's priority list? Thanks.

Eloquence added a comment.Via ConduitSep 21 2012, 6:23 PM

I don't think there's necessarily any ops work involved. We need to review the upload from URL code and verify whether there's still any Lucene breakage in production; the URL downloader itself is operational according to Mark.

kaldari added a comment.Via ConduitSep 25 2012, 12:28 AM

Unfortunately I don't know enough about Lucene to tell what would cause problems. The impression I got from Tim is that the search issue was no longer a problem, but it would be best to get confirmation from him directly or someone familiar with Lucene.

Nemo_bis added a comment.Via ConduitSep 25 2012, 5:08 AM

Oren?

tstarling added a comment.Via ConduitSep 26 2012, 4:20 AM

(In reply to comment #30)

Unfortunately I don't know enough about Lucene to tell what would cause
problems. The impression I got from Tim is that the search issue was no longer
a problem, but it would be best to get confirmation from him directly or
someone familiar with Lucene.

The Lucene issue was fixed with a hack in the MWSearch extension, but setting $wgHTTPProxy would still break any other Http::get() caller that is attempting to fetch things from the local intranet. I think the configuration should be split: UploadFromUrl should pass a "proxy" option to MWHttpRequest::factory() with a value derived from a global variable specific to this feature.

Yann added a comment.Via ConduitSep 26 2012, 6:15 AM

Could we bump up the priority of this? I am surprised it is still not working on Wikimedia Commons.

Reedy added a comment.Via ConduitSep 28 2012, 12:34 AM

(In reply to comment #32)

(In reply to comment #30)
> Unfortunately I don't know enough about Lucene to tell what would cause
> problems. The impression I got from Tim is that the search issue was no longer
> a problem, but it would be best to get confirmation from him directly or
> someone familiar with Lucene.

The Lucene issue was fixed with a hack in the MWSearch extension, but setting
$wgHTTPProxy would still break any other Http::get() caller that is attempting
to fetch things from the local intranet. I think the configuration should be
split: UploadFromUrl should pass a "proxy" option to MWHttpRequest::factory()
with a value derived from a global variable specific to this feature.

https://gerrit.wikimedia.org/r/#/c/25605/

Reedy added a comment.Via ConduitSep 28 2012, 4:23 PM

Enabled on testwiki

Uploads from HTTPS sites fail with an error of:
Error fetching URL: Received HTTP code 403 from proxy after CONNECT

TTO added a comment.Via ConduitSep 29 2012, 1:15 AM

I tried it using a script (via API), but it returned HTTP 500 when trying to print the error message :(

see http://pastebin.com/raw.php?i=NXLPRteT

kaldari added a comment.Via ConduitOct 18 2012, 12:03 AM

Looks like uploads from regular HTTP sites also return 403 Forbidden.

kaldari added a comment.Via ConduitOct 18 2012, 12:11 AM

Well, actually I can upload from http://farm1.staticflickr.com, but not http://upload.wikimedia.org

Reedy added a comment.Via ConduitOct 18 2012, 12:44 AM

Http worked before, as I did the responsible thing and imported copyrighted Google logos. Was a couple of weeks ago now.

Might need an exception in the squid config to do local requests..

Yann added a comment.Via ConduitOct 18 2012, 8:31 AM

We *need* this for this migration of Wiki Voyage/Travel to Wikimedia. There are 34,124 files to upload.

Reedy added a comment.Via ConduitOct 18 2012, 11:30 AM

No we don't. We can bulk upload them using a maintenance script

kaldari added a comment.Via ConduitOct 18 2012, 9:40 PM

We did need it for WLM, but that was 2 weeks ago :(

Since Flickr uploading seems to work, would anyone object to me moving forward with Flickr-only uploading for admins? This would involve turning on $wgAllowCopyUploads, turning off $wgCopyUploadsFromSpecialUpload, setting $wgCopyUploadsDomains to array( '*.flickr.com' ), finishing up https://gerrit.wikimedia.org/r/#/c/12269/ for UploadWizard, and setting $wgGroupPermissions['sysop']['upload_by_url'] to true.

Reedy added a comment.Via ConduitOct 18 2012, 10:31 PM

(In reply to comment #42)

We did need it for WLM, but that was 2 weeks ago :(

Since Flickr uploading seems to work, would anyone object to me moving forward
with Flickr-only uploading for admins? This would involve turning on
$wgAllowCopyUploads, turning off $wgCopyUploadsFromSpecialUpload, setting
$wgCopyUploadsDomains to array( '*.flickr.com' ), finishing up
https://gerrit.wikimedia.org/r/#/c/12269/ for UploadWizard, and setting
$wgGroupPermissions['sysop']['upload_by_url'] to true.

Seems sane to me, proxy should be alright for now...

faidon added a comment.Via ConduitOct 27 2012, 10:43 AM

Reading through the comments, I don't see any pending ops work for this, so I'm removing the ops tag. If I'm mistaken, please re-add and clarify what needs to be done from our side.

bzimport added a comment.Via ConduitNov 13 2012, 10:26 AM

rd232 wrote:

Ping! Commons would love to have this, as discussed again at http://commons.wikimedia.org/wiki/Commons:Village_pump/Proposals#Upload_by_URL

kaldari added a comment.Via ConduitNov 13 2012, 7:03 PM

Stay tuned...

Krenair added a comment.Via ConduitNov 20 2012, 8:10 PM

Looks like Gerrit change 34357 enables this with flickr domains for commons admins.

bzimport added a comment.Via ConduitNov 27 2012, 2:43 AM

sumanah wrote:

Gerrit change #34357 is now merged and https://gerrit.wikimedia.org/r/#/c/25605/ is merged; ok to close this ticket?

TTO added a comment.Via ConduitNov 27 2012, 9:51 AM

This isn't working at the moment, because it is apparently "disabled until author bug is fixed". Is this "author bug" tracked?

Nemo_bis added a comment.Via ConduitNov 27 2012, 10:25 AM

(In reply to comment #49)

This isn't working at the moment, because it is apparently "disabled until
author bug is fixed".

aka Ie59c3eb7

Is this "author bug" tracked?

Yes, bug 42312, and it's fixed. It's useful if you test it on [[testwiki:]] and mark it VERIFIED if ok.

kaldari added a comment.Via ConduitNov 27 2012, 7:23 PM

As soon as https://gerrit.wikimedia.org/r/#/c/34856/ is merged and deployed, I'll turn $wgAllowCopyUploads on and we can close this bug. We still need to file a separate bug for HTTPS support for the proxy service, but that isn't a blocker for enabling $wgAllowCopyUploads.

Also note that when $wgAllowCopyUploads is turned on, it's only going to be available in a very limited sense: for admins on Commons, via UploadWizard, from Flickr. People may want to file new bugs to change any of these individual limitations.

kaldari added a comment.Via ConduitDec 7 2012, 1:15 AM

$wgAllowCopyUploads turned on for Commons in https://gerrit.wikimedia.org/r/#/c/37353/

This feature is currently limited to administrators, via UploadWizard, from Flickr.com.

Feel free to file new bugs if you'd like any of these restrictions changed.

Yann added a comment.Via ConduitDec 26 2012, 8:41 AM

Upload from Flickr works for Firefox on Windows XP, but not with Chrome Version 23.0.1271.10. Should I open a new bug?

kaldari added a comment.Via ConduitDec 26 2012, 10:34 AM

Yes, please file a bug for that. BTW, some people have had trouble using Flickr uploading if they have the HTTPS Everywhere extension turned on, so you may want to check that.

Add Comment

Column Prototype
This is a very early prototype of a persistent column. It is not expected to work yet, and leaving it open will activate other new features which will break things. Press "\" (backslash) on your keyboard to close it now.