Page MenuHomePhabricator

Flickr2Commons tool: Throttle needed
Closed, ResolvedPublic

Description

Flickr2Commons is open to anyone to upload 10,000+ files from Flickr at a fast rate. A system for throttling would be beneficial. At the moment with parallel threads being used, it is too easy for less experienced users to create a very large batch upload without any plan or project for how they would be categorized or renamed.

A current contentious example of 15,000+ images uploaded by a user new to the tool is under discussion at https://commons.wikimedia.org/wiki/Commons:Administrators%27_noticeboard/User_problems/Archive_52#User:The_Photographer. The results have created a burden on the community and is discouraging for the uploader who acted in good faith.

Event Timeline

Fae assigned this task to Magnus.
Fae raised the priority of this task from to Medium.
Fae updated the task description. (Show Details)
Fae added a project: Commons.
Fae added subscribers: Fae, Steinsplitter.

@Fae: Is this a request about implementing such a throttle generically somewhere in the Commons uploading infrastructure, or about implementing this in the flickr2commons tool specifically?

@Fae: Is this a request about implementing such a throttle generically somewhere in the Commons uploading infrastructure, or about implementing this in the flickr2commons tool specifically?

The latter. I was not sure where to add it, thanks for shuffling it about. :-)

I am unsure how tricky this is, but it may be a good idea for user upload service type tools with the capability for running multiple threads to be limited to one-thread-per-user-account unless it is an approved bot or the user is white-listed (by some community agreed process).

@Magnus: Hi! This task has been assigned to you a while ago. Could you maybe share an update? Is there more work needed? Do you still plan to work on this task?
If this task has been resolved in the meantime: Please update the task status (via Add Action...Change Status in the dropdown menu).
If this task is not resolved and only if you do not plan to work on this task anymore: Please consider removing yourself as assignee (via Add Action...Assign / Claim in the dropdown menu): That would allow others to work on this (in theory), as others won't think that someone is already working on this. Thanks!

Aklapper renamed this task from Flickr2Commons throttle needed to Flickr2Commons tool: Throttle needed.Jun 19 2020, 6:52 AM

As I wrote five years ago, I added a limit to upload only two files concurrently.
No further comment has been made since, so I consider this issue to be solved.