Page MenuHomePhabricator

Allow GWT uploads from ethz.ch
Closed, ResolvedPublic

Tokens
"Like" token, awarded by MartinPoulter."Like" token, awarded by BeatEstermann."Like" token, awarded by Micha."Like" token, awarded by Sadads."Like" token, awarded by Fae."Like" token, awarded by Kelson.
Assigned To
Authored By
Kelson, Jan 8 2016

Description

Wikimedia CH has a GLAM partnership with the ETH Library
https://www.ethz.ch/en/the-eth-zurich/organisation/departments/eth-library.html

They plan to upload ~300.000 high-resolution pictures during the year using GWT. That's why we need to add the following domain to the wgCopyUploadsDomains whitelist (also on beta) of Commons:
www.e-pics.ethz.ch

Or if possible something more robus like:
*.ethz.ch

Event Timeline

Kelson created this task.Jan 8 2016, 10:58 AM
Kelson raised the priority of this task from to Needs Triage.
Kelson updated the task description. (Show Details)
Kelson added a subscriber: Kelson.
Restricted Application added subscribers: JEumerus, StudiesWorld, Steinsplitter and 2 others. · View Herald TranscriptJan 8 2016, 10:58 AM

Change 262893 had a related patch set uploaded (by Mdann52):
Add 2 sites to $wgCopyUploadsDomains

https://gerrit.wikimedia.org/r/262893

Mdann52 claimed this task.Jan 11 2016, 1:09 PM

Do we have a timeline for the rollout of this?

jenkins merge failed so it probably needs a rebase.

then it should be added to a SWAT window at wikitech:Deployments. (is that right @greg?)

Do we have a timeline for the rollout of this?

There is never a timeline for shell requests. Since Reedy left, your only chance if you're in a hurry is to do the SWAT dance.

Fae awarded a token.Jan 27 2016, 5:29 PM
Fae added a subscriber: Fae.
Micha awarded a token.Jan 27 2016, 5:58 PM

jenkins merge failed so it probably needs a rebase.

{{done}}

Couldn't whitelisting domains be made a regular routine for someone to do every 24 or 48 hours?
Waiting times of up to 3 weeks for just copy-pasting a domain into a list are hilarious and difficult to communicate to GLAM institutions...
Giving write access to more people to the whitelist might be a solution... ?!

Couldn't whitelisting domains be made a regular routine for someone to do every 24 or 48 hours?

We need to get a proper set of guidelines put together to determine which of these requests are okay and which are not before I am happy with dealing with them routinely.

Giving write access to more people to the whitelist might be a solution... ?!

Deployment access is not going to be given out just so that people can deal with CopyUploadsDomains requests.

We need to get a proper set of guidelines put together to determine which of these requests are okay and which are not before I am happy with dealing with them routinely.

Who is "We"? - Could the GLAM-Wiki people help with setting up guidelines or doing the screening?

Deployment access is not going to be given out just so that people can deal with CopyUploadsDomains requests.

Wouldn't you just need a whitelist somewhere in a place where you trust the security settings and the access management and give access to it to a bunch of people whom you trust to curate the whitelist adequately? - You could then automatically (or manually) write the new entries into your very secure place where you don't want to give access to other people, couldn't you? (I'm just asking naively, I have no idea of the actual security setup, user rights, potential risks and implications, etc.)

We need to get a proper set of guidelines put together to determine which of these requests are okay and which are not before I am happy with dealing with them routinely.

Who is "We"? - Could the GLAM-Wiki people help with setting up guidelines or doing the screening?

The deployers and other people helping process Wikimedia-Site-requests. The guidelines should probably be set by Security.

Deployment access is not going to be given out just so that people can deal with CopyUploadsDomains requests.

Wouldn't you just need a whitelist somewhere in a place where you trust the security settings and the access management and give access to it to a bunch of people whom you trust to curate the whitelist adequately? - You could then automatically (or manually) write the new entries into your very secure place where you don't want to give access to other people, couldn't you? (I'm just asking naively, I have no idea of the actual security setup, user rights, potential risks and implications, etc.)

The issue isn't really about getting more people able to press the button, I think it's uncertainty about what sites being whitelisted implies in terms of trust, security, etc.

jeremyb-phone set Security to None.Jan 29 2016, 4:02 AM
jeremyb-phone edited subscribers, added: jeremyb; removed: jeremyb-phone.

{{done}}

ok, you still need to do the other part:

then it should be added to a SWAT window at [[

https://wikitech.wikimedia.org/wiki/Deployments | wikitech:Deployments ]].

@jeremyb
I have added the deployment request in https://wikitech.wikimedia.org/wiki/Deployments#Week_of_February_1st. Hope this is OK like this.

@Krenair

Thank you for your explanations but what is the concrete path to have things working correctly next time? This is not a new situation, I have done many times this kind of request in the past but this is the first time it goes so long/wrong. From my POV, something has been broken. Now as a volunteer, I fell put in a position where I have to do many additional things unrelated to my GLAM projects (launch the creation of GWT white-listing request validation process, scheduling things on SWAT, stoking people to get thumb up, etc.). Wouldn't that be a better approach that I focus on the work with GLAMs I'm working with and the WMF employees focus to get Wikimedia-Site-Requests handled correctly?

Fae added a comment.EditedJan 30 2016, 2:11 PM

I agree this is a broken process.

A simple solution would be to have a live white-list on the GWT pages on Commons. New suggestions for the white-list can then follow a standard process of linking to beta-testing, a GLAM organizations website where there is some explanation of the upload project, or a project page on Commons with similar explanations about the scope of the project and who is going to remain responsible for all uploads (and problems created). If necessary, then when agreed on the Commons white-list, we can raise a "rubberstamp" task request here on Phabricator.

GWT users are active on Commons, not on Phabricator, and having this complex additional hurdle just feels meaninglessly bureaucratic.

I can raise this on the GWT email list, and as a process it could be added to the GWT user guide if acceptable to the right dev people (I no longer know exactly who that is).

Wouldn't that be a better approach that I focus on the work with GLAMs I'm working with and the WMF employees focus to get Wikimedia-Site-Requests handled correctly?

Which WMF employees do you have in mind?

Change 262893 merged by jenkins-bot:
Add 2 sites to $wgCopyUploadsDomains

https://gerrit.wikimedia.org/r/262893

Kelson closed this task as Resolved.Feb 3 2016, 6:59 PM
Kelson reopened this task as Open.Feb 28 2016, 5:57 PM
Kelson added a subscriber: Mdann52.

I have unfortunately to re-open that ticket because "*.ethz.ch" does not match "www.e-pics.ethz.ch". We always get "Copy uploads are not available from this domain.". After investigating a little bit with Mediawiki source code it seems that "*" does only match characters between ".".

Kelson triaged this task as High priority.Feb 28 2016, 6:00 PM
Restricted Application added a subscriber: Dereckson. · View Herald TranscriptFeb 28 2016, 6:00 PM

As a consequence it looks like we should put "www.e-pics.ethz.ch" instead of "*.ethz.ch". Can someone change the configuration that way please?

Luke081515 added a subscriber: Luke081515.
Restricted Application added a project: User-Luke081515. · View Herald TranscriptFeb 28 2016, 6:26 PM

Change 273776 had a related patch set uploaded (by Luke081515):
Correct one Domain at $wgCopyUploadsDomains

https://gerrit.wikimedia.org/r/273776

I replaced *.ethz.ch now by *.e-pics.ethz.ch

Luke081515 added a comment.EditedMar 1 2016, 4:33 PM

I already put this to SWAT today ;)

I removed your entry

Change 273776 merged by jenkins-bot:
Correct one Domain at $wgCopyUploadsDomains

https://gerrit.wikimedia.org/r/273776

Luke081515 closed this task as Resolved.Mar 1 2016, 4:40 PM

SWATed, should be fixed now, please try.

Kelson reopened this task as Open.Mar 1 2016, 6:12 PM

@Luke081515
Thx, this looks good on commons.wikimedia.org, but beta (http://commons.wikimedia.beta.wmflabs.org/wiki/Special:GWToolset) seems to not have been updated, so impossible to test.

I will contact someone for it.

Luke081515 closed this task as Resolved.Mar 1 2016, 6:40 PM

Solved now.