Page MenuHomePhabricator

Access request for Daniel Worley to analytics / hadoop
Closed, ResolvedPublic

Description

Daniel is a contractor working with the search team for the next 3 months. He will need access to the analytics cluster + hadoop to train and evaluate machine learning models used for search ranking.

This does not require the privatedata role, as all necessary private input data is in already pre-processed into hdfs at /wmf/data/discovery/query_clicks

Event Timeline

Restricted Application added a project: Operations. · View Herald TranscriptJun 20 2017, 5:42 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript

@EBjune This will require your approval

Dzahn added a subscriber: Dzahn.Jun 20 2017, 11:17 PM

Hi, here are the existing access groups and their description:

https://wikitech.wikimedia.org/wiki/Analytics/Data_access#Access_Groups

Does one of them seem to fit your requirements?

I don't think any of them give access to "the analytics cluster" as in machines like analytics1001 and so on but you probably just meant stat* machines, right?

@Dzahn I think he just needs access to the analytics-users group, which I approve.

Dzahn added a subscriber: RobH.Jun 21 2017, 3:18 AM
RobH assigned this task to EBernhardson.EditedJun 21 2017, 5:57 PM
RobH added a subscriber: RStallman-legalteam.

This request will need a few things for this access to be granted. All of these requirements are documented on https://wikitech.wikimedia.org/wiki/Requesting_shell_access

  • User needs to list their preferred login name, as well as their Wikitech Username.
  • User needs to provide (ideally on this task) their public ssh key. This key needs to be different than other keys, and dedicated ONLY to WMF production use.
  • User must read and sign the L3 document on phabricator (also we need to know what their phab username is, ideally by their commenting on this task with the required info above.)

An NDA must be on file with WMF legal. If they are a contractor, I think it is included in their contractor paperwork, but we'll get confirmation from @RStallman-legalteam about that!

RobH reassigned this task from EBernhardson to Dworley.Jun 21 2017, 5:58 PM
EBernhardson added a comment.EditedJun 21 2017, 6:05 PM

For training daniel will need access only to stat1004, to execute the training against the hadoop cluster. I often connect a socks proxy to analytics1001 to access the logs at runtime, but it looks like can use the proxy against stat1004, wikitech just happens to suggest analytics1001.

analytics_users grants access to stat1004 and the hadoop cluster (via analytics1001), so that is what is needed here.

I also regularly connect a socks proxy to analytics1001, because when checking for runtime logs

You don't need to do this to analytics1001. It should work just as well via a client box, like stat1004, and I'd prefer slightly if you used that instead. If I could technically restrict folks from logging into analytics1001 while still granting them accounts there for HDFS purposes, I would :)

EBernhardson added a comment.EditedJun 21 2017, 6:38 PM

analytics_users grants access to stat1004 and the hadoop cluster (via analytics1001), so that is what is needed here.

I also regularly connect a socks proxy to analytics1001, because when checking for runtime logs

You don't need to do this to analytics1001. It should work just as well via a client box, like stat1004, and I'd prefer slightly if you used that instead. If I could technically restrict folks from logging into analytics1001 while still granting them accounts there for HDFS purposes, I would :)

Noted! I've updated wikitech to reflect this:https://wikitech.wikimedia.org/w/index.php?title=Analytics/Systems/Cluster/Access&diff=1762653&oldid=1755287

@RobH: Yes, contractors sign an NDA through their paperwork with HR or there would be a clause in the contract created for them by legal if they are operating under a business entity. I think this one was likely done through HR as I don't have access to it unless it's under something other than Daniel Worley.

@RStallman-legalteam this would possibly be under 'Open Source Connections', the business entity.

Yes, we have a current contract with Open Source Connections through 8/31/2017. Thanks!

RobH added a comment.Jun 21 2017, 9:21 PM

Excellent, seems the NDA aspect is covered, and we can enter this shell request with an expiry date of 2017-08-31.

We'll still need the following from @Dworley:

  • wikitech user name
  • preferred shell login name
  • public ssh key (this should be a dedicated key for mwf production shell access, and not the same key used for labs or elsewhere.)
  • read and sign the L3 document

@RobH: I've signed the L3 document. Where should I send the other requested details?

RobH added a comment.Jun 22 2017, 4:14 PM

Just here on the task is fine (all of it becomes public when its input into our user files)

Wikitech User: dworley
Preferred Shell User: dworley

Public Key: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlyZ7OS3200wsY9HIv2i0FlN3tbUBvzXL/AqvGXADDZscRK3DOUTAAyv/oYDoiYGe8V/5GIujrN2EQ1BPo4FFqmQfHIjNv5yJ5xZJmPeFUtBlOLLMa1sknfAcYiOpvhn0T7gLpipf0JUq+0J5RmbILD0xq5GHZAu3rYAJUtz4ameoPkDy1mGJk66ajZw+ufnv+g4zTXG7RFAT4r1Tui9wHfNmQdxuLGhUsdjOWxUwJXIqSzhdXfhQGx0JkZLPPwLuKPmNo58GQESe3kZM6a7GqSBhvU8sDdLP21LbOKKyeISpn+V2WAa+XPFF6FI0cbhIHF7RGMvV1CK2g86TAPngl dworley@slippy

RobH claimed this task.Jun 22 2017, 10:03 PM
RobH moved this task from user confirm to 3 Business Day Wait on the SRE-Access-Requests board.

@Dworley: Sorry, I forgot to ask for your email address, but I'm assuming the one you used for wikitech is acceptable?

Change 360988 had a related patch set uploaded (by RobH; owner: RobH):
[operations/puppet@production] Daniel Worley shell access request

https://gerrit.wikimedia.org/r/360988

RobH added a comment.EditedJun 22 2017, 10:20 PM

This is now in the 3 day waiting period for objection. If no objections are noted, I'll merge this at the end of the waiting period.

Yes, my wikitech email is fine. Thanks!

Change 360988 merged by RobH:
[operations/puppet@production] Daniel Worley shell access request

https://gerrit.wikimedia.org/r/360988

RobH closed this task as Resolved.Jun 23 2017, 4:22 PM
RobH removed RobH as the assignee of this task.

No objections were noted & all other steps accomplished, so I've merged this access live. It'll take about 30 minutes for all affected hosts to call in and apply updates.

If this fails to work after 16:45 GMT, feel free to reopen with what isn't working and/or questions.