Page MenuHomePhabricator

JupyterHub access for meps not working (was: Requesting access to analytics servers for mepps)
Closed, ResolvedPublic

Description

Username: mepps
Full name: Margaret Epps

I'm on fr-tech and need access to the analytics-privatedata-users group. Let me know what additional information you need!

Event Timeline

mepps created this task.Apr 18 2018, 5:17 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 18 2018, 5:17 PM
Restricted Application added a project: Operations. · View Herald TranscriptApr 18 2018, 5:17 PM

You'll want approval from @K4-713 :)

mepps added a comment.Apr 18 2018, 8:09 PM

@Reedy, @K4-713 is on sabbatical until early June. I'm currently serving as her delegate.

Reedy added a comment.Apr 18 2018, 8:09 PM

Victoria then? Unless there's someone else in between. Needs to go up the chain for approval

Hi Margaret, could you explain a little bit why you need that specific group and what you are planning to do. cc: @Nuria

Dzahn triaged this task as Medium priority.Apr 18 2018, 8:37 PM
mepps added subscribers: AndyRussG, Ejegg.EditedApr 19 2018, 3:43 PM

@Dzahn I'm lookign for access to Pivot (especially https://pivot.wikimedia.org/#banner_activity_minutely and https://pivot.wikimedia.org/#pageviews-hourly), SWAP (python notebooks for more complex queries accessing NDA-protected data stores, see https://wikitech.wikimedia.org/wiki/SWAP) and Hive (see https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Hive) because otherwise I can't support @AndyRussG and @Ejegg in debugging many banner display issues and monitoring impression rates . We want to spread the work on these tools among more than one team member and have the ability to do this in case of emergencies and review each other's code. Thanks!

mepps added a comment.Apr 19 2018, 3:45 PM

We want access for all of fr-tech actually for these purposes: https://phabricator.wikimedia.org/T181629

Nuria added a comment.Apr 19 2018, 6:11 PM

@meeps: please note that the ticket you are linking to is also a request for access where I noted that these tools require different access levels, ALL of them give you access to nda protected data in different ways.

Pivot requires an ldap user alone, no ssh-keys. it is a visual tool that requires no technical expertise.
SWAT notebooks require ssh keys and shell access, it is a more technical tool that requires familiarity with python, if you are only going to look at data pivot might be enough. Can you be more specific?

review each other's code

There is no fr code that runs in any of these instances, fr-tech memeers might have notebooks but that code is not CR-ed.

@meeps: please note that the ticket you are linking to is also a request for access where I noted that these tools require different access levels, ALL of them give you access to nda protected data in different ways.

Pivot requires an ldap user alone, no ssh-keys. it is a visual tool that requires no technical expertise.
SWAT notebooks require ssh keys and shell access, it is a more technical tool that requires familiarity with python, if you are only going to look at data pivot might be enough. Can you be more specific?

@Nuria, @mepps is an engineer. She needs SWAP and Hive access, and access to query Druid directly, too, in fact.

The explanation was specific.

review each other's code

There is no fr code that runs in any of these instances, fr-tech memeers might have notebooks but that code is not CR-ed.

We need to review each other's code even when it isn't in Gerrit. Hive queries and other code that runs in Python notebooks frequently needs several pairs of eyes.

In addition to the above, we've been working on a Python library to facilitate querying impression rates. Please see T178930. It's not yet in Gerrit, but it will be. It currently runs in a SWAP notebook, and hopefully we'll be able to use it elsewhere to generate alerts, too.

Finally, though this is not essential, it's a good idea, I think, for FR-Tech engineers to be able to contribute, or at least collaborate in some capacity, in developing and maintaining code that runs on the Analytics cluster and that FR depends on. Though there are silos, let's perhaps keep the silo walls at least somewhat porous, eh?

Please don't hesitate to reach out if there are any misunderstandings we might work out... Thanks so much!!!! :)

Nuria added a comment.Apr 19 2018, 7:53 PM

Access approved on my end.

mepps added a comment.Apr 19 2018, 8:13 PM

Thanks @Nuria and @AndyRussG! Is there a next step @Dzahn @Reedy?

Dzahn added a comment.Apr 19 2018, 9:16 PM

@mepps Yea, the next step would be that we need a SSH key from you. Could you create one (https://wikitech.wikimedia.org/wiki/Production_shell_access#SSH_Key_Requirements) and paste it here please?

Also, do you have a user on https://wikitech.wikimedia.org yet? Please tell us your user name there.

mepps added a comment.Apr 20 2018, 5:17 PM

@Dzahn my wikitech username is MEpps and here's the public key:

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDMquf0ywthSAksqXIMATkeQt8ui6B2JxWES4zEMQVYtPlVUNnFGQyAbYN/Fe5kn1pEERkmMFJb+hin0d4V09r6al1LtJmFzyDBw9dfsmbPjUDwY+1T8ADNzBXgtySQ3W5U1KLd2YeQRXmirD26/uhaGwXEcq8GLN/BlFdqvpjeM5/9ESGvE0gP1gfmM+uxb4PyYMMAUpK8SxAGN5cBbSO45kdPOSKt1jd6kF2ZjLGCATzwC0a1USAz5ivVyaPnzVzFY3jjLaACRitwWWEycNdqXha5Vk7Y7xyOr64jaPQSzZq5XWWxrTWsNC7Qzy398a5cFbXL6k7/CgpDGYRbdB3M+ibGJllZcDznzNM6u4XGewHwjzIwlcxFHwBX6pG1n3GXOayGlPSj0VGelw363klpi7FKxw2g4Q2cp3SvYuEet+psU5qI8MexOQjoH7uK0M5axiTVN2muSTuulpUwcyxzunLMxr5PkoHmHVz7Ap+4L4PNencMiZ77PHm7gz3dBVrB70UTnvSxaYDwPZvj/OcPhvlMPLe70KZs5OAyjqCADNNF8m11mG7a4MJUb8a0uT3jamX7zAwgj/kVibGN0tBrZID+scpuNXRN6gfbOactUJ9glXvxKFtFerXVTDQXqpZxSUbp45+sNvHvQwxdU/PFzFzR46+bOXlIz1HZanX7Ow== mepps@saturn

Thanks!

Change 427944 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] admins: create shell account for mepps, add to analytics-privatedata

https://gerrit.wikimedia.org/r/427944

Dzahn added a comment.Apr 20 2018, 6:16 PM

@mepps Thanks! looks good. I made the needed puppet code change and uploaded to Gerrit. The next step will be getting this reviewed/merged (i don't foresee any issues but there is a 3 business day waiting period for policy reasons). Will let you know once it's done early next week, ok?

Dzahn claimed this task.Apr 20 2018, 6:50 PM
herron added a subscriber: herron.

Change 427944 merged by Dzahn:
[operations/puppet@production] admins: create shell account for mepps, add to analytics-privatedata

https://gerrit.wikimedia.org/r/427944

Dzahn added a comment.Apr 25 2018, 4:45 PM

Hi @mepps Your user has been created now and you are in the requested group.

On one of the bastion hosts:

[bast1002:~] $ id mepps
uid=16947(mepps) gid=500(wikidev) groups=500(wikidev),600(all-users)

On stat1005:
[stat1005:~] $ id mepps
uid=16947(mepps) gid=500(wikidev) groups=500(wikidev),731(analytics-privatedata-users)

After puppet runs (max 30 minutes) it will be created on all bastion hosts and other hosts having this admin group.

You can use any of the bastion servers. I would recommend the one closest to you. See the map and host names here:

https://wikitech.wikimedia.org/wiki/Bastion

On https://wikitech.wikimedia.org/wiki/Production_shell_access#Standard_config you can find an example SSH config how to jump via one of the bastion hosts to the host you need.

Dzahn closed this task as Resolved.Apr 25 2018, 4:45 PM
mepps added a comment.Jun 12 2018, 7:57 PM

Thank you @Dzahn! I'm currently trying to log into JupyterHub and my wikitech credentials aren't working. I just wanted to make sure I was added as a user there?

Dzahn added a comment.Jun 13 2018, 7:01 AM

@mepps Sorry, i don't know very much about JupyterHub. What I do know though is that the docs say "You will need production access (ask for the "researchers"/"analytics-privatedata-users"/"statistics-privatedata-users" groups, SWAP piggy backs on data access rules for the Analytics cluster, and any of these 3 groups should work)" (https://wikitech.wikimedia.org/wiki/SWAP#Access) and you have analytics-privatedata-users, so it should work.

I will reopen the ticket, add some more tags/people to the ticket and let the "on duty" person handle it. We are handling access requests on a rotating basis each week.

Dzahn renamed this task from Requesting access to analytics servers for mepps to JupyterHub access for meps not working (was: Requesting access to analytics servers for mepps).Jun 13 2018, 7:01 AM
Dzahn reopened this task as Open.
Dzahn removed Dzahn as the assignee of this task.
Dzahn edited projects, added Analytics, Jupyter-Hub; removed Patch-For-Review.
Ottomata added a subscriber: Ottomata.EditedJun 13 2018, 8:21 PM

@mepps, which part of the login steps aren't working for you? The SSH tunnel part, or the LDAP login part?

The SSH tunnel part should work for sure, as you are in the proper groups. For the LDAP login to work, you must either be in the 'wmf' or 'nda' LDAP group. I assume that since you are a WMF employee, you are in the wmf group already (if you can log into pivot/turnilo, you certainly are).

mepps added a comment.Jun 13 2018, 8:47 PM

@Ottomata just the LDAP login isn't working. I should be in wmf...

@mepps can you find me on IRC (ottomata) or google chat (aotto@wikimedia.org)? Not sure what is wrong then.

Mentioned in SAL (#wikimedia-operations) [2018-06-14T05:47:10Z] <mutante> LDAP - added user mepps to wmf group (T192472)

Dzahn added a comment.EditedJun 14 2018, 5:47 AM

@mepps Try again now. I added you to "wmf". Seems you were actually _not_ in that yet.

@Ottomata thanks for chiming in.. i think this will be it then ^ (There is no onboarding workflow that automatically adds people to 'wmf' group that i know of)

mepps added a comment.Jun 14 2018, 3:23 PM

That was it! I'm in. Thank you @Dzahn and @Ottomata!!

herron closed this task as Resolved.Jun 14 2018, 4:12 PM
herron claimed this task.