Page MenuHomePhabricator

Requesting access to analytics-privatedata-users group, sql_lab role, Kerberos Principal for Khantstop
Closed, ResolvedPublicRequest

Description

Requestor provided information and prerequisites

Complete ALL items below as the individual person who is requesting access:

  • Wikimedia developer account username: Khantstop
  • Email address: kadeemkhan-ctr@wikimedia.org
  • SSH public key (must be a separate key from Wikimedia cloud SSH access): ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEFkrUgMVVji39VP3BbWWZAP5/w2HJmfK5oKUfK21Ozj wmf3032@WMF3032s-MBP
  • Requested group membership: analytics-privatedata-users
  • Reason for access: As part of my role as a Data Scientist contractor within the Movement Insights team at WMF, I need access to private data to analyze large datasets related to Wikipedia usage and user behavior. This data will inform strategic decisions, support data-driven insights for the Product and Technology Advisory Council. My analyses will provide actionable insights and ensure alignment in understanding among stakeholders.
  • Name of approving party (manager for WMF/WMDE staff): @OSefu-WMF
  • Ensure you have signed the L3 Wikimedia Server Access Responsibilities document: Yes
  • Please coordinate obtaining a comment of approval on this task from the approving party.

SRE Clinic Duty Confirmation Checklist for Access Requests

This checklist should be used on all access requests to ensure that all steps are covered, including expansion to existing access. Please double check the step has been completed before checking it off.

This section is to be confirmed and completed by a member of the SRE team.

  • - User has signed the L3 Acknowledgement of Wikimedia Server Access Responsibilities Document.
  • - User has a valid NDA on file with WMF legal. (All WMF Staff/Contractor hiring are covered by NDA. Other users can be validated via the NDA tracking sheet)
  • - User has provided the following: developer account username, email address, and full reasoning for access (including what commands and/or tasks they expect to perform)
  • - User has provided a public SSH key. This ssh key pair should only be used for WMF cluster access, and not shared with any other service (this includes not sharing with WMCS access, no shared keys.)
  • - The provided SSH key has been confirmed out of band and is verified not being used in WMCS.
  • - access request (or expansion) has sign off of WMF sponsor/manager (sponsor for volunteers, manager for wmf staff)
  • - access request (or expansion) has sign off of group approver indicated by the approval field in data.yaml

For additional details regarding access request requirements, please see https://wikitech.wikimedia.org/wiki/Requesting_shell_access

Event Timeline

Approved as @Khantstop's (Kadeem Khan's) manager

Hi @Khantstop, I think you're a contractor - can you or @OSefu-WMF confirm the contract end date, please, so I can note it in the list of shell accounts, please?

Other than that, I just need to confirm the ssh key out-of-band and this will be good to go.

Thanks @MatthewVernon contract end date is Nov 4, 2025

Change #1088574 had a related patch set uploaded (by MVernon; author: MVernon):

[operations/puppet@production] admin: add analytics-privatedata-users + krb user khantstop

https://gerrit.wikimedia.org/r/1088574

Change #1088574 merged by MVernon:

[operations/puppet@production] admin: add analytics-privatedata-users + krb user khantstop

https://gerrit.wikimedia.org/r/1088574

MatthewVernon claimed this task.

Done, and Kerberos principal created.

@MatthewVernon are you able to add me to sql_lab role as well? I don't think this was granted but let me know if I'm mistaken.

Also, I went through all the approvals and ensured my credentials are correct but it seems I'm unable to access JupyterHub. Do you know what the issue could be?

{F57689691}

Hi @BTullis, I hope all is well. I’m trying to gain access to sql_lab and superset, are you able to help me with this? Please let me know if I should file a specific ticket with an associated tag and I’d be happy to do so. Many thanks!

mpopov removed MatthewVernon as the assignee of this task.
mpopov added a project: Data-Platform-SRE.
mpopov subscribed.

@Khantstop has reported that trying to kinit results in error

kinit: krb5_get_init_creds: unable to reach any KDC in realm WIKIPEDIA, tried 0 KDCs

Reopening, and I think this is now specifically for Data-Platform-SRE to assist with.

elukey subscribed.

I am removing the SRE tag on this, Data Platform SREs are the right target for the last requests :)

@Khantstop - could you possibly paste some command output or a screenshot, please?
This is what I see when I go to stat1008 and run kinit

(base) btullis@marlin:~$ ssh stat1008.eqiad.wmnet 
Linux stat1008 5.10.0-30-amd64 #1 SMP Debian 5.10.218-1 (2024-06-01) x86_64
Debian GNU/Linux 11 (bullseye)
stat1008 is a Statistics & Analytics cluster explorer (private data access, no local compute) (statistics::explorer)
stat1008 is statistics::explorer
Bare Metal host on site eqiad and rack A6
This host is capable of Kerberos authentication in the WIKIMEDIA realm.
For more info: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Kerberos/UserGuide
The last Puppet run was at Tue Nov 26 13:24:59 UTC 2024 (26 minutes ago). 
Last Puppet commit: (b9a0aff6c5) David Caro - cloudcephmon1004: provision as mon
Debian GNU/Linux 11 auto-installed on Thu May 23 11:03:16 UTC 2024.
Last login: Tue Nov 26 13:51:14 2024 from 2a02:ec80:600:1:185:15:58:6

You do not have a valid Kerberos ticket in the credential cache, remember to kinit.
btullis@stat1008:~$ kinit
Password for btullis@WIKIMEDIA: 
btullis@stat1008:~$

At this point I have authenticated and I can verify this by running klist.

btullis@stat1008:~$ klist
Ticket cache: FILE:/tmp/krb5cc_32741
Default principal: btullis@WIKIMEDIA

Valid starting       Expires              Service principal
11/26/2024 13:51:46  11/28/2024 13:51:38  krbtgt/WIKIMEDIA@WIKIMEDIA
	renew until 12/03/2024 13:51:46

You mentioned that you received the error:

kinit: krb5_get_init_creds: unable to reach any KDC in realm WIKIPEDIA, tried 0 KDCs

Were you asked to enter your kerberos password before receiving the error, or did this happen immediately upon running kinit?
If you do enter your password, what happens if you intentionally get it wrong? Is the error message the same?

If you could let us know any more about the issue, I'd be grateful. Thanks.

Hi @BTullis, I identified and resolved the issue while attempting to execute the the steps you outlined. We can close the ticket.

Here’s an overview:

Issue Summary
I encountered an issue while attempting to log in with kinit after setting up an SSH tunnel using the command:
ssh -N stat1008.eqiad.wmnet -L 8880:127.0.0.1:8880

Running kinit resulted in the error:
kinit: krb5_get_init_creds: unable to reach any KDC in realm WIKIMEDIA, tried 0 KDCs

However, when I directly SSH’d into stat1008.eqiad.wmnet using the following which you mentioned:
ssh stat1008.eqiad.wmnet

I was able to log in successfully with kinit.

Root Cause
The issue was caused by differences in the network environment:
• When using the SSH tunnel (ssh -N), kinit was executed locally on my machine, which does not have direct access to the Kerberos Key Distribution Centers (KDCs) for the WIKIMEDIA realm due to network routing or firewall restrictions.
• When logging in directly to stat1008.eqiad.wmnet, the Kerberos client on that host was able to reach the KDC servers (krb1001.eqiad.wmnet and krb2001.codfw.wmnet) because it resides in the same trusted network.

Great! Glad to hear that it's working. If you think our docs could be clearer, or if there is anything missing from them, please feel free to let me know, or simply edit them.