Page MenuHomePhabricator

Establish a workflow that scales for requesting Phab 2FA resets
Open, MediumPublic

Description

Situation

Problem

  • It does not scale when often only @Aklapper is getting contacted via a private message and @Aklapper is a bottleneck.
  • Recently a reset was performed by an SRE member (@Aklapper appreciates that) - resetter also wondered about the process and if they are entitled to act.
  • Lack of process of who and how to contact multiple folks - usually Phab admins (though not technically required) with shell access who can perform a Phabricator 2FA reset.
  • Lack of guidelines how to verify requests - committed identities on wiki, video calls, maybe WMF Slack messages if staff, etc?

Potential stakeholders

To do

Offtopic for this task: Security ticket access; user renames; adding members to restricted groups; etc

Event Timeline

Aklapper moved this task from To Triage to Policies on the Phabricator board.
Aklapper edited subscribers, added: Bmueller; removed: mseckington.

@Bmueller mentioned that this might also be in the realm of Security-Team. Comments welcome.
I know that nobody is happy about more on their list, but I'd like to reduce single points of failure, and this isn't needed too often.

sbassett moved this task from Incoming to In Progress on the Security-Team board.
sbassett moved this task from Backlog to In Progress on the user-sbassett board.
sbassett subscribed.

@Aklapper - Will plan to review with Security-Team leadership this week.

@sbassett: Thanks. Any updates? This is a bottleneck, e.g. I missed T330073 (and nobody else picked it up either)...

@sbassett: Thanks. Any updates? This is a bottleneck, e.g. I missed T330073 (and nobody else picked it up either)...

The update is an unfortunate one, in that we do not feel we are currently resourced to take on this additional service/workload.

@sbassett: Thanks for the update! @Bmueller: Please share how to best proceed here. Thanks!

Not sure what the process is now but I created a ticket T334480

Aklapper triaged this task as Medium priority.
Aklapper removed a project: Security-Team.

Removing Security-Team per T306708#8661705; assigning to myself

Aklapper added a subscriber: thcipriani.

Removing myself as assignee. Not sure who could drive this to a resolution that scales and is discoverable before I leave.

Paraphrasing/quoting a conversation with @acooper so we have thoughts in one place:

  • Staff 2FA resets are easier, could streamline and have an official policy.
    • Ideally, verification could be performed by someone (the verifier) who knows the affected person and who is able to identify
    • Ideally the staff member performs the verification over a video call, due to concerns about other potential auth systems having (too) long lived tokens
    • The verifier then updates the ticket (if the request was actually brought up in a ticket (see "Define a place where 2FA reset requests can be brought up when user cannot log into Phab anymore" above)
  • For scalability issues with the actual act of enacting the 2FA reset requests, that requires shell/admin permissions (see "That action requires being listed" above)

performs the verification over a video call

To verify someone over video call the verifier would have to know how each given user looks and who they are (and who they report to). I am not sure this is the case (anymore) unless the verifier has Namely access and Namely always has current data and profile photos uploaded by HR?

Namely goes away anyway. :)
It would mean finding a person who both the requestor and the resetter know how they look like, and other way round (latest example).
And that negotiation does not sound like a great use of time for anyone (but sure... security).

Namely goes away anyway. :)

I would hope that Dayforce has similar directory searching capabilities. If it does not, that would be very unfortunate, for a number of reasons.

It would mean finding a person who both the requestor and the resetter know how they look like, and other way round (latest example).
And that negotiation does not sound like a great use of time for anyone (but sure... security).

I'd agree that this isn't a very sustainable option, on its face (no pun intended). I think pairing it with asking a live individual certain questions that might only be available from data on officewiki or within Dayforce might be ok though, and would rely less upon verifying someone based upon their current physical appearance.

I was assuming finding the person would be easy - it would be maybe their manager? Which is easily available from the address book.

Oh and all that is required is for the verifier to also post a message authenticated to their accounted confirming they have verified the identity. The verifier in this case is someone from the persons team or management chain. So its not a fixed identity.

The admin would then only need to verify that the verifier posted an authenticated message (e.g. to a phab ticket comment) confirming they verified the user in question.

You could also require messages of acknowledgement to be sent across at least two authentication domains to increase the security. So say a slack message (linked in a phab comment) and a phab comment.

I was assuming finding the person would be easy - it would be maybe their manager? Which is easily available from the address book.

Are you referring to Namely or another address book?

posted an authenticated message (e.g. to a phab ticket comment)

I think part of the problem is that you don't know who actually created the Phab account in the first place, unless it's linked to a wiki user and that wiki user was created by office IT and you follow the whole chain. I could be wrong though.

Namely or just the gmail synced address book.

I think the problem you referred to about unknown usernames to employee mapping, can also be fixed quite easily and without any significant changes to the proposed process. Say a unknown employee user called PACMAN (I'm just making up a name this doesn't refer to anyone specifically). PACMAN knows who they are. They reach out to their manager for video verification. Once that's been done their manager writes a message confirming that they have validated PACMAN's identity and confirming which employee they are.

I have also been thinking about volunteers who are not employees. This is quite a difficult problem as we don't have many trusted means to verify. I have a few ideas I am thinking about:

  • Have a WMF staff member who knows the volunteer verify them over a video call. HOWEVER, there seems quite a high likelihood they either don't know a WMF staff member, or even if they do, they wouldn't know what they look like.
  • Have 1 or more other community members of good standing (admins?) verify the identity and vouch that it was done.
  • Prove access to the email address associated with the account, combined with waiting for a period of time before performing the reset (maybe 30 days) which allows time for a possible account takeover scenario to be detected and remediated.
  • Prove access to a wikipedia account with the same username by writing a talk message. I am not sure how common it is that the usernames match though?

Also could require some combination of the above. Has anyone got any other ideas?

If I might insert some input from the Service Desk team, we have established our own SOP for reset requests that perhaps may provide a sort of model here.

Essentially, we have two layers of verification for Foundation staff members requesting Okta password or 2fa resets. Of course, the limitation here is this only works if they're Foundation staff who have Okta accounts.

If a staff member can send the request from their Foundation Slack account, we consider this good enough proof of identity to reset their Okta access. The rationale being that you would have needed prior access to the Okta account in order to sign in to that Slack account and users very often are still signed in to Slack, even after forgetting their password, losing their 2fa method, or otherwise becoming locked out of Okta. However, should they not have access to Slack, our next verification step is to reach out directly to the user's manager. Their manager will in turn verify that the request is legitimate and then provide us secondary contact info to correspond with the user.

My thoughts on how this SOP could be adapted here is perhaps if a staff member needs a phabricator 2fa request and SRE needs identity verification before proceeding, then they can interface directly with the user via Slack or with the user's manager to verify the validity of the request and identity of the requester.

I know some of this may not quite apply the same way, since phabricator is different. Just thought I would chime in with some additional ideas.

@JLam-WMF Thanks for this! One small question, out of curiosity, how do you determine who the manager of a user is? Do you look it up in Namely (soon the system replacing it), LDAP or some other way like an orgchart somewhere?

Ah, there's the rub I guess. We have access to that info via Okta.

I guess that creates something of a stumbling block. Though, until we get a better tooling, maybe a stop gap in the SOP here would be you guys can reach out to techsupport@ requesting that info. Talent and Collaboration would also know.

I have been told that Dayforce should eventually obviate the need for this sort of telephone tag, since it can be hooked to Okta and can provide attributes such as manager info. @bcampbell could comment more.