@Dzahn: Human-resources added at your request.

@Dzahn Joady has informed me that HR is currently using Asana exclusively, so I am locking down the Phabricator HR project as best as possible while keeping the information that's currently there intact.

@ALantz, who could be a good contact in HR to help solving this task?

@Qgil @JKrauska

After talking with Joady about this: Could IT include Ops with their offboarding form results? Ops has previously requested access to the 2 HR internal tracking docs, so Ariel has access to WMF Contractors and Mark has access to Staffing & Tracking, additionally IT also has access to both docs. If anyone else needs access, they should just contact Joady.

Also, an offboarding alert and IT offboarding form link for Joel Sahleen was sent by HR to the hiring manger, the team admin, OIT & Finance before his last day.

@ALantz I believe it would be most efficient for Ops to have a direct process with HR instead of needing to go through IT to get this critical information.

@JKrauska We are happy to give access to our 2 tracking forms to those in Ops that need this information. Just email Joady if there's anyone that needs to be added for this process.

As for the IT offboarding form, it would be great if Ops was also shared on the results too. If they have the same permissions as Joady and can add in questions as needed that would be great. We'd have one simple process that touches everyone. Can you make this happen?

@ALantz From my perspective we have different teams each wanting to stick to their own process flows and not use each others tools.. :)

Ops would like HR to use Phabricator.
HR would like Ops to use the email form and go review locked down Google Docs.

I added Rob H and Mark to the proposed meeting on Onboarding/Offboarding on the 21st.

Cheers,

Joel

@ALantz: If you guys want to maintain a list of every single ops team member, that is the list that needs access to that data. We should be notified (we shouldn't have to ask) to know when someone is offboarded. We have to remove their access to production.

Production access can include advanced rights and access to services and private data.

As it stands now, this process fails. HR (or OIT) fails to notify operations of the majority of the offobarding. As a result, we have a regular issue of folks having access to things long after they leave.

I've proposed that HR simply create a task in phabricator, to no avail. I'm fine with them even just emailing ALL OF OPERATIONS whenever it happens. Whoever is on clinic duty that week will create the phabricator task and the associated work.

The simple fact is right now nothing happens, and we allow access to essential systems to exist longer than it should. This has been an ongoing problem for years, and operations has attempted (again for years) to get synced up with HR and OIT to no avail.

@RobH:

We could add another two people from Ops to our on and offboarding email notification list. The email notifications currently go to representatives from Administration, Finance, and OIT. Would that help? If so, who would Ops like to designate? We can't add the entire team, as some notifications are confidential initially.

As an aside, you, Mark, and Ariel have access to our staff & contractors tracking documents. Please contact Joady about viewing the documents if you have any difficulty.

@ALantz,

We're attempting to create a procedure that doesn't have any single point of failures, or bottlenecks. While allowing some of us to view the documents is a good first step, it requires now that someone on our team is dedicated to tracking all changes there. That is not a very scalable solution. We try to work in tasks/ticketing to avoid single person bottlenecks.

I understand that the announcements of the offboarding may be confidential, but HR realizes that the second we commit any changes to remove access, those changes are public, correct? So none of it is private once we take any kind of action.

As I can see you are using phabricator directly, is there any possibility of HR putting in a new task (and flagging it as private in security) whenever we offboard someone? We can make a specific project that only operations members and HR can view. So while the task is private to all but ops, anyone in operations can handle the disabling of access. This would better follow the operations clinic duty process. Every week, we designate a different person on our team to be the point person to triage and handle items like this.

Task note: I'll be creating a sub-task off of this for auditing the now shared HR document (shared to some of us) to audit our production access lists/ldap groups/wikitech groups against the document.

@ALantz: Again, thanks for sharing the document, it'll let me make sure we haven't missed anyone!

@RobH:
While creating a task in phabricator is a simple task for many, it's outside of our tools and as such is not a simple, quick task. I'm here to help figure out a compromise as our teams work with different tools. I (and my HR team members) don't have the bandwidth to consistently create those new tasks. I'm here to make sure it doesn't appear like we're ignoring a task that comes up, since many teams do use phabricator :)

To help with the tracking of changes on those docs, we can add you and someone else to our notification email. Could those people then create a ticket?

Sort of question/FYI: We do not know what contractors/temps are given access and need the same type of offboarding. We have never been involved in that process.

Additional Note for Offboarding: Who covers removal of phabricator groups. (This isn't directed to HR, this is a tech side question.)

@ALantz: The problem I have with that is IT needs to know, Ops needs to know, and anyone handling internal tooling (like phabricator) needs to know. As its multi-teams have to interact with this notification, at what point does HR change how they do the notifications, rather than forcing all these other teams to accommodate HR?

The problem with simply emailing a list of folks that someone is gone is anyone can spoof the email with ease. Anyone can change their from fields to make it look like they are sending an email from HR. So then we go from a trusted web service (where you have to authenticate to post), to a simple email.

I'll also repeat that I think sending these notifications to a single person for notification in any department is not a good long term solution.

In T108131#1547779, @RobH wrote:

Additional Note for Offboarding: Who covers removal of phabricator groups. (This isn't directed to HR, this is a tech side question.)

I would like to say the Phabricator admins should be able to, but since ridiculous project edit policies that only allow specific users to edit are tolerated, that could be problematic.

I just cannot think of an easier way for HR to notify multiple related teams of offboarding than to use the task tracking system used by the whole of engineering (and thus ~50% of our organization.) Anything else injects single points of failure.

Isn't it possible to create a Phabricator task via email? Even to a private Space? Maybe we can think of a scenario where HR sends their regular email to N addresses, one of them being the one that will generate the creation of a private task.

I'm saying private just in case there are confidentiality issues, usually related with timing. Maybe the email/task could say i.e. 'embargoed until September 1'. As @RobH says, once someone is removed from groups etc, those actions are public.

In T108131#1548425, @Qgil wrote:

Isn't it possible to create a Phabricator task via email?

Yes: https://www.mediawiki.org/wiki/Phabricator/Help#Using_e-mail

Even to a private Space?

Not yet if I interpret https://secure.phabricator.com/T8493#119804 correctly (and upstream docs do not cover Spaces either). Slightly related: T76563.

In T108131#1548425, @Qgil wrote:

Isn't it possible to create a Phabricator task via email? Even to a private Space? Maybe we can think of a scenario where HR sends their regular email to N addresses, one of them being the one that will generate the creation of a private task.

I'm saying private just in case there are confidentiality issues, usually related with timing. Maybe the email/task could say i.e. 'embargoed until September 1'. As @RobH says, once someone is removed from groups etc, those actions are public.

Heck, I'd even support emails into RT if that solves the problem for now (until we can do the exact same thing with Phabricator). We still use it for a few niche things, and this matches that. It's not optimal, but better than us manually having to check a document periodically - important here is that we get timely notifications.

We should be able to do the private task via email with some of our existing custom logic. We were basically intercepting emails to a specific address in some cases and creating them with a service user. It should be doable in this case if that's something we are interested in.

@chasemp Private tasks via email would be an awesome addition! +1

The downer is source address validation is weak at best and not a substitute for authentication.

Indeed, as Chase points out, we can email into Phabricator. The issue I was raising is that HR has stated that these notices are private and therefore not easily verified.

A random email from someone in HR saying something can be spoofed far more easily than an HR person logging into phabricator and creating a task stating the same thing. Edit Addition: Basically the only way to ensure email validity is to reply to every email, and request a secondary acknowledgement.)

However, we can setup a project/task/rt/phab_space/whatever, if HR agrees to send a single email for every offboarding to that email address. I just didn't want to set that all up in advance, and have HR dislike it. (It seems this was done in the past in phabricator having an HR project.)

Also ANY kind of notice beats checking the spreadsheet alone. So if we retain access to the sheet, we can get said notice, and confirm against the sheet. That would eliminate having the insecurity of email alone be a blocker/issue; as well as provide HR driven notice of offboarding which eliminates operations need to audit the offboard sheet daily.

This is important enough that while I don't want to become the permanent offboarding opsen on the sheet, I'm happy to head up the first dozen offboards on the ops side of things.

@RobH So here's some new news. I just met with Joady. She's been very helpful in restructuring some of the fields in these spreadsheets to help with automation. (eg firstname, lastname) She's also made a new tab to help track departures, and I'm going to help backfill it with names I know of departures.

I'm taking a quick pass on scripting some scraping of the page to generate some more structured/query-able data for all of our teams to work with.

Is anyone from Ops interested in helping me code some of this up to meet all of our needs?
(think API code to auto create private phab tickets, sanity check ssh keys, etc.)

If the google sheet automatically (supported by HR/OIT) fires off events to generate (private) phabricator tasks (via an preferred api or less preferred email route), I think it would quite handily covers everyone's preferred use cases.

That is assuming OIT is fine with either using phabricator, or adding in their own logic to the gsheet that when it fires off our phab task, it fires off theirs as well.

I'm not sure who in operations would best handle the phabricator side. If I cannot come up with a better answer by next Monday's ops meeting, I'll plan to bring it up then.

Let's see what cost we are comparing here:

a) sending a single email per on/off-boarding event

b) writing and maintaining scripts to scrape spreadsheets that will need ongoing maintenance forever and will break every time somebody changes that spreadsheet or there are updates

So update from out of band email converstations with Joel/HR and our hangout this morning.

Joel (as OIT) is currently implementing a script into the google sheets with HR to automatically generate a notification email(s) when someone is on/offboarded. This script will fire the email into phabricator.

The phabricator task will likely be in its own project (not yet 100% determined, but likely) and set to private flags. While they may be able to be made public later, the initial thought is some of these are confidential at the time of sheet update, so keeping the initial task private is likely safer. Once the offboard is complete, the tasks could be made public.

HR and Joel will be working on the updates to sheets and scripting next week (they've already started this week.)

Also since this will be automated, we do not need to add any more ops members to viewing those sheets. Once this process is fully ironed out and the automatic notifications are sent, ops will only login to that sheet for annual audits. (So just having a few of us with access then is fine.)

@RobH I disagree about limited access to the sheets. As Daniel points out, the scripts will eventually/occasionally fail, and we need robust backup options to get as this critical information until a more thorough HRIS (hr information system) is deployed by HR.

This is also why I'd like a partner in Ops to help me write/debug the scripts. :)

--Joel

@JKrauska: You are saying you think every single ops should have access? That was my original stance as well, but then we're asking HR to maintain a ~20 person access list for that. I'm not saying to remove the ops that are on there, but there is no need to maintain a large list for regular audits. (One opsen could simply ask for a data dump once or twice annually.)

I'm not against having all Ops have access mind you, I'm just not willing to fight HR for that.

Also, I didn't realize that IT was asking for Ops support in writing the scripts. If thats the case we'll have to bring this up at the ops meeting.

@RobH Not everyone, but anyone who regularly handles off boarding tasks, for sure.
If Ops is going to depend on these scripts, it would be good for them to understand them and be able to debug/fix them.
They can be fully open source. :)

The entire stance of HR was they didn't want to give ops access as a whole group, and that was the impression i just got from them during our meeting.

It seems we aren't all on the same page.

Edit change: Again, I don't think anyone in ops wants to start maintaining google sheet scripts. I know I certainly do not. The reason I supported this idea was I thought IT was supporting HR using google sheets.

I cannot speak for ops as a whole, hence my suggestion to ask in the ops meeting =]

Long ongoing discussion in IRC.

Joel insists that someone in ops should assist in the writing and maintaining of the script. I maintain that this is OIT, and that discussions with Mark about this lead to ops simply wanting to get an email when folks leave. We do not want to parse, maintain, or support a google document or sheet.

Joel and I are not in agreement, so he'll follow up with Mark directly.

We are a small organization. I feel we need cross team ownership of this process and procedure. I feel it's reasonable to have a developer from IT and Ops both understand and support how this works.

I feel that enforcing strict lines about WHO can help WHOM and HOW is toxic to getting things done.

Until we have a more API driven HRIS, Google Sheets is the best we have to work with.

I guess another thing you'd want to check is who has staff membership/adminship in labs projects when they leave.

Well, I'm told by HR that Joel is still working on this & @JKrauska stated earlier in this thread he was working on it.

He has stated that he wants Ops help, and stated he would follow up with Mark, as I advised we wouldn't have any Ops bandwidth to offer to support google sheet scripting.

Removing myself from this toxic phab ticket, it's not doing any help.

Hi All,

Voice out of the blue on this public ticket. Megan N. and I walked through some scenarios for using Phabricator as a system to help with on-boarding and off-boarding tasks for engineering staff members. Although its a great tool for open team communication, we didn't find it had many options for keeping information private. Because issues of on-boarding and off-boarding can be somewhat sensitive, I wouldn't recommend using Phabricator as tool for managing these tasks.

I'm curious if you have any concerns regarding the privacy level of the data you'll be adding to Phabricator?

[OT] For general information regarding private tickets in Phabricator, see the Spaces documentation, plus we (for example) handle MediaWiki security issues in Phabricator.
Feel free to use the Phabricator Help discussion page on mw.org for any further questions. :)

In T108131#1590377, @srodlund wrote:

Because issues of on-boarding and off-boarding can be somewhat sensitive, I wouldn't recommend using Phabricator as tool for managing these tasks.

I would like to point out that due to the nature of our work everything we do with these accounts in Operations is already public.
So as far as Ops is concerned, as soon as anything for us is actionable and we act on it, that action will be public no matter what. Code changes will be on gerrit, wiki changes will be in logs. We technically can't disable users but keep it secret due to configuration management. The tickets would basically contain the same information. It could be be a matter of timing though i guess. So i think it's just important to make sure the ticket gets created only once it's actionable and not too early but just in time.

@Dzahn
Is there an api/method to get a real-time dump of the wmf group in ldap? Is that simply in operations-puppet somewhere?

You can run ldaplist -l group wmf from labs

@Krenair: Thanks. Can you think of any web or code based approach for that to automated /without/ needing to ssh in and run that from time to time?

That'll give you people's UIDs, although I guess you'll need to check ldaplist -l passwd $uid for some of them to find their mail entry, which would likely be a better link to their identity. ldaplist is from operations/puppet's modules/ldap/files/scripts directory in case you want to make a custom one to return more useful output.

You could probably stick something simple (a tool?) up in labs to query LDAP from. I'm not aware of an existing tool for it.

no experience with labs tools. @mark perhaps here's a good opportunity to have someone in ops help code up something?

Isn't the LDAP server accessible from where you run your scripts? The code basically would probably follow the lines of the LDAP queries in create-dbusers.

I would say "ldaplist" is already the existing tool for it.

25 "An application that implements the functionality of Solaris's ldaplist."
..
37 parser.add_option("-a", "--showattributes", dest="showattributes", help="Show the given attributes")
38 parser.add_option("-r", "--recursive", action="store_true", dest="recursive", help="Recurse netgroups")
39 parser.add_option("--like", action="store_true", dest="like", help="Search for objects that equal or sound like [object-name]")
etc..

and the LDAP server is reachable from any labs instance. so i don't really see the need for another tool. Where do you plan to run your scripts? Is the LDAP server not reachable from there?

In T108131#1591412, @Krenair wrote:

You could probably stick something simple (a tool?) up in labs to query LDAP from. I'm not aware of an existing tool for it.

include ldap::role::client::labs should do the trick

Afraid of sidetracking, but I'm wondering if/how to cover assigned tasks in ticketing systems. For example springle left recently and still has 46 open Phabricator tasks assigned. Any ideas / past experience e.g. in RT (also as people might continue to contribute in their free time)?

I would suggest that tickets are mass re-assigned to 'nobody/up for grabs" unless there was a specific agreement with somebody else taking them. This is the experience i have from RT as well, we would use the tool to mass-edit all tickets at once and just put them back into the pool.

In T108131#1596491, @Aklapper wrote:

Afraid of sidetracking, but I'm wondering if/how to cover assigned tasks in ticketing systems. For example springle left recently and still has 46 open Phabricator tasks assigned. Any ideas / past experience e.g. in RT (also as people might continue to contribute in their free time)?

I think he's still sort of around as a volunteer root, not sure whether he's planning to continue with tasks from phabricator or not though. I had already been going around making sure some of his database-related assigned tickets were in #database so they didn't get lost.

In T108131#1596491, @Aklapper wrote:

Afraid of sidetracking, but I'm wondering if/how to cover assigned tasks in ticketing systems.

In T108131#1596877, @Dzahn wrote:

I would suggest that tickets are mass re-assigned to 'nobody/up for grabs" unless there was a specific agreement with somebody else taking them.

I really really really suggest only someone from the person's old team or their manager do this. PLEASE don't make it automatic / something that @Aklapper does on a routine basis. For example when Reedy was offboarded I wasn't going to unassign him from all of his tasks, but when Chris M was I did. It's a case by case basis and that's why we have smart humans working here (to figure out things computers can't).