I've reached out to @KCVelaga_WMF on slack to verify the provided ssh key out of band.

@mpopov is this ok with you? I'm not sure in this case if I need to confirm as the new account is just replacing an existing one. But best to follow process I guess. Your approval is also needed for the analytics-product-users group.

@odimitrijevic @WDoranWMF similar I assume it's fine to do this, new account kcvelaga to replace existing one kcv-wikimf with access to analytics-privatedata-users.

Thanks!

Adding a note, if possible, it would be helpful to have access to both the accounts for a couple of working days, so that I can ensure that everything is working, without getting blocked from being able to do regular work.

In T358658#9590829, @KCVelaga_WMF wrote:

Adding a note, if possible, it would be helpful to have access to both the accounts for a working couple days, so that I can ensure that everything is working, without getting blocked from being able to do regular work.

That should be no problem. Also I have confirmed the ssh key out of band so we are good to go pending approvals.

@KCVelaga_WMF I will need to follow up on this on Monday.

I've been looking at the setup and preparing patches to enable the new shell account. But I'm not sure that will be possible as we normally use the same uid as the LDAP account, which is not changing. I will discuss with colleagues who are back Monday to confirm the best way forward.

If we have to rename the existing account it will should be fine anyway, as it already has the permissions you need.

Approved from my side, both for the request in general and analytics-product-users membership :)

Thank you @cmooney for taking this non-standard case on and helping KC out! This dual account thing has become a real thorn for KC so I'm glad we're on a path to get it taken care of.

So, in order to move over the access from the existing kvc-wikimf account to kcvelaga we would need to do the following:

1. The kcvelaga Wikimedia Developer Account is currently associated with a gmail address, this needs to be changed to the wikimedia.org address. @KCVelaga_WMF You can do this by logging into https://idm.wikimedia.org/ and then under "e-mail" use the "Update" dialogue.

2. Moving from kvc-wikimf to kcvelaga will change your UID and thus the ownership of files. After merging the change to change your ownership we need to update the ownership of your files to the new username/UID. Which servers are you typically logging into (e.g. which stat hosts), for which we should change your ownership after updating the user name? Do you in addition have files stored in HDFS, which need to be migrated?

We don't need re-approval of the group ownerships for the rename. The process would look like this:

1. Add the group memberships to kcvelaga along with an updated user account
2. Change ownership of the files idenfied above
3. After a grace period of a few days when you've confirmed you can access all files correctly under the new username, you give us a headsup and the access for kvc-wikimf gets removed.

Change 1008450 had a related patch set uploaded (by Cathal Mooney; author: Cathal Mooney):

[operations/puppet@production] Add shell user for kcvelaga, mirroring kcv-wikimf

https://gerrit.wikimedia.org/r/1008450

@MoritzMuehlenhoff When I change my email to wikimedia.org for the developer account, I am encountering an error trying to login into other services for example, gitlab.wikimedia.org. The error message says email address has already taken, although kcv-wikimf doesn't have any associated email nor SUL account.

@KCVelaga_WMF : That is expected, your kcvelaga account isn't yet part of the cn=wmf LDAP group, it will be when https://gerrit.wikimedia.org/r/1008450 is merged

@MoritzMuehlenhoff Ah okay! Thanks for clarifying. Also, to answer your second question, all of my work is on stat1005 and HDFS file under database, kcvelaga.

Change 1008450 merged by Cathal Mooney:

[operations/puppet@production] Add shell user for kcvelaga, mirroring kcv-wikimf

https://gerrit.wikimedia.org/r/1008450

In T358658#9596140, @MoritzMuehlenhoff wrote:

@KCVelaga_WMF : That is expected, your kcvelaga account isn't yet part of the cn=wmf LDAP group, it will be when https://gerrit.wikimedia.org/r/1008450 is merged

I've merged the patch and added kcvelaga to the wmf ldap group.

@KCVelaga it will take approx 30 minutes for the update to be pushed out to all hosts, I believe you should be able to for instance access stats1005 with the new user at that stage.

Yes, approved

In T358658#9598742, @odimitrijevic wrote:

Yes, approved

Thanks Olja.

Just to update I've been working with KC on this and we've been making progress. The biggest outstanding problems are the login does not work on the gerrit or gitlab web interfaces, which I need to discuss with others in SRE to try to get to the bottom of.

In T358658#9596119, @KCVelaga_WMF wrote:

@MoritzMuehlenhoff When I change my email to wikimedia.org for the developer account, I am encountering an error trying to login into other services for example, gitlab.wikimedia.org. The error message says email address has already taken, although kcv-wikimf doesn't have any associated email nor SUL account.

When looking at GitLab I see two accounts:

KCVelaga linked to your gmail.com address and
KCVelaga (wikimf) linkted to your kcvelaga-ctr@wikimedia.org address and kcvelaga@wikimedia.org as the secondary email.

My guess is GitLab has problems because the new wikitech user KCVelaga is linked to your gmail address and not the new kcvelaga@wikimedia.org address and your kcvelaga@wikimedia.org is used in a different account.

I removed the secondary email from KCVelaga (wikimf) and updated KCVelaga with the new kcvelaga@wikimedia.org mail.
@KCVelaga_WMF can you try again to login to GitLab?

Taavi advised on IRC about the gerrit issue:

gerrit enforces that user emails are unique. they need to update the email on the old account to something else, the gmail plus syntax (kcvelaga+old@wikimedia.org) should be enough

Thanks @Jelto! GitLab works. I mistakenly assumed that updating the email at idm.wikimedia.org will get reflected across the board.

@cmooney I am unable change on gerrit because of LDAP association.

Error 409 (Conflict): Cannot remove e-mail 'kcvelaga@wikimedia.org' which is directly associated with LDAP authentication

Endpoint: /accounts/self/email/*

So maybe we can try it after removing permissions for kcv-wikimf in a couple of days.

@KCVelaga_WMF Can you try logging into https:/idm.wikimedia.org with your old account? Under "e-mail" you can click "Update" which will allow you to change your email to kcvelaga+old@wikimedia.org.

The +old is a Gmail-internal prefix, such mail will still be routed to your email account, but from the perspective of Gerrit kcvelega+old and kcvelaga are different adresses

What's up with Gerrit

I also see two accounts in Gerrit:

• KCVelaga (wikimf) – primary email kcvelaga@wikimedia.org, secondary email kcvelaga-ctr@wikimedia.org
• KCVelaga – primary email is your gmail, no secondary email

So there is no cn=KCVelaga (wikimf) in ldap anymore is that correct (or am I searching incorrectly)? If that's the case, I understand what's happening.

1. You are trying to login with KCVelaga, Gerrit matches that against it's gerrit schema.
2. It finds the matching account, and continues to authenticate via ldap
3. This works, but the email address has changed
4. Gerrit attempts to update your record with the new email address
5. This fails since the email is already associated with the KCVelaga (wikimf) user in gerrit's notedb

Fixing the user in Gerrit (maybe)

Do I understand correctly that the desired end-state is to use the KCVelaga login?

If so it sounds like we need to cook up some variant of our rename scripts.

We'll need to ensure:

1. Remove the new KCVelaga record from the gerrit schema and username schema
2. We rename the old KCVelaga (wikimf) user in the gerrit schema AND the username schema (since it seems the old sn is also different from the new sn)
3. Reindex accounts via ssh -p 29418 gerrit.wikimedia.org -- gerrit index start accounts --force

This will be some tricky git surgery. And I'd like someone else with more up-to-date gerrit knowledge to weigh in. @hashar is out until next week—can this wait until then?

User rename problems

I note we still have documentation on wiki which reads:

We do not rename users (Developer accounts) anymore. It can (and has) lead to various problems and errors all over the many separate systems which consume Developer accounts as their local databases and authentication methods will get out of sync. We can reconsider when we have better tooling for identity management." (source)

I realize this is out-of-sync with implicit documentation on Editing info about a user. But, as this task shows, it's still not trivial to rename users, and I'd ask that we update the documentation on Wikitech to strongly discourage it.

Making a new user is the way to change a username; however, that means the work associated with the old user will not follow the new user. It's possible to update your display name in Gerrit and to add a secondary email, that should cover many use cases. I know this is unideal in some situations, which we can handle on a case-by-case basis.

As @MoritzMuehlenhoff suggested, I have updated my email to kcvelaga+old@wikimedia.org at idm.wikimedia.org, which is now being reflected on Gerrit, and I am able to successfully login with KCVelaga on Gerrit now. So I think that resolves the last bit of the migration.

@KCVelaga_WMF great news!

I think the next steps would be to move any files you have. I can do this for the stats boxes or other Linux systems. The HDFS stuff I’ll need to get our data engineering colleagues to help with.

In either case if you could advise on exactly what needs to be moved we can get started. Thanks!

@cmooney I have moved over the files from stat1005:kcv-wikimf to stat1008:kcvelaga, and everything is working fine.

After a couple of days for the old user can be removed.

In T358658#9605712, @KCVelaga_WMF wrote:

@cmooney I have moved over the files from stat1005:kcv-wikimf to stat1008:kcvelaga, and everything is working fine.

Even easier, thanks!

After a couple of permissions for the old user can be removed.

You mean after a couple of days? Let me know if you need anything from us anyway :)

Oh yes, days, thank you.

@cmooney all permissions and access for kcvelaga are working fine without any trouble, permissions/access for LDAP user KCVelaga (wikimf) and shell user kcv-wikimf can be removed.

Thanks for confirming @KCVelaga_WMF. I’ll get that done over the next day or so; we have our annual SRE meet up this week but I should get a few minutes to make the changes.

Change 1011187 had a related patch set uploaded (by Dzahn; author: Dzahn):

[operations/puppet@production] admin: absent user kcv-wikimf, renamed to kcvelaga

https://gerrit.wikimedia.org/r/1011187

Change 1011187 merged by Dzahn:

[operations/puppet@production] admin: absent user kcv-wikimf, renamed to kcvelaga

https://gerrit.wikimedia.org/r/1011187

In T358658#9618875, @KCVelaga_WMF wrote:

@cmooney all permissions and access for kcvelaga are working fine without any trouble

Great, thanks for confirming that.

permissions/access for LDAP user KCVelaga (wikimf) and shell user kcv-wikimf can be removed.

This is also done now with the change merged above.

So we should be able to close this as resolved now.