Page MenuHomePhabricator

Striker's GitLab account lookup broken as a result of ODIC migration
Closed, ResolvedPublicBUG REPORT

Description

[19:26]  <    wm-bb> <Jorgemet> Hi. My name is Jorge. My username is Pepe piton. I am new to toolforge and wikimedia gitlab accounts. I have an account and a tool on toolforge and an authrised account on gitlab. But when I go to "Create new GitLab repository" in my tool account, I receive an error message which says "No GitLab accounts found for tool maintainers." I have double checked that I am listed as a mai <clipped message>
[19:26]  <    wm-bb> <Jorgemet> nteiner of the tool. I'd appreciate receiving some guidance to create the repository. Thank you very much!

Investigation shows this is directly related to the ODIC protocol change from T320390: migrate gitlab away from the CAS protocol.

Event Timeline

The error log shows the expected behavior for a lookup when there is no existing target account on the GitLab side. The log event is timestamped 2023-08-03T19:20:22.676942+00:00. The user account at https://gitlab.wikimedia.org/pepepiton was created Aug 1, 2023 10:17pm (timezone uncertain).

The code in question looks like:

def user_lookup(self, uids):
    """Lookup GitHub user data for a list of LDAP uid values."""
    uids = list(filter(None, uids))
    r = {}
    for uid in uids:
        try:
            r[uid] = self.get(
                "users",
                {
                    "provider": settings.GITLAB_PROVIDER,
                    "extern_uid": settings.GITLAB_EXTERN_FORMAT.format(
                        uid
                    ),
                }
            )[0]
        except (APIError, IndexError):
            logger.exception("Failed to lookup user '%s'", uid)
    return r

My hunch is that settings.GITLAB_PROVIDER and/or settings.GITLAB_EXTERN_FORMAT need to be updated as a result of recent activity in T320390: migrate gitlab away from the CAS protocol.

Poking around in the GitLab admin console makes the underlying problem clear. The switch from CAS to OIDC for T320390: migrate gitlab away from the CAS protocol has changed two things about how new accounts need to be looked up:

  • The provider identifier is different (a semi-obvious outcome)
  • The external user id has changed from the Developer account's uid (shellname) to the account's cn (username)
ProviderProvider IDGroupIdentifier
Wikimedia Dev Account (OIDC) (openid_connect)--Pepe piton
Cas3 (cas3)--bd808
bd808 triaged this task as High priority.Aug 3 2023, 8:03 PM
bd808 renamed this task from "No GitLab accounts found for tool maintainers." failure when attempting to create GitLab repo for paulina tool to Striker's GitLab account lookup (and creation?) broken as a result of ODIC migration.Aug 3 2023, 8:06 PM
bd808 updated the task description. (Show Details)

To unblock @Pepe_piton I did this to create their desired GitHub repo:

This can be repeated for other tools as needed, but obviously getting Striker updated is a better plan than replacing self-service automation with a fragile manual process.

bd808 changed the task status from Open to In Progress.Aug 4 2023, 12:19 AM
bd808 claimed this task.
bd808 moved this task from Backlog to Doing on the Striker board.

I have a potential fix for this issue in my local dev environment, but I am running into unrelated issues in completing my testing. The updated Keystone service container added in rLSTRf614deaf5d29: dev: Update Keystone container to "yaga" has a very new upstream build that is experiencing segfault crashes on my laptop.

Change 945880 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[labs/striker@master] gitlab: Use LdapUser objects with GitLab integration

https://gerrit.wikimedia.org/r/945880

Change 945880 merged by jenkins-bot:

[labs/striker@master] gitlab: Use LdapUser objects with GitLab integration

https://gerrit.wikimedia.org/r/945880

Change 946601 had a related patch set uploaded (by BryanDavis; author: Bryan Davis):

[operations/puppet@production] striker: Bump container version to 2023-08-07-172444-production

https://gerrit.wikimedia.org/r/946601

Change 946601 merged by Andrew Bogott:

[operations/puppet@production] striker: Bump container version to 2023-08-07-172444-production

https://gerrit.wikimedia.org/r/946601

I was able to use Striker to create https://gitlab.wikimedia.org/toolforge-repos/bd808-test-T343485 with the changes attached to this task now deployed.

The production deployment of Striker is now configured to search for GitLab users associated with "openid_connect" provider keyed by the Developer account cn property (username). Developer accounts which were attached to GitLab via the prior "cas3" provider will not be found until they receive an "openid_connect" provider association. These associations could be back-filled via the GitLab API, but they will happen automatically as folks re-authenticate to GitLab so maybe we do not need an active backfill process?

bd808 renamed this task from Striker's GitLab account lookup (and creation?) broken as a result of ODIC migration to Striker's GitLab account lookup broken as a result of ODIC migration.Aug 7 2023, 8:57 PM