Page MenuHomePhabricator

Transition service groups to new globally unique names and UIDs
Closed, ResolvedPublic

Description

The current service group scheme, while workable, has design issues that cause problems with services global to all projects (NFS and databases being the primary examples).

Proposed new implementation:

  • Globally unique UID/GIDs (that, I believe, is already the case)
  • Names in the form $projectname.$groupname (rather than local-$groupname)
  • All service groups under a single OU (and not per-project OUs)

Implicit:

  • usernames and group names must disallow '.'

Ideally, the division on what system to use should be per-region so that functionality in pmtpa is not impaired while the new system is geared up in eqiad.

(The change from one to the other implies changes in many system settings/scripts, not all of which could be tweaked to understand both schemes).


Version: unspecified
Severity: enhancement

Details

Reference
bz58997

Event Timeline

bzimport raised the priority of this task from to Needs Triage.
bzimport set Reference to bz58997.
bzimport added a subscriber: Unknown Object (MLST).
coren created this task.Dec 27 2013, 4:39 PM

Can you tell me more about the motivation for this? Could the need for this refactor be obviated with a couple of ldap tools that do the lookups?

coren added a comment.Dec 27 2013, 5:47 PM

There are a number: NFS expect usernames and group names to match between client and server when not using Kerberos, that's obviously not possible with non-unique names (right now it's hacked around by having a cron job read from ldap and construct a local /etc/group and /etc/passwd based on it; that's not robust, and very hacky)

Gerrit, otoh, needs to have a single source of group names and cannot combine things coming from more than one OU at all (unique or not).

Ryan had a couple of other scenarios where having a list of unique service groups names was desirable or necessary.

Gerrit was honestly the main motivation for me. It would allow service groups to maintain gerrit repos.

Hm... what will happen to service users and home directories during this change?

Presumably there's no need for the homedir to move. But, the usernames are local-<groupname> currently, right? What are the implications of renaming a user but leaving the UID the same?

coren added a comment.Dec 30 2013, 8:30 PM

For /most/ thing it's a noop. There are a few tools that will not like groups and users being renamed (mostly those that match against /^local-.*/ and such), but I'll perfrom a hunt-and-seek for those before switching which OU the instances use.

ok -- I've started to code to support both schema at the same time, but the special usernames seem like a stumbling block. We can't support both of those at once without creating extra users and uids, which would just make things worse.

scfc added a comment.Jan 6 2014, 6:51 PM

(For reference: Announcement on labs-l: http://permalink.gmane.org/gmane.org.wikimedia.labs/1652 (2013-09-16).)

scfc added a comment.Feb 6 2014, 10:40 AM

As I suppose this requires converting the records for service groups from one LDAP section to another, there are two housekeeping items that could be bundled with that:

  • Fixing the "chown" rule in sudoers (cf. Gerrit change #111755),
  • trimming trailing slashes on the home directories (cf. bug #54074, comment #3).
scfc added a comment.Mar 21 2014, 1:23 AM

List of members in old, but not new service groups in the Tools project:

  • acc-utilities: deltaquad
  • anagrimes: jackpotte, psychoslave
  • arkivbot: profoss
  • betaweb: bharris
  • bookmanagerv2: danilo, mollywhite
  • catmonitor: profoss
  • checkpersondata: sk
  • citation-bot: smith609
  • citations: maximilianklein, smith609
  • citeimage: dominic
  • cluestuff: damian, legoktm
  • cobain: arnaugir, coet, madutgn
  • codelookup: robin
  • common-interests: cyberpower678
  • connectivity: jkroll, lvova
  • csbot: laner
  • cyberworm: wormtt
  • dispatcher: yuvipanda
  • doi-bot: maximilianklein
  • dplbot: jason
  • dumpscan: valhallasw, yuvipanda
  • editcountitis: legoktm
  • geohack: kolossos
  • geoloc: gretal, marianneh, pidancier
  • grantsbot: ciphers
  • grouplens: auduwage
  • hasteurbot: earwig, theopolisme
  • intuition: siebrand
  • logger: petrb, yuvipanda
  • logs: marc
  • mahdiz: ladsgroup, reza
  • math: mattflaschen, ori
  • mfw-bww: mf-warburg
  • morebots: andrew, dzahn, jeremyb, mattflaschen, ori
  • mwp: earwig
  • nara: dominic
  • periodssuck: yuvipanda
  • quentintools: quentinv57
  • quentinv57-tools: quentinv57
  • reasonator: ladsgroup
  • render: daniel, knissen
  • render-tests: daniel, knissen
  • repi: valhallasw
  • revisionstats: mahmoud
  • robin: robin
  • stats: legoktm, yuvipanda
  • suchaserver: legoktm
  • sulinfo: johnflewis, quentinv57
  • svenbot: svenmanguard
  • taweetham: nullzero
  • testwikistats: steinsplitter
  • typoscan: bgwhite
  • ukbot: zache-tool
  • unblock: hersfold, martijn, thehelpfulone
  • usrd-tools: fredddie, happy5214, tcn7jm
  • voxelbot: vacation9
  • weeklypedia: mahmoud
  • wikicaptcha: cristiancantoro
  • wikidata-analysis: vrandezo
  • wikifeeds: daniel
  • wikimetrics: erosen, milimetric
  • wikitest-rtl: adamw, amire80, ori
  • wikt-mwtest: jackpotte, psychoslave
  • wm-metrics: lena, pierreselim
  • wmk-dev: mistrx
  • wmk-tools: mistrx
  • wyimportbot: hazard-sj
  • xstools: tparis
  • xtools: tparis
  • yifeibot: gabrielchihonglee, steinsplitter
scfc added a comment.Apr 10 2014, 12:37 AM

I've refreshed all service groups in comment #9 ([[wikitech:Special:NovaServiceGroup]], "Manage members", "[Submit]"), so all service group memberships are now in sync apart from tools.local-awb which is the subject of bug #63754.

Change 142051 had a related patch set uploaded by Krinkle:
Purge support for the old-school "local-*" service groups

https://gerrit.wikimedia.org/r/142051

Change 142051 had a related patch set uploaded by Andrew Bogott:
Purge support for the old-school "local-*" service groups

https://gerrit.wikimedia.org/r/142051

Change 142051 merged by jenkins-bot:
Purge support for the old-school "local-*" service groups

https://gerrit.wikimedia.org/r/142051

All groups are renamed and the GUI now reflects the new <project>-<toolname> scheme.

Anything left to do here?

scfc added a comment.Jul 14 2014, 7:41 PM

I think this bug can be closed.