Page MenuHomePhabricator

The future of service groups and service users on Labs
Closed, ResolvedPublic

Description

Within the tools project, a service group is a tool. Soon striker will simplify the UI for this, and tools users won't hear the phrases 'service group' or 'service user' ever again.

If we can stop support service groups and services users altogether outside of tools, then the world will become simple. Can we?

Here is the current list of non-tools service groups:

cn=bots.admin,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.asurabot,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.chemobot,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.cluebot,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.coibot,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.czarchivebot,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.linkwatcher,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.logs,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.rezabot,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.stemmeberettigelse,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.unblockbot,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.wmib,ou=servicegroups,dc=wikimedia,dc=org
cn=bots.xlinkbot,ou=servicegroups,dc=wikimedia,dc=org
cn=catgraph.catgraph,ou=servicegroups,dc=wikimedia,dc=org
cn=chasetestproject.testuserone,ou=servicegroups,dc=wikimedia,dc=org
cn=cvn.cvnservice,ou=servicegroups,dc=wikimedia,dc=org
cn=maps.maps,ou=servicegroups,dc=wikimedia,dc=org
cn=rcm.cac,ou=servicegroups,dc=wikimedia,dc=org
cn=toolsbeta.admin,ou=servicegroups,dc=wikimedia,dc=org
cn=toolsbeta.test2,ou=servicegroups,dc=wikimedia,dc=org
cn=toolsbeta.test,ou=servicegroups,dc=wikimedia,dc=org
cn=toolsbeta.toolschecker,ou=servicegroups,dc=wikimedia,dc=org

I would not stake my life on the accuracy of that query but I think it's a good first approximation.

Event Timeline

Andrew created this task.Apr 13 2017, 8:47 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 13 2017, 8:47 PM
Andrew updated the task description. (Show Details)Apr 13 2017, 8:50 PM
Andrew updated the task description. (Show Details)
Andrew updated the task description. (Show Details)Apr 13 2017, 8:58 PM
Andrew added a subscriber: Petrb.

I cleaned up a few obviously-unused groups and deleted them from the lists above. I also emailed @Petrb to ask about the groups in the 'bots' project.

Andrew updated the task description. (Show Details)Apr 13 2017, 9:17 PM

orgcharts and webplatform seem to be defunct. So, outside of bots the remaining possible users are:

cn=catgraph.catgraph,ou=servicegroups,dc=wikimedia,dc=org
cn=cvn.cvnservice,ou=servicegroups,dc=wikimedia,dc=org
cn=maps.maps,ou=servicegroups,dc=wikimedia,dc=org
cn=rcm.cac,ou=servicegroups,dc=wikimedia,dc=org

Short list!

Andrew updated the task description. (Show Details)Apr 14 2017, 1:43 PM

Petrb has confirmed that service groups in the 'bots' project are no longer needed.

Email sent to labs-announce, subject "Does anyone care about service groups?"

Back when we created tools, we implemented tool accounts as a general labs-wide feature, 'service groups.'  A tool is a service group, but any other project can also create a single-use group or user via the 'Manager Service Groups' link in Wikitech.
 I just ran a big report of all uses of this feature outside of tools, and reached out to many of the projects that contain service groups.  The use cases I encountered were:
  1. "I'm a Labs Op and am using that to test a feature for the tools project"
  2. "I used that feature by mistake/I was just clicking around at random"
  3. "That feature seemed like it might be useful but it wasn't and I never cleaned up afterwards" So! I propose deprecate and remove the service group feature outside of Tools. Specifically:
  4. The new tools admin tool (https://toolsadmin.wikimedia.org/) will implement a tool request/approval/creation system that replaces the wikitech 'service groups' feature for tool creation purposes.
  5. Once that is done, I will remove the 'Manage Service Groups' link from wikitech, and disable all wikitech features related to service groups.
  6. I will delete all existing service groups outside of case 1 above
  7. The backend implementation of service groups will remain the same as it is now, and by special request (mostly, again, for case 1), operators will still be able to create/manipulate service groups outside of the Tools project via arduous by-hand commandline machinations. Does that sound OK to everyone? Does anyone want to speak up in favor of preserving this feature?

-Andrew

https://lists.wikimedia.org/pipermail/labs-announce/2017-April/000227.html

https://lists.wikimedia.org/pipermail/labs-l/2017-April/004989.html

All the needs I have had for this feature are in relation to toolsbeta or working on the Tools environment directly fwiw. So +1 from me.

Krinkle added a subscriber: Krinkle.EditedJul 25 2017, 9:17 PM

Please don't delete cvn.cvnservice from CVN project without prior coordination first. This user is in active use.

Our use case is to have the cvn bot service code and artefacts owned and executed by a common user that is not a personal account, root, or nobody. Creating a separate user account on the instances made sense. As a way of ensuring the same user and user group config exists on all our instances, including new instances, I created a service account within the cvn project, and granted access to this user to bot operators that are not roots within cvn.

This is somewhat similar to the mwdeploy/wikidev concept we have in production.

bd808 triaged this task as Normal priority.Mar 6 2018, 4:42 AM

Related changes from T167204: disable service groups for non-tools projects (which I just merged here as a duplicate):

No LDAP tree cleanup has been done, so any projects that have existing service groups will continue to function. Currently only projects with a Striker deployment (i.e. Toolforge) have a user interface for creating new service groups in the LDAP tree.

@bd808 For the CVN use case, do you have a recommendation for how to ensure that a particular service user exists on all instances within a project? As far as I can tell, the only way would be Puppet, which seems overkill for something that simple, and I'd really rather avoid having to maintain a puppetmaster just for that.

bd808 added a comment.Mar 7 2018, 1:13 AM

@bd808 For the CVN use case, do you have a recommendation for how to ensure that a particular service user exists on all instances within a project? As far as I can tell, the only way would be Puppet, which seems overkill for something that simple, and I'd really rather avoid having to maintain a puppetmaster just for that.

CVN is grandfathered in, but we may eventually need to come up with a better solution for other projects. I would say however that the www-data user is pretty much the same thing. We could easily ensure that a few generic service users are available on all VPS instances for this type of usecase.

bd808 closed this task as Resolved.Mar 26 2019, 9:03 PM
bd808 claimed this task.

Per T162945#4026017

  • No cleanup of existing service groups will be done
  • Only Toolforge has the ability for self-service creation of new service groups
  • Other needs for service groups can be discussed on an individual basis