Page MenuHomePhabricator

Getting parsing-team members sudo access to manage (start, stop, restart) services on ruthenium
Closed, ResolvedPublic

Description

On ruthenium, the parsing team members need ability to start/stop/restart service (parsoid-rt, parsoid-vd, parsoid-rt-client, parsoid-vd-client, parsoid, and diffservice later). & also the ability to periodically refresh code for rt-testing in /usr/lib/parsoid.

The proposed changes are on https://gerrit.wikimedia.org/r/#/c/266632/ which include the following group:

parsoid-rt-admin:
  gid: 771
  description: parsing team members for parsoid regression testing adminstration
  members: []
  privileges: ['ALL = NOPASSWD: /usr/sbin/service parsoid *',
             'ALL = NOPASSWD: /usr/sbin/service parsoid-rt *',
             'ALL = NOPASSWD: /usr/sbin/service parsoid-rt-client *',
             'ALL = NOPASSWD: /usr/sbin/service parsoid-vd *',
             'ALL = NOPASSWD: /usr/sbin/service parsoid-vd-client *',
             'ALL = NOPASSWD: /usr/sbin/service diffservice *',
             'ALL = (parsoid-rt) NOPASSWD: /home/parsoid-rt/update-code.sh']

The following users are requested to add to the above group: @ssastry, @tstarling, @Arlolra, @cscott, @GWicke.

Both the group permissions and the addition of the above users to the group will require operations meeting review (due to sudo requests). The next meeting is on 2016-02-01.

Details

Related Gerrit Patches:
operations/puppet : productionadmin: add arlolra,cscott,gwicke to parsoid-test-admins
operations/puppet : productionadmin: add parsoid-test-admins to ruthenium
operations/puppet : productioncreation of parsoid-test-admins group

Event Timeline

ssastry created this task.Jan 25 2016, 8:07 PM
ssastry raised the priority of this task from to Needs Triage.
ssastry updated the task description. (Show Details)
ssastry added projects: Operations, Parsing-Team.
ssastry added a subscriber: ssastry.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptJan 25 2016, 8:07 PM
ssastry renamed this task from Getting parsing-team memebrs sudo access to manage (start, stop, restart) services on ruthenium to Getting parsing-team members sudo access to manage (start, stop, restart) services on ruthenium.Jan 25 2016, 8:08 PM
ssastry set Security to None.

Also need access to open mysql CLI and view / modify the databases used in testing.

Also need access to open mysql CLI and view / modify the databases used in testing.

T124703: Need databases provisioned for parsoid-rt testing, visual diff testing is related to this.

jcrespo added a subscriber: jcrespo.
RobH assigned this task to ssastry.Jan 27 2016, 12:03 AM
RobH added a subscriber: RobH.

It sounds like a new usergroup needs to be created for the administration of the ruthenium parsoid regression test server. Perhaps we should call it parsoid-rt-admins and set it up similar (but not identical) to parsoid admins:

parsoid-rt-admin:
  privileges: ['ALL = NOPASSWD: /usr/sbin/service parsoid *',
               'ALL = NOPASSWD: /usr/sbin/service parsoid-rt *',
               'ALL = NOPASSWD: /usr/sbin/service parsoid-rt-client *',
               'ALL = NOPASSWD: /usr/sbin/service parsoid-vd *',
               'ALL = NOPASSWD: /usr/sbin/service parsoid-vd-client *',
               'ALL = NOPASSWD: /usr/sbin/service diffservice *',
               'ALL = (parsoid-rt) NOPASSWD: /home/parsoid-rt/update-code.sh']

I've not added any sudo rights like this in awhile, so I'd like to get a fellow opsen to review the above suggestion for validity.

If this is correct, it is a sudo group, so we'll need to get approval in the ops meeting to add anyone to it. As it happens, I'm on clinic duty this week, so I would relay this info into said meeting.

@ssastry: Please review the above and see if it would work for your request. If so, please list off all users you wish to add to this group. If those users have NOT gotten shell access yet, we'll need to make independent tasks for each new shell request. If they all have shell access (or the ones that already do) you can keep grouped into this single task to add them.

Please advise. I've assigned the task back to you for your input on what users you are requesting have those rights. Additionally, I'll need your input if my proposed sudo rights above would work. I realize this doesn't address your request of "Also need access to open mysql CLI and view / modify the databases used in testing."

Please assign back to me once you provide feedback. Thanks!

@RobH Thanks. Looks good.

@tstarling, @Arlolra, @cscott are the others besides me that will need access. Let us also add @GWicke (as honorary parsoid alumnus :) ) but mostly so we have someone outside our team who is familiar with this setup and can manage these if ever required.

If there are any additional services that we discover that we might need access to, I'll either update this ticket or open a new phab ticket. But, this should get us off the starting blocks.

Change 266632 had a related patch set uploaded (by RobH):
creation of parsoid-rt-admin group

https://gerrit.wikimedia.org/r/266632

RobH claimed this task.Jan 27 2016, 12:27 AM
RobH updated the task description. (Show Details)

Please note my patchset does NOT include the actual users yet. I plan to append them in post ops meeting review (once they have been approved in the meeting.)

I'll keep this task assigned to me until then.

RobH changed the task status from Open to Stalled.Jan 27 2016, 12:28 AM
RobH reassigned this task from RobH to ssastry.Jan 27 2016, 1:07 AM

@ssastry: Can you have your manager approve the request to have these expanded rights bestowed upon the folks listed in the task description: @ssastry, @tstarling, @Arlolra, @cscott, @GWicke.

I neglected to request that from you when I assigned it to you earlier today. Once we have manager approval please assign back to me and I'll handle in ops meeting next Monday, thanks!

I've asked @TrevorParscal to approve.

But, one other sudo permission required is for accessing systemd journals .. so journalctl access is required.

RobH added a comment.Jan 27 2016, 7:20 PM

I've updated the patchset to include:

'ALL = NOPASSWD: /bin/journalctl *' which allows log access to all users. @dzhan was kind enough to provide the info and background. (It seems we don't really limit admin users to specific service logs, we just let them see the logs or we dont.)

ssastry reassigned this task from ssastry to RobH.Jan 27 2016, 9:35 PM
ssastry added a comment.EditedJan 30 2016, 10:18 PM

Can parsoid-rt-admin members get mysql client access to the testreduce_0715 and testreduce_vd databases from ruthenium (or wherever else is appropriate)? (cc @jcrespo).

Separately, does it make sense to use the existing parsoid-admin as the group for all these access requests (since not all test setups are strictly about rt-testing .. see T125166#1983578)?

Or, should I file a separate ticket?

RobH added a comment.Feb 1 2016, 8:21 PM

DB rights should likely be a different ticket, since they require @jcrespo to review and you likely rather not block the rest of this on db rights, correct? Otherwise this task will be delayed, as its adding db rights, shell rights, etc... (So I would split the db rights to an independent task if I were you.)

Additionally, there was confusion to the processing of this task during the operations meeting, but I'm not certain what clarification needs to take place.

This request will add the ability to start/stop/restart service (parsoid-rt, parsoid-vd, parsoid-rt-client, parsoid-vd-client, parsoid, and diffservice later). & also the ability to periodically refresh code for rt-testing in /usr/lib/parsoid. Additionally, this request will include giving access to all log files on ruthenium to the parsoid-rt-admins.

If any of the above is incorrect, please correct me.

Got it. I'll split the db access into a separate ticket.

And, you are right about the access request.

My only suggestion is to use the existing parsoid-admins group instead of creating a new one .. but, I don't care necessarily.

RobH added a comment.EditedFeb 1 2016, 8:32 PM

IRC Discussion Update: I pointed out how appending those rights to the existing parsoid-admins would mean then appending @ssastry, @tstarling, @Arlolra, @cscott, @GWicke into parsoid-admins. (Some of them have that, some do not, so it would be a larger privilege escalation than just creating the new group.)

We could also rename the new group parsoid-testing-admins rather than parsoid-rt-admins.

RobH removed RobH as the assignee of this task.Feb 1 2016, 8:48 PM
RobH added a subscriber: mark.

My understanding (though I could be mistaken) is with the clarification above, this now only needs one of the following:

A) Ops team meeting review (this was attempted today but there was confusion regarding the scope of the two pending sudo tasks.)
B) @mark's approval to proceed

I'm removing myself as the assignee (I was assigned as part of ops clinic duty last week) and leaving unassigned in the queue so it is processed normally.

ssastry triaged this task as High priority.Feb 1 2016, 8:50 PM
Dzahn added a subscriber: Dzahn.Feb 5 2016, 9:39 PM

We could also rename the new group parsoid-testing-admins rather than parsoid-rt-admins.

I did (almost) that and amended it to be called "parsoid-test-admins", to match "parsoid-test-roots" which we have meanwhile.

Dzahn claimed this task.Feb 5 2016, 9:39 PM

Change 266632 merged by Dzahn:
creation of parsoid-test-admins group

https://gerrit.wikimedia.org/r/266632

Change 268808 had a related patch set uploaded (by Dzahn):
admin: add arlolra,cscott,gwicke to parsoid-test-admins

https://gerrit.wikimedia.org/r/268808

Change 268809 had a related patch set uploaded (by Dzahn):
admin: add parsoid-test-admins to ruthenium

https://gerrit.wikimedia.org/r/268809

Change 268809 merged by Dzahn:
admin: add parsoid-test-admins to ruthenium

https://gerrit.wikimedia.org/r/268809

Dzahn added a comment.Feb 5 2016, 10:11 PM
  • merged the new empty group created by rob, just renamed to parsoid-test-admins
  • added group on ruthenium
  • made change that adds arlolra, cscott and gwicke to that group
  • the request also asked for tstarling and ssastry to be added but it seemed redundant, they both already have root on this machine

The actual access request for Monday is now https://gerrit.wikimedia.org/r/#/c/268808/

Dzahn changed the task status from Stalled to Open.Feb 5 2016, 11:57 PM

Change 268808 merged by Dzahn:
admin: add arlolra,cscott,gwicke to parsoid-test-admins

https://gerrit.wikimedia.org/r/268808

Dzahn closed this task as Resolved.Feb 8 2016, 6:07 PM

@Arlolra @cscott @GWicke

you have new permissions to control services on ruthenium now

[ruthenium:~] $ grep test-admins /etc/group
parsoid-test-admins:x:773:arlolra,cscott,gwicke

[ruthenium:~] $ sudo cat /etc/sudoers.d/parsoid-test-admins 
# This file is managed by Puppet!

%parsoid-test-admins ALL = NOPASSWD: /usr/sbin/service parsoid *
%parsoid-test-admins ALL = NOPASSWD: /usr/sbin/service parsoid-rt *
%parsoid-test-admins ALL = NOPASSWD: /usr/sbin/service parsoid-rt-client *
%parsoid-test-admins ALL = NOPASSWD: /usr/sbin/service parsoid-vd *
%parsoid-test-admins ALL = NOPASSWD: /usr/sbin/service parsoid-vd-client *
%parsoid-test-admins ALL = NOPASSWD: /usr/sbin/service diffservice *
%parsoid-test-admins ALL = (parsoid-rt) NOPASSWD: /home/parsoid-rt/update-code.sh
%parsoid-test-admins ALL = NOPASSWD: /bin/journalctl *