Page MenuHomePhabricator

Allow *-admin groups to see systemd logs for their units
Closed, ResolvedPublic

Description

With the switch to Jessie, services using service::node have started using SystemD, which keeps its own log (containing its start/stop actions and the service's stderr). Alas, these logs are not visible to people in the respective *-admin groups:

mobrovac@scb1001:~$ sudo journalctl -u changeprop

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

    #1) Respect the privacy of others.
    #2) Think before you type.
    #3) With great power comes great responsibility.

[sudo] password for mobrovac:

This request extends to the following groups: citoid-admin, changeprop-admin, cxserver-admin, mathoid-admin, maps-admins, graphoid-admin, mobileapps-admin, parsoid-admin (the switch to Jessie will happen soon enough that it makes sense in this request), apertium-admins (idem), ocg-render-admins and sc-admins.

Event Timeline

Restricted Application added a project: Operations. · View Herald TranscriptJun 15 2016, 12:23 PM
Restricted Application added subscribers: Zppix, Aklapper. · View Herald Transcript

I looked through some of the other -admin groups. Are any of these also relevant?

gerrit-admin
ocg-render-admins
analytics-admins
chromium-admin
zotero-admin
apertium-admins
RobH changed the task status from Open to Stalled.Jun 15 2016, 4:00 PM
RobH claimed this task.
RobH added a subscriber: RobH.

The scope of this is fairly large, but the request/change itself is a single permission (ability to read that log file) for all the admin groups.

I'm not sure if it should cover just what @mobrovac lists, or also include @Krenair's additions. It seems this should likely be added to all admin groups for services that will run on jessie. (It is likely easier to just add to all admin groups if doing so wouldn't break things.)

Since this is a proposed change to the operation of admin/sudo groups, this will require ops meeting review. I'm on clinic duty this and next week, so I'll claim this task (and stall it) until the meeting next Monday.

mobrovac updated the task description. (Show Details)Jun 15 2016, 5:17 PM

I added apertium and OCG to the list. Zotero is not relevant as we'll never switch it to Jessie. For the rest, I really don't know if that would be useful.

mobrovac updated the task description. (Show Details)Jun 15 2016, 5:18 PM
RobH added a comment.Jun 15 2016, 6:56 PM

@mobrovac: Thank you for the feedback! We'll go with your appended list unless others can point out where the other admin services will migrate to jessie and need it.

I'll be certain to mention the other potential groups when relaying this into the ops meeting.

RobH reassigned this task from RobH to Joe.Jun 20 2016, 5:53 PM
RobH added a subscriber: Joe.

This request has been denied in the operations meeting, in its currently proposed use of sudo rights to view the syslog.

Rather, its been suggested we parse syslog and output the affected/required service lines into their own file, rather than granting sudo.

@Joe will provide feedback on this task (so I am assigning it to him), however this week is immediately preceding Wikimania, and response times will be delayed accordingly.

fgiunchedi triaged this task as Medium priority.Jul 11 2016, 10:48 AM
Joe changed the task status from Stalled to Open.Jul 13 2016, 12:34 PM
Joe added a comment.Jul 13 2016, 12:37 PM

So my point is basically that for every service (at least the standard ones that we define via service::node, we can:

  1. ensure that journald forwards the interesting logs to syslog
  2. Make syslog write to a logfile with mode 0540 owned by the appropriate -admins group
  3. Prepare the corresponding logrotate rules

This should be relatively easy, clean, and avoid adding sudo rules we don't really need.

Wouldn't it be simpler just to fwd the logs to syslog and allow admins to read /var/log/syslog (which they currently can't) ?

Change 299000 had a related patch set uploaded (by Mobrovac):
service::node: Output std out/err to a file

https://gerrit.wikimedia.org/r/299000

Change 299000 merged by Giuseppe Lavagetto:
service::node: Output std out/err to a file

https://gerrit.wikimedia.org/r/299000

Joe closed this task as Resolved.Jul 27 2016, 11:23 AM
mobrovac reopened this task as Open.Aug 8 2016, 5:46 PM

Hm, this patch doesn't actually seem to work. All of the services' syslog.log files are empty, but there definitely are entries for them. Perhaps an explicit mention of syslog in the SystemD service files is needed after all?

Joe added a comment.Aug 10 2016, 12:49 PM

@mobrovac I already know what to do in order to fix it. At the moment all logs are registered as "firejail".

Duh! Good point. So obvious ...

Change 304028 had a related patch set uploaded (by Giuseppe Lavagetto):
service::node: expliticly set syslog identifier

https://gerrit.wikimedia.org/r/304028

Change 304028 merged by Giuseppe Lavagetto:
service::node: expliticly set syslog identifier

https://gerrit.wikimedia.org/r/304028

mobrovac closed this task as Resolved.Aug 10 2016, 2:02 PM

Mentioned in SAL [2016-08-10T15:50:32Z] <_joe_> restarting parsoid for T137878