Page MenuHomePhabricator
Create Task
Maniphest T256467

Make atsmtail-backend.service depend on fifo-log-demux
Closed, ResolvedPublic
Actions

Description

@Volans noticed in T256449 that atsmtail-backend spams the logs if it cannot connect to the notpurge socket. We should make atsmtail-backend.service depend on fifo-log-demux.

Details

SubjectRepoBranchLines +/-
atsmtail: make systemd unit depend on fifo-log-demux@operations/puppetproduction+5 -2
Systemd::Servicename: add '@' to Patternoperations/puppetproduction+1 -1
Customize query in gerrit

Related Objects

Event Timeline

• ema created this task.Jun 26 2020, 10:29 AM
Restricted Application added a project: SRE. · View Herald TranscriptJun 26 2020, 10:29 AM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
• ema triaged this task as Low priority.Jun 26 2020, 10:29 AM
• ema moved this task from Backlog to Caching on the Traffic board.
Vgutierrez subscribed.Jun 29 2020, 2:50 PM
• ema mentioned this in T268883: fifo-log-tailer: gracefully handle missing unix socket.Nov 27 2020, 10:50 AM
gerritbot added a comment.Nov 27 2020, 11:30 AM

Change 643922 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] atsmtail: make systemd unit depend on fifo-log-demux@

https://gerrit.wikimedia.org/r/643922

gerritbot added a project: Patch-For-Review.Nov 27 2020, 11:30 AM
gerritbot added a comment.Nov 27 2020, 1:59 PM

Change 643934 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] Systemd::Servicename: add '@' to Pattern

https://gerrit.wikimedia.org/r/643934

gerritbot added a comment.Nov 30 2020, 9:28 AM

Change 643934 merged by Ema:
[operations/puppet@production] Systemd::Servicename: add '@' to Pattern

https://gerrit.wikimedia.org/r/643934

gerritbot added a comment.Nov 30 2020, 9:35 AM

Change 643922 merged by Ema:
[operations/puppet@production] atsmtail: make systemd unit depend on fifo-log-demux@

https://gerrit.wikimedia.org/r/643922

Maintenance_bot removed a project: Patch-For-Review.Nov 30 2020, 10:11 AM
Stashbot added a comment.Nov 30 2020, 10:18 AM

Mentioned in SAL (#wikimedia-operations) [2020-11-30T10:18:09Z] <ema> cp4031: reboot to test atsmtail/fifo-log-demux service dependencies -- https://gerrit.wikimedia.org/r/c/operations/puppet/+/643922 T256467

• ema closed this task as Resolved.Nov 30 2020, 10:33 AM
• ema claimed this task.

Unit ordering at boot time is now correct:

root@cp4031:~# journalctl -u trafficserver-tls.service | grep Started
Nov 30 10:22:11 cp4031 systemd[1]: Started trafficserver-tls.service.
root@cp4031:~# journalctl -u fifo-log-demux@analytics.service | grep Started
Nov 30 10:22:11 cp4031 systemd[1]: Started FIFO log demultiplexer (instance analytics).
root@cp4031:~# journalctl -u atsmtail@tls.service | grep Started
Nov 30 10:22:12 cp4031 systemd[1]: Started mtail instance for Apache Traffic Server tls.

A few Unable to read from socket errors (T256449) are still shown in atsmtail's journal due to the fact that it takes a second for the socket to show up and T268883, but as far as systemd service dependencies are concerned this is now done.

• ema mentioned this in T264398: 8-10% response start regression (Varnish 5.1.3-1wm15 -> 6.0.6-1wm1).Nov 30 2020, 2:14 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL