Page MenuHomePhabricator

cron-spam to root@: lsof stderr generates large emails on boron from wmf-auto-restart
Closed, ResolvedPublic

Description

Since yesterday we're getting large emails to root@ from boron for the wmf-auto-restart crontabs due lsof printing to stderr lsof: no pwd entry for UID 1234 thousands of times.
UID 1234 is the pbuilder ID and there is a build of the JDK packages that takes long time.

Given that the script is currently not checking stdout from lsof, as a quick bandaid I'll add an ignore to stderr when running lsof, and then we can add a better solution later.

Event Timeline

Volans triaged this task as High priority.May 30 2019, 11:09 AM
Volans added a subscriber: jbond.

Change 513270 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] base: ignore lsof stderr in wmf-auto-restart

https://gerrit.wikimedia.org/r/513270

Change 513270 merged by Volans:
[operations/puppet@production] base: ignore lsof stderr in wmf-auto-restart

https://gerrit.wikimedia.org/r/513270

Volans lowered the priority of this task from High to Medium.May 30 2019, 11:18 AM
Volans removed a project: Patch-For-Review.

Bandaid applied, no output from running wmf-auto-restart on boron. It should be good for now. Leaving open for a better long-term solution.

Marostegui assigned this task to Volans.
Marostegui subscribed.

boron is gone - marking this as resolved