cronspam from prometheus-directory-size (on labstore1004)
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	Dzahn
	Nov 20 2020, 4:59 PM

Description

There is a prometheus-directory-size exporter at:

profile::prometheus::node_directorysize
modules/profile/templates/prometheus/node-directory-size.erb

On labstore1004 it creates daily cronspam due to:

find: ‘/srv/tools/shared/tools/project/paws/userhomes/*’: No such file or directory

Not sure if the proper fix is to ignore that path above or to generally make sure it does not alert when a directory does not exist but probably BOTH.

Would be nice to stop the daily mails one way or the other.

also see: T211861

Details

	Project	Branch	Lines +/-	Subject
	operations/puppet	production	+0 -1	nfs monitoring: remove the old paws path

Customize query in gerrit

Related Objects

Mentioned Here: T211861: prometheus-directory-size: "du: cannot access ‘No such file or directory'"

Event Timeline

Dzahn created this task.Nov 20 2020, 4:59 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 20 2020, 4:59 PM

Dzahn renamed this task from cronspam from prometheus-directory-size to cronspam from prometheus-directory-size (on labstore1004).Nov 20 2020, 5:02 PM

Dzahn added projects: cloud-services-team, observability.

Dzahn updated the task description. (Show Details)

Restricted Application edited projects, added cloud-services-team (Kanban); removed cloud-services-team. · View Herald TranscriptNov 20 2020, 5:02 PM

Dzahn triaged this task as Low priority.Nov 20 2020, 5:04 PM

That actually makes sense since that path was removed, and I bet it was coded in somewhere.

Change 642558 had a related patch set uploaded (by Bstorm; owner: Bstorm):
[operations/puppet@production] nfs monitoring: remove the old paws path

https://gerrit.wikimedia.org/r/642558

gerritbot added a project: Patch-For-Review.Nov 20 2020, 10:09 PM

Change 642558 merged by Bstorm:
[operations/puppet@production] nfs monitoring: remove the old paws path

https://gerrit.wikimedia.org/r/642558

Thanks a lot for the quick response!

Maintenance_bot removed a project: Patch-For-Review.Nov 20 2020, 11:10 PM

lmata moved this task from Backlog to Radar on the SRE board.Nov 23 2020, 4:19 PM

lmata moved this task from Inbox to Radar on the observability board.

I did not get the mail today anymore. Thank you Brooke!

cronspam from prometheus-directory-size (on labstore1004)Closed, ResolvedPublicActions

Description

Details

Related Objects

Event Timeline

cronspam from prometheus-directory-size (on labstore1004)
Closed, ResolvedPublic
Actions