Page MenuHomePhabricator
Create Task
Maniphest T140976

labmon1001 renderer requires authentication breaking labs Shinken probes
Closed, ResolvedPublic
Actions

Description

Shinken on labs has monitored probe based on Graphite metrics which probe labmon1001.eqiad.wmnet . Since July 20th ~ 14:10 UTC the /render URL yields a 401 Not authorized which causes all monitoring probe to be in UNKNOWN state.

That breaks monitoring for Puppet freshness, disk free and others.

Example:

UNKNOWN: Got status 401 from the graphite server at http://labmon1001.eqiad.wmnet/render?format=json&from=-10min&target=cvn.cvn-apache8.diskspace.%2A.byte_percentfree

http://shinken.wmflabs.org/service/cvn-apache8/Free%20space%20-%20all%20mounts#impacts

Most probably related to work on T120295: Setup "official labs grafana" instance

Details

SubjectRepoBranchLines +/-
shinken: Use new labs graphite URLoperations/puppetproduction+3 -3
Customize query in gerrit

Related Objects

Event Timeline

hashar created this task.Jul 21 2016, 8:48 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 21 2016, 8:48 AM
hashar triaged this task as High priority.Jul 21 2016, 8:48 AM
hashar mentioned this in T120295: Setup "official labs grafana" instance.
gerritbot subscribed.Jul 21 2016, 1:17 PM

Change 300265 had a related patch set uploaded (by Yuvipanda):
shinken: Use new labs graphite URL

https://gerrit.wikimedia.org/r/300265

gerritbot added a project: Patch-For-Review.Jul 21 2016, 1:17 PM
yuvipanda mentioned this in rOPUP58105dec8308: shinken: Use new labs graphite URL.Jul 21 2016, 1:19 PM
yuvipanda claimed this task.Jul 21 2016, 1:22 PM
gerritbot added a comment.Jul 21 2016, 2:00 PM

Change 300265 merged by Yuvipanda:
shinken: Use new labs graphite URL

https://gerrit.wikimedia.org/r/300265

yuvipanda mentioned this in rOPUPa08e23c4bc24: shinken: Use new labs graphite URL.Jul 21 2016, 2:04 PM
yuvipanda closed this task as Resolved.Jul 21 2016, 2:35 PM

Thanks for reporting :D is sorted out now.

hashar added a comment.Jul 22 2016, 8:25 AM

Awesome thank you @yuvipanda

• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:41 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL