Alert in need of triage: PuppetConstantChange (instance wikikube-worker-exp1001:9100)
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	LSobanski
	Aug 26 2025, 8:20 AM

Description

The alert PuppetConstantChange has started firing 1 month ago.

Labels

alertname=PuppetConstantChange
cluster=kubernetes
instance=wikikube-worker-exp1001:9100
job=node
prometheus=ops
severity=warning
site=eqiad
source=prometheus
team=serviceops

Annotations

Name	Content
dashboard	https://grafana.wikimedia.org/d/yOxVDGvWk/puppet
description	Puppet performing a change on every puppet run wikikube-worker-exp1001:9100
runbook	https://puppetboard.wikimedia.org/nodes?status=changed
summary	Puppet performing a change on every puppet run on wikikube-worker-exp1001:9100

Links

Triage metadata. Do not delete.

fingerprint=a18cc8833c59f87f

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	mw_experimental: More permission fixes	operations/puppet	production	+2 -0

Customize query in gerrit

Related Objects

Mentioned In: T402888: Alert in need of triage: PuppetConstantChange (instance wikikube-worker-exp2001:9100)
Mentioned Here: T276994: Provide an mwdebug functionality on kubernetes (mw-experimental)

Event Timeline

LSobanski created this task.Aug 26 2025, 8:20 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 26 2025, 8:20 AM

@jijiki could you take a look? I think this is alerting since you set the instances up in T276994: Provide an mwdebug functionality on kubernetes (mw-experimental)

I pushed https://gerrit.wikimedia.org/r/c/operations/puppet/+/1181673?usp=dashboard yesterday but apparently that wasn't enough. I'll take another look as @jijiki is OOO

Clement_Goubert triaged this task as Medium priority.Aug 26 2025, 9:35 AM

Change #1182101 had a related patch set uploaded (by Clément Goubert; author: Clément Goubert):

[operations/puppet@production] mw_experimental: More permission fixes

https://gerrit.wikimedia.org/r/1182101

gerritbot added a project: Patch-For-Review.Aug 26 2025, 9:49 AM

Change #1182101 merged by Clément Goubert:

[operations/puppet@production] mw_experimental: More permission fixes

https://gerrit.wikimedia.org/r/1182101

Clement_Goubert mentioned this in T402888: Alert in need of triage: PuppetConstantChange (instance wikikube-worker-exp2001:9100).Aug 26 2025, 10:03 AM

Maintenance_bot removed a project: Patch-For-Review.Aug 26 2025, 10:31 AM

Permission fix looks like it resolved the alert.

Clement_Goubert merged a task: T402888: Alert in need of triage: PuppetConstantChange (instance wikikube-worker-exp2001:9100).Aug 26 2025, 11:49 AM

Alert in need of triage: PuppetConstantChange (instance wikikube-worker-exp1001:9100)Closed, ResolvedPublicActions