Icinga showed warnings that there are puppet errors on contint1001 and contint2001 since about 2days and 8 hours from writing this.
They are related to deployment-charts, specifically helmfiles for push-notifications:
WARNING: Puppet has 3 failures. Last run 6 seconds ago with 3 failures. Failed resources (up to 3 shown): File[/srv/deployment-charts/helmfile.d/services/staging/push-notifications],File[/srv/deployment-charts/helmfile.d/services/eqiad/push-notifications],File[/srv/deployment-charts/helmfile.d/services/codfw/push-notifications]
The reason for that is Error: Could not find user mwdeploy
Error: /Stage[main]/Profile::Kubernetes::Deployment_server::Helmfile/File[/srv/deployment-charts/helmfile.d/services/eqiad/push-notifications]/owner: change from 'root' to 'mwdeploy' failed: Could not find user mwdeploy
See the puppet run on the servers for more details.
Should the user mwdeploy be added on contint servers or is it a different fix?