Page MenuHomePhabricator

Puppet failures on integration-agent-qemu hosts
Closed, DuplicatePublic

Description

These started showing up in our Prometheus-based monitoring system yesterday:

root@integration-agent-qemu-1001:~# sudo run-puppet-agent
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Retrieving locales
Info: Loading facts
Info: Caching catalog for integration-agent-qemu-1001.integration.eqiad.wmflabs
Info: Applying configuration version '(a7435e5afb) Cwhite - Revert "Revert "logstash: install logstash-plugins on logging logstash clusters""'
Notice: The LDAP client stack for this host is: classic/sudoldap
Notice: /Stage[main]/Profile::Ldap::Client::Labs/Notify[LDAP client stack]/message: defined 'message' as 'The LDAP client stack for this host is: classic/sudoldap'
Notice: /Stage[main]/Profile::Ci::Dockervolume/Labs_lvm::Volume[docker]/Exec[available-space-docker]/returns: Less than 1.5G is available for partitioning.
Error: '/usr/local/sbin/pv-free' returned 1 instead of one of [0]
Error: /Stage[main]/Profile::Ci::Dockervolume/Labs_lvm::Volume[docker]/Exec[available-space-docker]/returns: change from 'notrun' to ['0'] failed: '/usr/local/sbin/pv-free' returned 1 instead of one of [0] (corrective)
Notice: /Stage[main]/Profile::Ci::Dockervolume/Labs_lvm::Volume[docker]/Exec[create-vd-docker]: Dependency Exec[available-space-docker] has failures: true
Warning: /Stage[main]/Profile::Ci::Dockervolume/Labs_lvm::Volume[docker]/Exec[create-vd-docker]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Dockervolume/Labs_lvm::Volume[docker]/Exec[create-mountpoint-docker]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Dockervolume/Labs_lvm::Volume[docker]/Mount[/var/lib/docker]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Dockervolume/Labs_lvm::Volume[docker]/File[/var/lib/docker]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Dockervolume/Labs_lvm::Volume[docker]/Labs_lvm::Extend[/var/lib/docker]/Exec[extend-vd-/var/lib/docker]: Skipping because of failed dependencies
Warning: /Stage[main]/Docker/Package[docker-ce]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Docker/Exec[jenkins user docker membership]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Labs::Lvm::Srv/Labs_lvm::Volume[second-local-disk]/Exec[available-space-second-local-disk]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Labs::Lvm::Srv/Labs_lvm::Volume[second-local-disk]/Exec[create-vd-second-local-disk]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Labs::Lvm::Srv/Labs_lvm::Volume[second-local-disk]/Exec[create-mountpoint-second-local-disk]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Labs::Lvm::Srv/Labs_lvm::Volume[second-local-disk]/Mount[/srv]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Labs::Lvm::Srv/Labs_lvm::Volume[second-local-disk]/File[/srv]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Slave::Labs::Common/File[/srv/home]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Slave::Labs::Common/File[/srv/jenkins]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Slave::Labs::Common/File[/srv/jenkins-workspace]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Slave::Labs::Common/File[/srv/jenkins/cache]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Slave::Labs::Common/File[/srv/jenkins/workspace]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Slave::Labs::Common/File[/srv/home/jenkins-deploy]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/File[/srv/git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/File[/srv/git/operations]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/File[/srv/git/mediawiki]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/File[/srv/git/mediawiki/extensions]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/File[/srv/git/mediawiki/skins]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Slave::Labs::Common/Git::Userconfig[.gitconfig for jenkins-deploy user]/File[/srv/home/jenkins-deploy/.gitconfig]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[operations/puppet]/File[/srv/git/operations/puppet.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[operations/puppet]/Exec[git_clone_operations/puppet]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Cron[operations/puppet git cron]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/core]/File[/srv/git/mediawiki/core.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/core]/Exec[git_clone_mediawiki/core]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Cron[mediawiki/core git cron]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/vendor]/File[/srv/git/mediawiki/vendor.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/vendor]/Exec[git_clone_mediawiki/vendor]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/AbuseFilter]/File[/srv/git/mediawiki/extensions/AbuseFilter.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/AbuseFilter]/Exec[git_clone_mediawiki/extensions/AbuseFilter]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Cite]/File[/srv/git/mediawiki/extensions/Cite.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Cite]/Exec[git_clone_mediawiki/extensions/Cite]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/cldr]/File[/srv/git/mediawiki/extensions/cldr.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/cldr]/Exec[git_clone_mediawiki/extensions/cldr]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Echo]/File[/srv/git/mediawiki/extensions/Echo.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Echo]/Exec[git_clone_mediawiki/extensions/Echo]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/EventLogging]/File[/srv/git/mediawiki/extensions/EventLogging.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/EventLogging]/Exec[git_clone_mediawiki/extensions/EventLogging]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/MobileFrontend]/File[/srv/git/mediawiki/extensions/MobileFrontend.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/MobileFrontend]/Exec[git_clone_mediawiki/extensions/MobileFrontend]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Scribunto]/File[/srv/git/mediawiki/extensions/Scribunto.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Scribunto]/Exec[git_clone_mediawiki/extensions/Scribunto]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/TemplateData]/File[/srv/git/mediawiki/extensions/TemplateData.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/TemplateData]/Exec[git_clone_mediawiki/extensions/TemplateData]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Translate]/File[/srv/git/mediawiki/extensions/Translate.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Translate]/Exec[git_clone_mediawiki/extensions/Translate]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/VisualEditor]/File[/srv/git/mediawiki/extensions/VisualEditor.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/VisualEditor]/Exec[git_clone_mediawiki/extensions/VisualEditor]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Wikibase]/File[/srv/git/mediawiki/extensions/Wikibase.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/extensions/Wikibase]/Exec[git_clone_mediawiki/extensions/Wikibase]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/skins/MinervaNeue]/File[/srv/git/mediawiki/skins/MinervaNeue.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/skins/MinervaNeue]/Exec[git_clone_mediawiki/skins/MinervaNeue]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/skins/Vector]/File[/srv/git/mediawiki/skins/Vector.git]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Ci::Gitcache/Git::Clone[mediawiki/skins/Vector]/Exec[git_clone_mediawiki/skins/Vector]: Skipping because of failed dependencies
Warning: /Stage[main]/Profile::Labs::Lvm::Srv/Labs_lvm::Volume[second-local-disk]/Labs_lvm::Extend[/srv]/Exec[extend-vd-/srv]: Skipping because of failed dependencies
Info: Stage[main]: Unscheduling all events on Stage[main]
Notice: Applied catalog in 6.35 seconds