Page MenuHomePhabricator

Puppet errors on glampipe.glampipe.eqiad.wmflabs
Closed, ResolvedPublic

Description

Puppet has been failing on glampipe.glampipe.eqiad.wmflabs:

Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, Could not find data item profile::docker::storage::physical_volumes in any Hiera data file and no default supplied at /etc/puppet/modules/profile/manifests/docker/storage.pp:12:25 on node glampipe.glampipe.eqiad.wmflabs

If that's a one-off test server, please let us know so we can delete it.

Otherwise, could you take a few minutes to fix that? Thank you!

Event Timeline

GTirloni triaged this task as High priority.Nov 6 2018, 7:33 PM
GTirloni created this task.

Not test server, but i think that "profile::docker::storage::physical_volume" can be removed from the VM config if it will fix it?

$ sudo -i puppet agent -tv
Info: Using configured environment 'production'
Info: Retrieving pluginfacts
Info: Retrieving plugin
Info: Loading facts
Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, Failed to parse template mysql/apparmor.template.usr.sbin.mysqld.erb:
  Filepath: /etc/puppet/modules/mysql/templates/apparmor.template.usr.sbin.mysqld.erb
  Line: 27
  Detail: undefined local variable or method `config_file' for #<Puppet::Parser::TemplateWrapper:0x000000025b9328>
 at /etc/puppet/modules/mysql/manifests/server.pp:59:20 on node glampipe.glampipe.eqiad.wmflabs
Warning: Not using cache on failed catalog
Error: Could not retrieve catalog; skipping run

This is T128642: role::simplelamp fails to start mysql due to apparmor I think.

Puppet has apparently not run to completion on this instance for 39 days. The only role applied seems to be role::simplelamp per https://tools.wmflabs.org/openstack-browser/server/glampipe.glampipe.eqiad.wmflabs.

Dzahn added a subscriber: Dzahn.

I fixed this by creating "role::simplelamp2" which uses mariadb instead of mysql. This has been applied on glampipe instead of the old simplelamp role and puppet runs fine again.