Page MenuHomePhabricator

Can't ssh into new instance created at toolsbeta
Closed, ResolvedPublic

Description

With my normal cloud key:
(jessie)

$ ssh toolsbeta-flannel-etcd-01.toolsbeta.eqiad.wmflabs
Permission denied (publickey).

(trusty)

$ ssh toolsbeta-bastion-01.toolsbeta.eqiad.wmflabs
zhuyifei1999@toolsbeta-bastion-01.toolsbeta.eqiad.wmflabs's password:

They seemed to have failed first puppet run, some got:

  • Error: Could not retrieve catalog; skipping run (toolsbeta-flannel-etcd-01)
  • Error: Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Error while evaluating a Function Call, Could not find data item active_redis in any Hiera data file and no default supplied at /etc/puppet/modules/toollabs/manifests/init.pp:159:21 on node toolsbeta-bastion-01.toolsbeta.eqiad.wmflabs (toolsbeta-bastion-01.toolsbeta.eqiad.wmflabs)
  • Error: Could not set 'present' on ensure: invalid byte sequence in US-ASCII at /etc/puppet/modules/standard/manifests/init.pp:42 (toolsbeta-grid-master)

Event Timeline

zhuyifei1999 created this task.

Could not find data item active_redis in any Hiera data file and no default supplied

My fault, removed too many hiera keys

toolsbeta-proxy-01 was rebuilt and is now getting Could not retrieve catalog from remote server: Error 500 on SERVER: Server Error: Evaluation Error: Error while evaluating a Resource Statement, Evaluation Error: Operator '[]' is not applicable to an Undef Value. at /etc/puppet/modules/toollabs/manifests/kube2proxy.pp:37:21 on node toolsbeta-proxy-01.toolsbeta.eqiad.wmflabs

https://github.com/wikimedia/puppet/blob/production/modules/toollabs/manifests/kube2proxy.pp#L36: $users = hiera('k8s_infrastructure_users')
Anyone has any idea where k8s_infrastructure_users is from?

I'll look into whether the puppet issue is resolved when I get time for it, hopefully tomorrow.

Mentioned in SAL (#wikimedia-cloud) [2018-05-02T20:31:01Z] <zhuyifei1999_> Added k8s_infrastructure_users to project hiera on horizon T192618

zhuyifei1999 claimed this task.