Page MenuHomePhabricator

Prevents labs/private to disappear from integration and deployment-prep puppet master
Closed, ResolvedPublic


Some patch in puppet caused labs/private to be set as absent:,unified

Luckily deployment-puppetmaster04.deployment-prep.eqiad.wmflabs and integration-puppetmaster-02.integration.eqiad.wmflabs had merge conflicts preventing them from automagically applying the faulty puppet.git patch

Event Timeline

On both, I have made a copy to /var/lib/git/labs/private-back

hashar triaged this task as Unbreak Now! priority.Jun 4 2020, 1:43 PM

I have:

  • disabled puppet agent on both puppet master
  • rebased /var/lib/git/operations/puppet on both puppet master.

Confirmed the faulty commit got reverted ;)

Enabled puppet again and ran it again.

The one on beta fails because of a missing etcd::autogen_pwd_seed. I have made a dummy
puppet patch in hieradata/cloud/eqiad1/deployment-prep/common.yaml

hashar claimed this task.

Looks good to me now. We have been lucky to have both puppet master blocked on a merge conflict!