Page MenuHomePhabricator

tools cluster: packages have conffile prompts and needs to be upgraded manually
Closed, ResolvedPublic

Description

When working in T177920 unattended-upgrades detected that the packages puppet-common and nginx-common can't be upgraded because they have a conffile prompt.

Options are:

  • forget about these package, do nothing --> the packages stay outdated
  • find a solution and let the packages upgrade automagically, i.e. Which conffile? why?
  • force upgrade by hand in all the required nodes

Event Timeline

aborrero created this task.Nov 17 2017, 5:12 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 17 2017, 5:12 PM
aborrero renamed this task from tools cluster: package 'puppet-common' has conffile prompt and needs to be upgraded manually to tools cluster: packages have conffile prompts and needs to be upgraded manually.Nov 17 2017, 6:18 PM
aborrero updated the task description. (Show Details)
aborrero added a subscriber: chasemp.

Today I found a lot of emails from servers in the tools cluster with this content:

/etc/cron.daily/apt:
Package 'puppet-common' has conffile prompt and needs to be upgraded manually

We would rather address this issue to prevent this flooding in the future.

In T175885#3772306 @bd808 mentioned the issue with the nginx package was a simple missing config expected by dpkg but deleted by puppet.

aborrero updated the task description. (Show Details)Nov 20 2017, 12:10 PM

So, we can forget about nginx-common which has been manually upgraded by @bd808 the other day.

The problem with puppet-common is this (debug output):

[...]
found pkg: puppet-common
conffile line: '/etc/ldap/schema/puppet.schema ceb38d9f5b326f16e6c7786de590f9e0'
current md5: ceb38d9f5b326f16e6c7786de590f9e0
conffile line: '/etc/logcheck/ignore.d.server/puppet-common 9ab890e43ff4a02da8ec4ebe9651526b'
current md5: 9ab890e43ff4a02da8ec4ebe9651526b
conffile line: '/etc/puppet/etckeeper-commit-post 3783e0c108d846b3f2d2c3e40f567814'
current md5: 3783e0c108d846b3f2d2c3e40f567814
conffile line: '/etc/puppet/puppet.conf 52a58ea3d8f046d4b5e607a0eb5ece13'
current md5: 59908151982d861b3b27d530b7dfa42d
pkg_md5sum: a75fcfe47884f20dd4d81a4c0bc12d74
Package 'puppet-common' has conffile prompt and needs to be upgraded manually
Package 'puppet-common' has conffile prompt and needs to be upgraded manually
blacklist: ['puppet-common']
[...]

The file /etc/puppet/puppet.conf is changed (contains our config) and this summons the prompt. This check is directly done by the unattended-ugprade script.
And, according to the source code, there is no way to workaround this behavior, i.e, we can't say 'ok, upgrade this package anyway and preserve old conf'.

So I guess the easiest way to address this is to force a manual upgrade and choose to preserve current config file. This could be done with clush, right?

BTW, both puppet-common and puppet are pending an upgrade, the puppet upgraded being blocked because of the puppet-common blockade.

aborrero@tools-exec-1403:~$ apt-cache policy puppet
puppet:
  Installed: 3.4.3-1ubuntu1.2
  Candidate: 3.8.5-2~bpo8trusty+2
  Version table:
     3.8.5-2~bpo8trusty+2 0
       1001 http://apt.wikimedia.org/wikimedia/ trusty-wikimedia/main amd64 Packages
 *** 3.4.3-1ubuntu1.2 0
        500 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
        100 /var/lib/dpkg/status
     3.4.3-1 0
        500 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages
aborrero@tools-exec-1403:~$ apt-cache policy puppet-common
puppet-common:
  Installed: 3.4.3-1ubuntu1.2
  Candidate: 3.8.5-2~bpo8trusty+2
  Version table:
     3.8.5-2~bpo8trusty+2 0
       1001 http://apt.wikimedia.org/wikimedia/ trusty-wikimedia/main amd64 Packages
 *** 3.4.3-1ubuntu1.2 0
        500 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages
        500 http://security.ubuntu.com/ubuntu/ trusty-security/main amd64 Packages
        100 /var/lib/dpkg/status
     3.4.3-1 0
        500 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty/main amd64 Packages

I was wrong, we can indeed force to use old config by deploying a new file /etc/apt/apt.conf.d/00dpkg-conf:

Dpkg::Options {
   "--force-confold";
}

And then:

[...]
pkgs that look like they should be upgraded: puppet
puppet-common
Fetched 0 B in 0s (0 B/s)                                                       
fetch.run() result: 0
dpkg is configured not to cause conffile prompts
InstCount=2 DelCount=0 BrokenCount=0
Option --dry-run given, *not* performing real actions
Packages that will be upgraded: puppet puppet-common
Writing dpkg log to '/var/log/unattended-upgrades/unattended-upgrades-dpkg_2017-11-20_12:37:29.961126.log'
All upgrades installed

Will prepare a patch for evaluation.

Change 392421 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] apt: add class apt::dpkg-confold and include it from apt::unattendedupgrades

https://gerrit.wikimedia.org/r/392421

aborrero claimed this task.Nov 20 2017, 6:24 PM

Change 392421 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] apt: add --force-confold/--force-confdef dpkg option to apt calls

https://gerrit.wikimedia.org/r/392421

All seems fine now.

aborrero closed this task as Resolved.Nov 30 2017, 6:31 PM