I'm pretty well convinced that there are no significant differences in catalogs, so getting clients to work properly should be fine.
Since clients aren't backwards-compatible and our puppetmasters are also clients, this needs to proceed top to bottom:
then
then
then
if you want you can use puppet-paladox3.git.eqiad.wmflabs.org as a test for puppet 4.8 :). Though i just need to know when you would do it :).
Change 394353 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] Move labtestpuppetmaster2001 to puppet 4
Change 394353 merged by Andrew Bogott:
[operations/puppet@production] Move labtestpuppetmaster2001 to puppet 4
Change 397575 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] puppetmaster: include @extra_auth_rules in v4 auth.conf
Change 397575 merged by Andrew Bogott:
[operations/puppet@production] puppetmaster: include @extra_auth_rules in v4 auth.conf
Change 397579 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] WMCS puppetmaster: modify auth.conf extra rules to account for url changes
Change 397579 merged by Andrew Bogott:
[operations/puppet@production] WMCS puppetmaster: modify auth.conf extra rules to account for url changes
Change 397584 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] profile::puppetmaster::backend: fix extra_auth_rules for backend masters
Change 397584 merged by Andrew Bogott:
[operations/puppet@production] profile::puppetmaster::backend: fix extra_auth_rules for backend masters
Change 397711 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] WMCS: set puppet_major_version to 4
Change 397711 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] WMCS: set puppet_major_version to 4
Change 397821 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] labs-bootstrapvz-jessie: include puppet pinning in base images
Change 397821 merged by Andrew Bogott:
[operations/puppet@production] labs-bootstrapvz-jessie: include puppet pinning in base images
Change 397850 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] bootstrap-vz: include debian backports in sources.list
Change 397850 merged by Andrew Bogott:
[operations/puppet@production] bootstrap-vz: include debian backports in sources.list
Change 397966 had a related patch set uploaded (by Andrew Bogott; owner: Andrew Bogott):
[operations/puppet@production] bootstrapvz: allow default (v4) puppet packages on stretch base images.
Change 397966 merged by Andrew Bogott:
[operations/puppet@production] bootstrapvz: allow default (v4) puppet packages on stretch base images.
Change 397711 merged by Andrew Bogott:
[operations/puppet@production] WMCS: set puppet_major_version to 4
All masters are upgraded, and Debian base images now include puppet 4.
All that remains is upgrading agents, like so:
$ sudo cumin "*" "disable-puppet 'puppet upgrades - andrew'" $ sudo cumin "*" "apt-get update" sudo cumin "*" 'DEBIAN_FRONTEND=noninteractive apt-get -q -y --assume-no -o DPkg::Options::="--force-confdef" -o DPkg::Options::="--force-confold" install puppet' $ sudo cumin "*" "enable-puppet 'puppet upgrades - andrew'"
I'm not doing that right now because currently there are no puppet 4 candidate packages for trusty. @herron is investigating that now; soon we'll decide whether to backport a package or just leave Trusty with v3 agents.
I'm trying to accomplish this following the workflow we are developing in T181647
The upgrade script in that workflow crash if new package are being installed due to dependencies:
aborrero@tools-webgrid-generic-1402:~$ sudo apt-upgrade trusty-wikimedia -sv
Updating cache ...
puppet-common 3.8.5-2~bpo8trusty+2 --> 4.8.2-5~trusty1
Traceback (most recent call last):
File "/usr/local/sbin/apt-upgrade", line 127, in <module>
main()
File "/usr/local/sbin/apt-upgrade", line 123, in main
run(args.source, args.s, args.v)
File "/usr/local/sbin/apt-upgrade", line 101, in run
print_verbose_pkg(verbose, pkg)
File "/usr/local/sbin/apt-upgrade", line 34, in print_verbose_pkg
orig = pkg.installed.version
AttributeError: 'NoneType' object has no attribute 'version'It should instead show something like this:
aborrero@tools-webgrid-generic-1402:~$ sudo apt-upgrade trusty-wikimedia -sv Updating cache ... puppet-common 3.8.5-2~bpo8trusty+2 --> 4.8.2-5~trusty1 python-ecdsa unknown --> 0.10-2 puppet 3.8.5-2~bpo8trusty+2 --> 4.8.2-5~trusty1 ruby-deep-merge unknown --> 1.1.1-1~trusty1 python-paramiko 1.10.1-1git1build1 --> 1.15.2-1 Simulate, not performing changes
I'm patching this first before going on.
BTW my notes are here: https://etherpad.wikimedia.org/p/arturo-toolforge-upgrades
Change 404963 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] apt: apt-upgrades: dont fail if new packages are being installed
Change 404963 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] apt: apt-upgrades: dont fail if new packages are being installed
Mentioned in SAL (#wikimedia-cloud) [2018-01-18T11:42:00Z] <arturo> T178717 aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent --test'
Mentioned in SAL (#wikimedia-cloud) [2018-01-18T12:11:06Z] <arturo> T178717 aborrero@tools-webgrid-generic-1402:~$ sudo apt-upgrade trusty-wikimedia -v
Mentioned in SAL (#wikimedia-cloud) [2018-01-18T12:24:17Z] <arturo> T178717 aborrero@tools-exec-1401:~$ sudo apt-upgrade trusty-wikimedia -v
Mentioned in SAL (#wikimedia-cloud) [2018-01-18T13:52:31Z] <arturo> T178717 aborrero@tools-clushmaster-01:~$ clush -f 1 -w @all 'sudo facter | grep lsbdistcodename | grep trusty && sudo apt-upgrade trusty-wikimedia -v'
Mentioned in SAL (#wikimedia-cloud) [2018-01-18T15:42:32Z] <arturo> T178717 aborrero@tools-clushmaster-01:~$ clush -w @all 'sudo puppet agent -t -v'