Page MenuHomePhabricator
Create Task
Maniphest T177920

unattended-upgrades not upgrading "-wikimedia" packages automatically in wmcs
Closed, ResolvedPublic
Actions

Description

Noticed this while investigating something else, I believe u-a in wmcs is supposed to upgrade all packages coming from e.g. jessie-wikimedia too. That doesn't seem to be the case on a couple of instances I spot checked, namely filippo-test-jessie3.eqiad.wmflabs and deployment-prometheus01.eqiad.wmflabs

root@deployment-prometheus01:~# apt-cache policy diamond
diamond:
  Installed: 3.5-6
  Candidate: 4.0.515-4~bpo8+2
  Version table:
     4.0.515-4~bpo8+2 0
       1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/backports amd64 Packages
 *** 3.5-6 0
        100 /var/lib/dpkg/status
root@filippo-test-jessie3:~# apt-cache policy hhvm
hhvm:
  Installed: 3.12.7+dfsg-1+wmf4
  Candidate: 3.18.5+dfsg-1+wmf1
  Version table:
     3.18.5+dfsg-1+wmf1 0
       1001 http://apt.wikimedia.org/wikimedia/ jessie-wikimedia/main amd64 Packages
 *** 3.12.7+dfsg-1+wmf4 0
        100 /var/lib/dpkg/status

Details

SubjectRepoBranchLines +/-
apt: unattended upgrades for wikimedia packages by defaultoperations/puppetproduction+9 -9
Customize query in gerrit

Related Objects

Event Timeline

fgiunchedi created this task.Oct 11 2017, 10:02 AM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 11 2017, 10:02 AM
Paladox subscribed.Oct 11 2017, 11:36 AM
bd808 added a project: Cloud-VPS.Oct 12 2017, 8:08 PM
aborrero subscribed.Nov 3 2017, 6:57 PM
aborrero claimed this task.Nov 3 2017, 7:08 PM
aborrero added a comment.Nov 3 2017, 7:30 PM

Probably we just need something like this patch in operations/puppet:

diff --git a/modules/profile/manifests/base/labs.pp b/modules/profile/manifests/base/labs.pp
index 23816b3f28..65c00d875f 100644
--- a/modules/profile/manifests/base/labs.pp
+++ b/modules/profile/manifests/base/labs.pp
@@ -23,6 +23,15 @@ class profile::base::labs {
         }
     }
 
+    apt::conf { 'unattended-upgrades-wikimedia':
+        priority => '51',
+        # Key with trailing '::' to append to potentially existing entry
+        key      => 'Unattended-Upgrade::Allowed-Origins::',
+        # lint:ignore:single_quote_string_with_variables
+        value    => 'Wikimedia:${distro_codename}-wikimedia',
+        # lint:endignore
+    }
+
     file { '/usr/local/sbin/notify_maintainers.py':
         ensure => present,
         owner  => 'root',
bd808 edited projects, added cloud-services-team (Kanban); removed cloud-services-team.Nov 3 2017, 7:36 PM
bd808 subscribed.

Related: T159254: Blacklist apache from unattended-upgrades on tools puppetmaster

bd808 moved this task from Inbox to Doing on the cloud-services-team (Kanban) board.Nov 3 2017, 7:36 PM
gerritbot subscribed.Nov 6 2017, 2:05 PM

Change 389480 had a related patch set uploaded (by Arturo Borrero Gonzalez; owner: Arturo Borrero Gonzalez):
[operations/puppet@production] base: labs: unattended upgrades for wikimedia packages

https://gerrit.wikimedia.org/r/389480

gerritbot added a project: Patch-For-Review.Nov 6 2017, 2:05 PM
aborrero added a comment.EditedNov 7 2017, 6:48 PM

New patchset with a different approach: rOPUP9e9c5e1c06c9

Trying puppet compiler for unexpected side changes in other hosts:
https://puppet-compiler.wmflabs.org/compiler02/8680/

hashar subscribed.Nov 10 2017, 10:25 AM

hhvm is not upgraded because the package has origin=Wikimedia and it is not taken in account by unattended-upgrade default configuration.

Ages ago, I had the same requirements: have HHVM to magically upgrade on CI. I went with https://gerrit.wikimedia.org/r/#/c/183019/ later fixed up with Filippo via https://gerrit.wikimedia.org/r/298568 . On the side, Unattended-Upgrade::Allowed-Origins is deprecated. Should be replaced with Unattended-Upgrade::Origins-Pattern:: which expect a slightly different value.

End result would be:

apt::conf { 'unattended-upgrades-wikimedia':
     priority => '51',
     # Key with trailing '::' to append to potentially existing entry
     key      => 'Unattended-Upgrade::Origins-Pattern::',
     value    => 'origin=Wikimedia,codename=${distro_codename}-wikimedia',
 }

To confirm one can then:

unattended-upgrade --debug --dry-run 2>&1|grep ^Allowed
aborrero added a comment.Nov 10 2017, 10:32 AM

Updated the patch: https://gerrit.wikimedia.org/r/389480

aborrero added a comment.Nov 10 2017, 10:46 AM

BTW the test that @hashar mentioned pass in my standalone puppet master:

aborrero@puppet-vm:~$ sudo unattended-upgrade --debug --dry-run 2>&1|grep ^Allowed
Allowed origins are: ['origin=Debian,codename=jessie,label=Debian-Security', 'origin=Wikimedia,codename=jessie-wikimedia']
aborrero mentioned this in T180254: unattended-upgrades: upgrade packages from -updates suites as well.Nov 10 2017, 4:27 PM
gerritbot added a comment.Nov 16 2017, 3:46 PM

Change 389480 merged by Arturo Borrero Gonzalez:
[operations/puppet@production] apt: unattended upgrades for wikimedia packages by default

https://gerrit.wikimedia.org/r/389480

aborrero added a subscriber: • chasemp.Nov 16 2017, 5:47 PM

@chasemp and me merged this patch and did a controlled deployment into the tools cluster.

Notes here: https://etherpad.wikimedia.org/p/389480

aborrero mentioned this in T175885: Toolforge's static webserver broken by Puppet changes and stale nginx packages.Nov 17 2017, 4:52 PM
aborrero mentioned this in T180809: tools cluster: pending linux kernel upgrades.Nov 17 2017, 5:02 PM
aborrero mentioned this in T180811: tools cluster: packages have conffile prompts and needs to be upgraded manually.Nov 17 2017, 5:12 PM
aborrero added a comment.Nov 23 2017, 6:03 PM

Just to don't forget, I created this script: P6365

• chasemp mentioned this in T181647: create 'attended' upgrade workflow for cloud with Toolforge as canonical case.Nov 30 2017, 6:15 PM
aborrero closed this task as Resolved.Jan 26 2018, 6:24 PM

This should be fixed now, by means of https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Attended_package_upgrades

bd808 moved this task from Doing to Done on the cloud-services-team (Kanban) board.Jan 31 2018, 12:50 AM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL