Page MenuHomePhabricator
Create Task
Maniphest T120961

Nodepool snapshot refresh should run apt-get upgrade
Closed, ResolvedPublic
Actions

Description

The Nodepool base image is created using the puppet script /dib/puppet/ciiimage.pp. It reuses operations/puppet.git and install a bunch of packages.

Then that image is pushed to labs infrastructure and used to spawn an instance out of it. The script /nodepool/scripts/setup_node.sh runs as user debian and reapply the manifest. The snapshot are recreated once per day automatically around 14:00 UTC.

A problem is that the packages are usually marked with ensure => present and they end up being frozen to the version we had when build the reference image.

I guess the setup_node.sh script should run apt-get dist-upgrade --yes to catch up.

Details

SubjectRepoBranchLines +/-
nodepool: run apt-get upgrade while snapshotingintegration/configmaster+3 -0
Customize query in gerrit

Related Objects

Event Timeline

hashar created this task.Dec 9 2015, 4:25 PM
hashar raised the priority of this task from to Needs Triage.
hashar updated the task description. (Show Details)
hashar added projects: Continuous-Integration-Infrastructure, Continuous-Integration-Scaling.
hashar added subscribers: hashar, Joe.
Restricted Application added subscribers: StudiesWorld, Aklapper. · View Herald TranscriptDec 9 2015, 4:25 PM
hashar added a comment.EditedDec 9 2015, 4:30 PM

Example from an instance:

$ apt-get -s upgrade
NOTE: This is only a simulation!
      apt-get needs root privileges for real execution.
      Keep also in mind that locking is deactivated,
      so don't depend on the relevance to the real current situation!
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
The following packages have been kept back:
  linux-image-amd64
The following packages will be upgraded:
  etcd libjs-jquery libssl-dev libssl1.0.0 linux-base openssl
6 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.
Inst libssl-dev [1.0.2d-1~wmf1] (1.0.2e-1~wmf1 Wikimedia:8/jessie-wikimedia [amd64]) []
Inst libssl1.0.0 [1.0.2d-1~wmf1] (1.0.2e-1~wmf1 Wikimedia:8/jessie-wikimedia [amd64])
Inst libjs-jquery [1.7.2+dfsg-3.2] (1.11.3+dfsg-4~bpo8+1 Debian Backports:jessie-backports [all])
Inst linux-base [3.5] (4.0~bpo8+1 Debian Backports:jessie-backports [all])
Inst openssl [1.0.2d-1~wmf1] (1.0.2e-1~wmf1 Wikimedia:8/jessie-wikimedia [amd64])
Inst etcd [2.0.10-1] (2.2.1+dfsg-1 Wikimedia:8/jessie-wikimedia [amd64])
Conf libssl1.0.0 (1.0.2e-1~wmf1 Wikimedia:8/jessie-wikimedia [amd64])
Conf libssl-dev (1.0.2e-1~wmf1 Wikimedia:8/jessie-wikimedia [amd64])
Conf libjs-jquery (1.11.3+dfsg-4~bpo8+1 Debian Backports:jessie-backports [all])
Conf linux-base (4.0~bpo8+1 Debian Backports:jessie-backports [all])
Conf openssl (1.0.2e-1~wmf1 Wikimedia:8/jessie-wikimedia [amd64])
Conf etcd (2.2.1+dfsg-1 Wikimedia:8/jessie-wikimedia [amd64])
W: Duplicate sources.list entry http://mirrors.wikimedia.org/debian/ jessie-backports/main amd64 Packages (/var/lib/apt/lists/mirrors.wikimedia.org_debian_dists_jessie-backports_main_binary-amd64_Packages)
W: You may want to run apt-get update to correct these problems

(the duplicate jessie-backports is unrelated / I have no idea). Filled as T120963: Nodepool instances have duplicate entry for jessie-backports/main.

gerritbot subscribed.Dec 9 2015, 5:42 PM

Change 257940 had a related patch set uploaded (by Hashar):
nodepool: run apt-get upgrade while snapshoting

https://gerrit.wikimedia.org/r/257940

gerritbot added a project: Patch-For-Review.Dec 9 2015, 5:42 PM
hashar claimed this task.Dec 9 2015, 5:42 PM
hashar added a subscriber: zeljkofilipin.

Gotta manually refresh a snapshot to make sure it works as described on https://wikitech.wikimedia.org/wiki/Nodepool#Manually_generate_a_new_snapshot

@zeljkofilipin I guess we can pair this on Thursday.

hashar triaged this task as Medium priority.Dec 9 2015, 5:45 PM
hashar moved this task from Untriaged to In-progress on the Continuous-Integration-Infrastructure board.
hashar moved this task from Backlog to In-progress on the Continuous-Integration-Scaling board.
hashar set Security to None.
gerritbot added a comment.Dec 10 2015, 9:32 AM

Change 257940 merged by jenkins-bot:
nodepool: run apt-get upgrade while snapshoting

https://gerrit.wikimedia.org/r/257940

hashar closed this task as Resolved.Dec 10 2015, 9:42 AM
2015-12-10 09:36:13,725 INFO nodepool.image.build.wmflabs-eqiad.ci-jessie-wikimedia:
apt-get upgrade
Reading package lists...
Building dependency tree...
Reading state information...
The following packages will be upgraded:
   etcd (2.0.10-1 => 2.2.1+dfsg-1)
   libssl-dev (1.0.2d-1~wmf1 => 1.0.2e-1~wmf1)
   libssl1.0.0 (1.0.2d-1~wmf1 => 1.0.2e-1~wmf1)
   openssl (1.0.2d-1~wmf1 => 1.0.2e-1~wmf1)
4 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 13.0 MB of archives.

Available since snapshot image: Image ci-jessie-wikimedia-1449740024 in wmflabs-eqiad is ready

New instances will start using it from now.

greg added a project: OKR-Work.Jan 11 2016, 10:57 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits