Fix maps puppet to make sure apt-get update runs after configuration change
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• Mathew.onipe
	Jan 17 2019, 6:52 PM

Description

We ran into some issues when updating maps1003 from jessie to stretch. There was a config(hieradata) change that make sure we use cassandra-2.2 version. Puppet fails with this error:

Error: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold --force-yes install cassandra=2.2.6-wmf5' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
E: Version '2.2.6-wmf5' for 'cassandra' was not found
Error: /Stage[main]/Cassandra/Package[cassandra]/ensure: change from purged to 2.2.6-wmf5 failed: Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold --force-yes install cassandra=2.2.6-wmf5' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
E: Version '2.2.6-wmf5' for 'cassandra' was not found

running apt-get update manually fixed this.
We should look at the puppet config to make sure apt-get update runs after any config change. or package version change.

Details

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	cassandra: package should be installed after apt-get update	operations/puppet	production	+43 -2

Customize query in gerrit

Event Timeline

• Mathew.onipe created this task.Jan 17 2019, 6:52 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 17 2019, 6:52 PM

CDanis triaged this task as Medium priority.Jan 17 2019, 7:57 PM

• EBjune moved this task from needs triage to Up Next on the Discovery-Search board.Jan 24 2019, 6:11 PM

Peachey88 added a project: Puppet.Jan 26 2019, 11:22 PM

• debt moved this task from Up Next to Ops / SRE on the Discovery-Search board.Jan 29 2019, 6:30 PM

Looking at the code:

Apt repository is added by the main cassandra class. It is tagged as before => Package['cassandra'].
[[ https://github.com/wikimedia/puppet/blob/production/modules/apt/manifests/repository.pp#L23-L30 | apt::repository ]] has the proper notify => Exec['apt-get update'], which ** should trigger an update.
[[ https://github.com/wikimedia/puppet/blob/production/modules/apt/manifests/init.pp#L5-L10 | apt class ]] has the definition of the exec, which does not seem to have any guard that would prevent it from running.

Conclusion: the code looks correct, but isn't working. Another pair of eyes would be welcomed!

@Gehel I would say what's missing is a clear dependency between the installation of the cassandra package and the apt-get update.

We'll need to check if that creates some dependency cycle though.

Change 490303 had a related patch set uploaded (by Gehel; owner: Gehel):
[operations/puppet@production] cassandra: package should be installed after apt-get update

https://gerrit.wikimedia.org/r/490303

gerritbot added a project: Patch-For-Review.Feb 13 2019, 10:42 AM

Change 490303 merged by Gehel:
[operations/puppet@production] cassandra: package should be installed after apt-get update

https://gerrit.wikimedia.org/r/490303

Gehel closed this task as Resolved.Feb 15 2019, 9:48 AM

Gehel claimed this task.

Gehel edited projects, added Discovery-Search (Current work); removed Discovery-Search.Feb 12 2025, 9:41 AM

Maintenance_bot removed a project: Patch-For-Review.Feb 12 2025, 10:32 AM