deployment-apertium01 puppet failing due to missing packages on trusty
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	• AlexMonk-WMF
	Oct 3 2016, 5:21 PM

Description

krenair@deployment-apertium01:~$ sudo puppet agent -tv 2>&1 | grep E: | sort -d | uniq
E: Unable to locate package apertium-arg
E: Unable to locate package apertium-arg-cat
E: Unable to locate package apertium-cat
E: Unable to locate package apertium-fra
E: Unable to locate package apertium-fra-cat
E: Unable to locate package apertium-ita
E: Unable to locate package apertium-sme-nob
E: Unable to locate package apertium-spa
E: Unable to locate package apertium-spa-arg
E: Unable to locate package apertium-srd
E: Unable to locate package apertium-srd-ita
E: Unable to locate package apertium-swe
E: Unable to locate package apertium-swe-dan
E: Unable to locate package apertium-swe-nor
E: Unable to locate package giella-core
E: Unable to locate package giella-sme

We might want to do T142152 instead. T107306 already dealt with these for jessie.

Related Objects
Search...

		Status	Subtype	Assigned	Task
		Resolved		EddieGP	T132259 Deployment-prep hosts with puppet errors (tracking)
		Resolved		akosiaris	T147210 deployment-apertium01 puppet failing due to missing packages on trusty

Event Timeline

• AlexMonk-WMF created this task.Oct 3 2016, 5:21 PM

This is happening due to cherry-pick of https://gerrit.wikimedia.org/r/#/c/308679/ which is for testing before deployment in Production (https://phabricator.wikimedia.org/T144588) (scheduled this week).

deployment-apertium01 is a Trusty instance. Maybe we can move apertium to the deployment-sca* instances which are jessie? that is T142152: Move apertium to deployment-sca* hosts in Beta Cluster.

hashar moved this task from To Triage to Backlog on the Beta-Cluster-Infrastructure board.Oct 4 2016, 8:36 AM

https://gerrit.wikimedia.org/r/#/c/308679/ fixes this. I 'll close as resolved

deployment-apertium01 (Trusty) has a bunch of missing packages again eg:

# tail -n 180 /var/log/puppet.log|egrep ^E
E: Unable to locate package apertium-swe
E: Unable to locate package apertium-swe
E: Unable to locate package giella-sme
E: Unable to locate package giella-sme
E: Unable to locate package apertium-cat
E: Unable to locate package apertium-cat
E: Unable to locate package apertium-arg
E: Unable to locate package apertium-arg
E: Unable to locate package apertium-srd
E: Unable to locate package apertium-srd
E: Unable to locate package apertium-swe-nor
E: Unable to locate package apertium-swe-nor
E: Unable to locate package apertium-spa-arg
E: Unable to locate package apertium-spa-arg
E: Unable to locate package apertium-spa
E: Unable to locate package apertium-spa
E: Unable to locate package apertium-sme-nob
E: Unable to locate package apertium-sme-nob
E: Unable to locate package apertium-ita
E: Unable to locate package apertium-ita
E: Unable to locate package apertium-swe-dan
E: Unable to locate package apertium-swe-dan
E: Unable to locate package apertium-srd-ita
E: Unable to locate package apertium-srd-ita
E: Unable to locate package apertium-fra
E: Unable to locate package apertium-fra
E: Unable to locate package apertium-arg-cat
E: Unable to locate package apertium-arg-cat
E: Unable to locate package giella-core
E: Unable to locate package giella-core
E: Unable to locate package apertium-fra-cat
E: Unable to locate package apertium-fra-cat
root@deployment-apertium01:~#

Instance IP address is 10.68.16.79

@akosiaris confirmed production moved to Jessie. So I guess we can delete deployment-apertium01 and clean up left over configuration files :}

@KartikMistry I am pretty sure we can delete that VM, could you please confirm ?

@akosiaris @hashar Yes. We can delete deployment-apertium01.

Instance terminated.

deployment-apertium01 is still around and complaining. Maybe the deletion failed in wikitech/horizon?

I have terminated deployment-apertium01 using Horizon.

Mentioned in SAL (#wikimedia-releng) [2016-11-17T11:34:17Z] <hasharLunch> Deleted instance deployment-apertium01 . Was Trusty and lacked packages, replaced by a Jessie one ages ago. T147210

Somehow the instance deployment-apertium01 is back!

$ uptime
 10:52:58 up 21 days, 15:25,  1 user,  load average: 0.00, 0.09, 0.10
             ^^^^^^^

And thus Shinken cames back to complain about puppet being broken: ** PROBLEM alert - deployment-apertium01/Puppet run is CRITICAL **

Mentioned in SAL (#wikimedia-releng) [2016-11-23T10:57:43Z] <hashar> Terminating deployment-apertium01 again T147210

With 21 days uptime ? I think it's just not deleted. otherwise this does not make sense.

From https://wikitech.wikimedia.org/wiki/Nova_Resource:Deployment-apertium01.deployment-prep.eqiad.wmflabs

Sorry, this page was recently deleted (within the last 24 hours). The deletion and move log for the page are provided below for reference.

10:57, 23 November 2016 Labslogbot (talk | contribs) deleted page Nova Resource:Deployment-apertium01.deployment-prep.eqiad.wmflabs (Resource deleted)

19:10, 18 November 2016 Delete page script (talk | contribs) deleted page Nova Resource:Deployment-apertium01.deployment-prep.eqiad.wmflabs

21:53, 9 November 2016 Delete page script (talk | contribs) deleted page Nova Resource:Deployment-apertium01.deployment-prep.eqiad.wmflabs

23:41, 4 December 2015 Andrew Bogott (talk | contribs) deleted page Nova Resource:Deployment-apertium01.deployment-prep.eqiad.wmflabs (Cleaning up old instance pages)

Nov 9th would be Alexandros deleting it, Nov 18th and Nov 23th are me. Maybe it is worth filling a new task against OpenStack ?

Apparently it is gone for real. deployment-apertium01 hasn't reappeared and does not show up in the Horizon interface.

deployment-apertium01 puppet failing due to missing packages on trustyClosed, ResolvedPublicActions

Description

Related ObjectsSearch...

Event Timeline

deployment-apertium01 puppet failing due to missing packages on trusty
Closed, ResolvedPublic
Actions

Related Objects
Search...