NovafullstackSustainedFailures cloudcontrol1005:9100 The automated tests were unable to create, provision and decommission a VM in the last 5h
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	phaultfinder
	Nov 3 2022, 2:34 AM

Description

dashboard: https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-nova-fullstack?orgId=1
description: The automated tests were unable to create, provision and decommission a VM in the last 5h
runbook: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/NovafullstackSustainedFailures
summary: Novafullstack tests have been failing for more than 5hours in eqiad

dashboard: https://grafana.wikimedia.org/d/ebJoA6VWz/wmcs-nova-fullstack?orgId=1
description: The automated tests were unable to create, provision and decommission a VM in the last 5h
runbook: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/NovafullstackSustainedFailures
summary: Novafullstack tests have been failing for more than 5hours in eqiad
alertname: NovafullstackSustainedFailures
cluster: wmcs
instance: cloudcontrol1005:9100
job: node
prometheus: ops
service: openstack,cloudvps,novafullstack
severity: task
site: eqiad
source: prometheus
team: wmcs
Source

Related Changes in Gerrit:

	Subject	Repo	Branch	Lines +/-
	novafullstack: don't crash if got error cleaning up some VMs	operations/puppet	production	+15 -2

Mentioned in SAL (#wikimedia-cloud) [2022-12-16T08:46:39Z] <dcaro> restart designate-sink on both cloudservice hosts (T322279)

Change 868618 had a related patch set uploaded (by David Caro; author: David Caro):

[operations/puppet@production] novafullstack: don't crash if got error cleaning up some VMs

Change 868618 merged by David Caro:

[operations/puppet@production] novafullstack: don't crash if got error cleaning up some VMs

fnegri moved this task from Kanban to Inbox on the cloud-services-team board.

taavi closed this task as Resolved.Jan 29 2023, 10:13 AM

taavi claimed this task.