CloudVirtDown Cloudvirt node cloudvirt1063 is down. #page
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	phaultfinder
	Apr 3 2025, 8:25 PM

Description

dashboard: https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=cloudvirt1063
description: The cloudvirt node cloudvirt1063 is unreachable. This is a major outage that requires you to either restore the server or evacuate manually the VMs on it.

runbook: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/CloudVirtDown
summary: Cloudvirt node cloudvirt1063 is down. #page

dashboard: https://grafana.wikimedia.org/d/000000377/host-overview?orgId=1&var-server=cloudvirt1063
description: The cloudvirt node cloudvirt1063 is unreachable. This is a major outage that requires you to either restore the server or evacuate manually the VMs on it.
runbook: https://wikitech.wikimedia.org/wiki/Portal:Cloud_VPS/Admin/Runbooks/CloudVirtDown
summary: Cloudvirt node cloudvirt1063 is down. #page
alertname: CloudVirtDown
cluster: wmcs
instance: cloudvirt1063:9100
job: node
prometheus: ops
severity: page
site: eqiad
source: prometheus
team: wmcs
Source

Status	Assigned	Task
Open	None	T53494 Use Beta cluster as a true canary for code deployments (epic)
Declined	None	T87220 Minimize infrastructure differences between Beta Cluster and production
Open	None	T211677 Support IPv6 in beta
Open	None	T404466 Support IPv6-only VMs
Open	None	T392688 Enable IPv6 on Cloud VPS infrastructure services
Resolved	taavi	T379175 Enable IPv6 for the Cloud VPS web proxy
Open	None	T270694 CloudVPS: introduce tenant networks
Resolved	• aborrero	T364725 Migrate Cloud VPS instances to VXLAN based networks
Open	None	T392509 Enable IPv6 for Toolforge services
Resolved	taavi	T392506 Enable IPv6 for tools.wmflabs.org / *.toolserver.org legacy redirector service
Resolved	taavi	T211575 Enable IPv6 on toolforge.org
Open	None	T220306 Add IPv6 monitoring
Resolved	• aborrero	T37947 Enable IPv6 on CloudVPS
Resolved	• aborrero	T380174 CloudVPS: IPv6 in eqiad1
Resolved	• aborrero	T380728 openstack: network problems when introducing new networks
Resolved	cmooney	T389958 WMCS Eqiad: Enable IPv6 in cloud vrf on switches
Resolved	• aborrero	T391040 CloudVirtDown Cloudvirt node cloudvirt1063 is down. #page