cloudvirt1071 crash
Closed, ResolvedPublic
Actions

Assigned To

None

Authored By

	Andrew
	Nov 18 2025, 11:48 PM

Description

I reimaged cloudvirt1071 to Trixie about an hour ago; just now it crashed and rebooted itself.

Here's the moment of truth:

2025-11-18T23:24:17.864256+00:00 cloudvirt1071 neutron-openvswitch-agent: 2025-11-18 23:24:17.863 1986 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-65568c4b-6c5f-4752-9ab2-f797e16235cb req-27e3aaa3-8d18-485f-b508-3fd43e0f36ec osstackcanary admin-monitoring - - default default] Agent rpc_loop - iteration:4921 completed. Processed ports statistics: {'regular': {'added': 0, 'updated': 0, 'removed': 0}}. Elapsed:0.003
2025-11-18T23:24:19.863739+00:00 cloudvirt1071 neutron-openvswitch-agent: 2025-11-18 23:24:19.862 1986 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-65568c4b-6c5f-4752-9ab2-f797e16235cb req-27e3aaa3-8d18-485f-b508-3fd43e0f36ec osstackcanary admin-monitoring - - default default] Agent rpc_loop - iteration:4922 started
2025-11-18T23:24:19.866606+00:00 cloudvirt1071 neutron-openvswitch-agent: 2025-11-18 23:24:19.865 1986 INFO neutron.plugins.ml2.drivers.openvswitch.agent.ovs_neutron_agent [req-65568c4b-6c5f-4752-9ab2-f797e16235cb req-27e3aaa3-8d18-485f-b508-3fd43e0f36ec osstackcanary admin-monitoring - - default default] Agent rpc_loop - iteration:4922 completed. Processed ports statistics: {'regular': {'added': 0, 'updated': 0, 'removed': 0}}. Elapsed:0.004
2025-11-18T23:24:20.896720+00:00 cloudvirt1071 kernel: tap8de54afb-21: dropped over-mtu packet: 1480 > 1450
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@2025-11-18T23:30:10.278286+00:00 cloudvirt1071 lvm[1033]:   3 logical volume(s) in volume group "vg0" monitored
2025-11-18T23:30:10.278674+00:00 cloudvirt1071 systemd-modules-load[1041]: Inserted module 'br_netfilter'
2025-11-18T23:30:10.278783+00:00 cloudvirt1071 systemd[1]: Mounted dev-hugepages.mount - Huge Pages File System.
2025-11-18T23:30:10.278797+00:00 cloudvirt1071 systemd-modules-load[1041]: Inserted module 'nf_conntrack'
2025-11-18T23:30:10.278807+00:00 cloudvirt1071 systemd[1]: Mounted dev-mqueue.mount - POSIX Message Queue File System.
2025-11-18T23:30:10.278815+00:00 cloudvirt1071 systemd[1]: Mounted run-lock.mount - Legacy Locks Directory /run/lock.
2025-11-18T23:30:10.278824+00:00 cloudvirt1071 systemd-modules-load[1041]: Inserted module 'ipmi_devintf'
2025-11-18T23:30:10.278833+00:00 cloudvirt1071 systemd[1]: Mounted sys-kernel-debug.mount - Kernel Debug File System.
2025-11-18T23:30:10.278850+00:00 cloudvirt1071 systemd[1]: Mounted sys-kernel-tracing.mount - Kernel Trace File System.

That MTU warning is concerning and likely related to T408543 but the log is quite full of them so I don't think that particular one is directly related to the crash.

Related Objects

Mentioned Here: T408543: MTU setting in IPv6 VMs causes issues with Docker

Event Timeline

Andrew created this task.Nov 18 2025, 11:48 PM

Restricted Application added a project: cloud-services-team. · View Herald TranscriptNov 18 2025, 11:48 PM

Restricted Application added a subscriber: Aklapper. · View Herald Transcript

SEL shows this, although the timing seems to be after the reboot:

-------------------------------------------------------------------------------
Record:      7
Date/Time:   11/18/2025 23:26:06
Source:      system
Severity:    Critical
Description: A high-severity issue has occurred at the Power-On Self-Test (POST) phase which has resulted in the system BIOS to abruptly stop functioning.
-------------------------------------------------------------------------------

dcaro triaged this task as High priority.Nov 19 2025, 8:30 AM

dcaro moved this task from Inbox to Hardware on the cloud-services-team board.

dcaro edited projects, added cloud-services-team (Hardware); removed cloud-services-team.

dcaro moved this task from Hardware to FY2025/2026-Q1-Q2 on the cloud-services-team board.

dcaro edited projects, added cloud-services-team (FY2025/2026-Q1-Q2); removed cloud-services-team (Hardware).

The server did not crash again since, and seems to be working fine:

fnegri@cloudvirt1071:~$ uptime
 17:48:26 up 55 days, 18:18,  1 user,  load average: 8.96, 11.11, 11.58
fnegri@cloudvirt1071:~$ sudo virsh list
 Id   Name         State
----------------------------
 1    i-000e0c88   running
 3    i-0005e549   running
 4    i-0006774c   running
 5    i-00086a2e   running
 6    i-00090593   running
 7    i-000a7e13   running
 9    i-000a9460   running
 11   i-000bcf23   running
 12   i-000bd865   running
 13   i-000c4929   running
 14   i-000e6f31   running
 15   i-000e91a8   running
 16   i-000ead11   running
 18   i-000f2d7e   running
 19   i-000f0840   running
 20   i-000a82db   running
 22   i-000eb080   running
 23   i-000d3c35   running
 24   i-0006db13   running
 25   i-0004642f   running

taavi removed fnegri as the assignee of this task.Jan 13 2026, 5:52 PM

cloudvirt1071 crashClosed, ResolvedPublicActions

Description

Related Objects

Event Timeline

cloudvirt1071 crash
Closed, ResolvedPublic
Actions