Page MenuHomePhabricator

Do not share shm files for openstack uwsgi processes
Closed, ResolvedPublic

Description

While investigating OpenstackAPIResponse I noticed this in the nova-api logs:

2026-04-14 18:44:04.040 2557605 ERROR nova.api.openstack.wsgi PermissionError: [Errno 13] Permission denied: '/dev/shm/cloudcontrol1011_uwsgi_qmanager'

And increased API latencies on 1006/1011 but not 1007 where indeed that message is not present. Digging a little into what that file does turns out it is uwsgi shared memory file, which might end up shared between different uwsgi processes. The end result is a race to write that unique file name, and in this case nova lost the race.

The fix is to provide openstack with a service name for the queue manager to use and thus not have the filenames clash

Event Timeline

Change #1271719 had a related patch set uploaded (by Filippo Giunchedi; author: Filippo Giunchedi):

[operations/puppet@production] openstack: set oslo.messaging processname in uwsgi

https://gerrit.wikimedia.org/r/1271719

Change #1271719 merged by Filippo Giunchedi:

[operations/puppet@production] openstack: set oslo.messaging processname in uwsgi

https://gerrit.wikimedia.org/r/1271719

Mentioned in SAL (#wikimedia-cloud) [2026-04-15T15:02:48Z] <godog> deploy per-service oslo.messaging shared memory file name - T423378

Mentioned in SAL (#wikimedia-cloud) [2026-04-16T06:43:12Z] <godog> roll-restart nova-api to pick up changes - T423378