Page MenuHomePhabricator

Openstack: many orphaned (or seemingly orphaned) VMs in codfw1dev
Closed, ResolvedPublic

Description

I'm trying to enumerate VMs in codfw1dev and finding many VMs that are associated with projects that can't be found, for example:

k8s-dev
pawsdev
andrewtestproject
f93ed978bc5943c4b9c3fb1bfc70d3b4

Most likely the projects were deleted before being emptied first, in which case we just need to clean up the resources they used to contain. Less likely, something interesting is happening causing me and my sript to be unable to see those projects...

I recall deleting a few projects a few months ago but I don't think I deleted all of these.

Event Timeline

Here are the orphaned VMs:

  • f93ed978bc5943c4b9c3fb1bfc70d3b4: f950d7b4-ac86-4d12-b19a-096c0d52cd19
  • f93ed978bc5943c4b9c3fb1bfc70d3b4: 4e67370c-1db0-411a-b2a4-e3107f8a9057
  • pawsdev: ef794b03-03b8-46f4-9901-d9a7e6beaaa7
  • pawsdev: b3bbe68d-064b-46a9-b2ea-95eda53292b4
  • k8s-dev: 9f57d4fb-506d-4c6c-a2a3-566b57abe095
  • andrewtestproject: 84864e4b-5ac8-4c56-bbc8-3a0dbeaea1cb
  • taavi-test-project: 7837b970-51ec-404a-8d81-3d71ab01c923
  • taavi-test-project: 6c7ff713-8af7-4761-a996-4ce0dbf09618

I'm going to clean those up now but likely there are other resources left around (mostly dns domains)

Hmph, 84864e4b-5ac8-4c56-bbc8-3a0dbeaea1cb is wedged in a way that was harmless until andrewtestproject was deleted and now I need to do some database-diving.