Page MenuHomePhabricator

Old instance showing in openstack-browser tool, possibly still in keystone
Closed, ResolvedPublic

Description

While browsing openstack-browser for a role at https://openstack-browser.toolforge.org/puppetclass/role::ci::slave::labs::docker I have encountered a couple no more existing instances:

integration-slave-docker-c2-m4-d40-1004.integration.eqiad.wmflab*
integration-slave-docker-c2-m4-d40-1005.integration.eqiad.wmflab*

They have been created in October 2017 (SAL search ) and have long been deleted.

The instance do not show up on the project view https://openstack-browser.toolforge.org/project/integration

Not sure why there is * at the end of the hostname, I thought it might be due to a Puppet prefix set in Horizon but there are no prefix defined.

Event Timeline

Mentioned in SAL (#wikimedia-cloud) [2022-01-29T18:24:48Z] <taavi> delete 2 puppet prefixes in a weird state T299750

taavi claimed this task.
taavi added a subscriber: taavi.

The puppet data is stored in a completely separate database from Nova instances. Not sure what has happened here, but those two hosts are indeed in a weird state with a wrong domain name. Fixed:

MariaDB [labspuppet]> select * from prefix where prefix like '%.wmflab';
+------+-------------+------------------------------------------------------------------+
| id   | project     | prefix                                                           |
+------+-------------+------------------------------------------------------------------+
| 3772 | integration | integration-slave-docker-c2-m4-d40-1004.integration.eqiad.wmflab |
| 3546 | integration | integration-slave-docker-c2-m4-d40-1005.integration.eqiad.wmflab |
+------+-------------+------------------------------------------------------------------+
2 rows in set (0.00 sec)

MariaDB [labspuppet]> delete from prefix where project = 'integration' and id in (3546, 3772);
Query OK, 2 rows affected (0.02 sec)

Thank you for the detailed explanation @Majavah