Page MenuHomePhabricator

Cumin's OpenStack backend appears to be broken after labs keystone region merge
Closed, ResolvedPublic

Description

Follow-up from T201504: cloudvps: main/eqiad1 keystone merge

krenair@deployment-cumin:~$ sudo cumin '*' id
No hosts found that matches the query
krenair@deployment-cumin:~$ sudo cumin 'P{*}' id
70 hosts will be targeted:

Event Timeline

Krenair created this task.Aug 13 2018, 10:55 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 13 2018, 10:55 PM

It does have the new keystone URL so I'm guessing this is a regions thing:

root@deployment-cumin:~# grep auth /etc/cumin/config.yaml 
    auth_url: http://cloudcontrol1003.wikimedia.org:5000

Yeah I live hacked deployment-cumin's /usr/lib/python3/dist-packages/cumin/backends/openstack.py to set region_name='eqiad' when constructing nova_client.Client and it works now.
So maybe we should either pull the region from config, or iterate through each one like https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/452427/3/modules/dnsrecursor/files/labs-ip-alias-dump.py

Volans claimed this task.Aug 14 2018, 6:28 AM
Volans triaged this task as High priority.

Change 452608 had a related patch set uploaded (by Volans; owner: Volans):
[operations/software/cumin@master] OpenStack: add custom parameters for the client

https://gerrit.wikimedia.org/r/452608

Change 452609 had a related patch set uploaded (by Volans; owner: Volans):
[operations/puppet@production] cumin: add region_name to the WMCS openstack config

https://gerrit.wikimedia.org/r/452609

Change 452608 merged by jenkins-bot:
[operations/software/cumin@master] OpenStack: add custom parameters for the client

https://gerrit.wikimedia.org/r/452608

Change 452609 merged by Volans:
[operations/puppet@production] cumin: add region_name to the WMCS openstack config

https://gerrit.wikimedia.org/r/452609

Mentioned in SAL (#wikimedia-operations) [2018-08-14T10:41:26Z] <volans> upgraded cumin on labpuppetmaster* to fix cumin with the new openstack region - T201881

Volans closed this task as Resolved.Aug 14 2018, 10:43 AM

Thanks for reporting the issue.

cumin 3.0.2-2 has been uploaded to apt.w.o and labpuppetmaster* have been upgraded, together with a Puppet patch to use the region set in hiera.

Resolving, feel free to re-open if you encounter additional issues in WMCS VMs related to this.