Page MenuHomePhabricator
Paste P8407

instance metadata problem
ActivePublic

Authored by Krenair on Apr 16 2019, 4:45 PM.
Tags
None
Referenced Files
F28678188: raw.txt
Apr 16 2019, 4:45 PM
Subscribers
0root@deployment-puppetmaster03:/var/lib/git/operations/puppet(production u+23)# curl -v 'http://169.254.169.254/latest/meta-data/ami-launch-index'
* Trying 169.254.169.254...
* TCP_NODELAY set
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> GET /latest/meta-data/ami-launch-index HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 200 OK
< Content-Type: text/plain; charset=UTF-8
< Content-Length: 1
< Date: Tue, 16 Apr 2019 16:44:20 GMT
<
* Curl_http_done: called premature == 0
* Connection #0 to host 169.254.169.254 left intact
0root@deployment-puppetmaster03:/var/lib/git/operations/puppet(production u+23)# curl -v 'http://169.254.169.254/latest/meta-data/ami-launch-index'
* Trying 169.254.169.254...
* TCP_NODELAY set
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> GET /latest/meta-data/ami-launch-index HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.52.1
> Accept: */*
>
< HTTP/1.1 500 Internal Server Error
< Content-Length: 92
< Content-Type: text/plain; charset=UTF-8
< Date: Tue, 16 Apr 2019 16:44:51 GMT
<
500 Internal Server Error
Remote metadata server experienced an internal server error.
* Curl_http_done: called premature == 0
* Connection #0 to host 169.254.169.254 left intact

Event Timeline

I have seen this, and it has generally cleared up on its own. I don't see anything obvious in the nova-api logs (the actual back-end metadata service) to indicate trouble.

So the next thing to figure out is whether when this happens it only affects clusters of VMs that live on the same cloudvirt host.