If I'm reading the python-swiftclient code correctly, there's no timeout on get_object. As surfaced by the Varnish slow log, this means that Thumbor sometimes spends a lot of time on that operation (and possibly writing to Swift as well, but that part doesn't slow down responses to clients).
Measuring that duration and reporting it in the response headers would allow us to verify whether or not 60+ seconds 200 responses from Thumbor are caused by Swift or something else in the thumbnailing stack.