Got this in logs:
07:10:52.892 [qtp226170135-45141] INFO o.w.q.r.b.t.ThrottlingFilter IP:localhost UA:python-requests/2.12.4 - A request is being banned.
So this is the result of "bad user agent" policy where user agents having only python-requests can get banned by user agent. Which has the problem that we are our own enemy - we also are using bad user agent for our monitoring scripts, apparently. We should fix that ASAP.