Page MenuHomePhabricator

Rethink aiohttp's session reuse in the isvc code
Open, MediumPublic3 Estimated Story Points

Description

We are currently using a function called get_http_client_session in various isvcs as attempt to reuse the same connection pools when connecting to endpoints. For example, we have calls like:

get_http_client_session("mwapi")
get_http_client_session("restgateway")
etc..

Our current network model on Lift Wing relies on the following:

  • http (plaintext) request to the endpoint, that ends up being redirected to the localhost Envoy proxy.
  • the Envoy proxy (Istio) makes then a TLS connection (so using a separate TCP conn pool) to the endpoint (behind the scenes).

This rewrite is needed to allow the Envoy proxy to publish L7/HTTP metrics for our dashboards. We use the Istio Destination Rule config to tune the TCP/TLS connection pool, so we should be ok in simply using a context manager with a new aiohttp session every time in the Python isvc code.

This task should investigate pros/cons of the current approach, and refactor/simplify the code in case a simpler approach is better. With better I mean several things:

  • More maintainable, since there will be no issues related to session re-use by mistake (say if we use the same parameter in get_http_client_session() for two different endpoints).
  • Easier to read and modify.
  • Easier to debug/reason-about when a connection fails for some reason (since with a simple context manager + new session every time it will be less likely to be a problem).

Event Timeline

TL;DR review the current code and investigate if this simplification is useful

calbon reassigned this task from elukey to AikoChou.
calbon set the point value for this task to 3.
calbon moved this task from Unsorted to Ready To Go on the Machine-Learning-Team board.