Change Details

We have a small MinIO cluster and have Trino configured to use the first node (by hostname) in the cluster as its S3 endpoint. This creates a SPOF, if that node goes down Trino loses it's MinIO connection though other MinIO nodes are available. We should come with an approach to distribute connections across available MinIO nodes since they're functionally equivalent. Trino doesn't appear to be able to work with a list of S3 endpoints. If the goal were just to distribute load we could use a basic DNS RR. But to handle down MinIO servers, what about a service like HAProxy on each Trino server as a layer between Trino and the MinIO pool?

We have a small MinIO cluster and have Trino configured to use the first node (by hostname) in the cluster as its S3 endpoint. This creates a SPOF, if that node goes down Trino loses it's MinIO connection though other MinIO nodes are available. We should come up with an approach to distribute connections across available MinIO nodes since they're functionally equivalent. Trino doesn't appear to be able to work with a list of S3 endpoints. If the goal were just to distribute load we could use a basic DNS RR. But to handle down MinIO servers, what about a service like HAProxy on each Trino server as a layer between Trino and the MinIO pool?

We have a small MinIO cluster and have Trino configured to use the first node (by hostname) in the cluster as its S3 endpoint. This creates a SPOF, if that node goes down Trino loses it's MinIO connection though other MinIO nodes are available. We should come up with an approach to distribute connections across available MinIO nodes since they're functionally equivalent. Trino doesn't appear to be able to work with a list of S3 endpoints. If the goal were just to distribute load we could use a basic DNS RR. But to handle down MinIO servers, what about a service like HAProxy on each Trino server as a layer between Trino and the MinIO pool?