Page MenuHomePhabricator

Realtime Client connecting to same kslqdb instance
Closed, DeclinedPublic5 Estimated Story PointsBUG REPORT

Description

This behaviour is seen in production.
Ticket Related to O 6 KR 1
We have 2 instances of ksqldb running but when a client connects, within a short period of time, all is connections are going to the same ksqldb instance.

How to replicate

  • Scale ksqldb to 2 instances
  • connect to realtime API with multiple clients.

Acceptance Criteria

  • Agree on solution.
  • Implement and test in DEV.

Developer Notes

  • DNS resolution in same order due to quick dns lookups, skip the dns balancing. We should resolve the dns before connecting. The dial will use the sequencialDial to connect.

RFC
Refer to RfC-realtime-reconnection-delay

Related Objects

StatusSubtypeAssignedTask
ResolvedNone
DeclinedBUG REPORTprabhat

Event Timeline

@Protsack.stephan Can you add context here where we doing it already, please?

Thanks

We reviewed the GoDropBox 3rd party library, but it doesn't do what we need to server-side round-robin scheduling. It does connection pooling for the client side by keeping a set of persistent connections and doing round-robin scheduling for the client connections.

The code changes that Ricardo made are good, we can probably proceed with his MR. The only question is do we use random IP selection, or save a counter and cycle through the IPs sequentially? The problem with counter is we need to deal with state and mutex, whereas random IP selection is stateless.

I vote that we stay with Ricardo's MR and merge to dev and run some QA tests

Research finalized, team decided on a different approach.