Currently only machines within the same cluster can communicate with each others, problem is that with cross cluster search we want node1 to open node2 on port 9700 even if node node1 is not on the cluster responsible for 9700.
- from elastic2042: curl elastic2027.codfw.wmnet:9700 => DENIED
- from elastic2042: curl elastic2027.codfw.wmnet:9600 => OK
- from elastic2042: curl elastic2027.codfw.wmnet:9643 => OK
- from elastic2027: curl elastic2042.codfw.wmnet:9500 => DENIED
- from elastic2027: curl elastic2042.codfw.wmnet:9400 => OK
- from elastic2027: curl elastic2042.codfw.wmnet:9443 => OK
connection opens on all these ports
Same problem is present on eqiad.