Page MenuHomePhabricator

dropped packets to phab1003 22280/tcp
Closed, ResolvedPublic

Description

Working on a Kibana dashboard for iptables I noticed that a lot of internal hosts try to reach phab1003 on port 22280/tcp

See https://logstash.wikimedia.org/goto/23faeb7f40dc1205cb58007b65020c68

Maybe there is a miss-configuration somewhere or a missing ferm rule?

Event Timeline

ayounsi triaged this task as Medium priority.Nov 20 2019, 7:06 PM
ayounsi created this task.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptNov 20 2019, 7:06 PM
Dzahn claimed this task.Nov 20 2019, 7:10 PM

Port 22280/tcp is the aphlict service which is currently disabled (T238593).

In addition to the backend It has also been disabled in ATS (https://gerrit.wikimedia.org/r/c/operations/puppet/+/551731)

Change 552122 had a related patch set uploaded (by Dzahn; owner: Dzahn):
[operations/puppet@production] varnish: remove config for disabled phab_aphlict

https://gerrit.wikimedia.org/r/552122

Dzahn added a comment.Nov 20 2019, 7:36 PM

@ema @Vgutierrez So this was removed from ATS in https://gerrit.wikimedia.org/r/c/operations/puppet/+/551731 but does it also need https://gerrit.wikimedia.org/r/c/operations/puppet/+/552122 ?

The source hosts in the logstash link above are several cp servers.

f.e. :

10.64.48.101 - cp1087
10.64.32.67 - cp1083
2620:0:861:103:10:64:32:67 - cp1083
10.64.32.69 - cp1085

@Dzahn that's right, Removing the wss:// -> ws:// from ats-tls doesn't stop the requests from reaching varnish-fe as they are accepted as part of the catch-all remap rule.

Change 552122 merged by Dzahn:
[operations/puppet@production] varnish: remove config for disabled phab_aphlict

https://gerrit.wikimedia.org/r/552122

Dzahn added a comment.Nov 21 2019, 5:59 PM

@ayounsi After the merge above this is expected to stop soon, now.

ayounsi closed this task as Resolved.Nov 22 2019, 7:41 PM

Confirmed!