Page MenuHomePhabricator

Fix recdns config on various hardware devices
Closed, ResolvedPublic


There are two legacy recdns IPs which are still supported (for now) and still in active use by a few devices: and These should both be replaced in all config with with instead. Most of the migration was done months ago, but some hardware devices remain on the old config. A recent 1-week packet capture survey turned up two clients:

  • scs-a1-codfw.mgmt.codfw.wmnet
  • ps1-b5-eqiad.mgmt.eqiad.wmnet

Additionally, SYN packets were found from a "WMCS eqiad/codfw + drmrs + esams OOB" client on both DNS servers:

IP > Flags [S], seq 2423093459, win 1024, options [mss 1460], length 0 (WMCS eqiad/codfw + drmrs + esams OOB)
IP > Flags [S], seq 3220011912, win 1024, options [mss 1460], length 0 (WMCS eqiad/codfw + drmrs + esams OOB)

Related Objects

Event Timeline

Dzahn triaged this task as Medium priority.Jun 4 2020, 9:19 AM

The swap of Traffic for Traffic-Icebox in this ticket's set of tags was based on a bulk action for all such tickets that haven't been updated in 6 months or more. This does not imply any human judgement about the validity or importance of the task, and is simply the first step in a larger task cleanup effort. Further manual triage and/or requests for updates will happen this month for all such tickets. For more detail, have a look at the extended explanation on the main page of Traffic-Icebox . Thank you!

I've updated the list based on a recent week-long tcpdump on both DNS servers.

@Papaul, could you update scs-a1-codfw.mgmt.codfw.wmnet to point to the newer DNS, please?

@Jclark-ctr, could you update ps1-b5-eqiad.mgmt.eqiad.wmnet to point to the newer DNS, please?

@BCornwall I took care of both of them :) is the generic NAT IP for the WMCS realm.

@aborrero, for context, is our old recursive DNS IP and it's being decommissioned (replaced with the internal IP). As WMCS have its own recursive DNS I think those flows are bogus, but we need to make sure it won't break anything when we decom' them. Is it possible to know which VM try to reach them and why?

In a quick search using cumin I didn't find anything relevant:

aborrero@cloud-cumin-03:~$ sudo cumin --force -x '*' "grep /etc/resolv.conf"
[.. nothing relevant ..]

I need to investigate more.

NAT logs or tcpdump on the device doing NAT should help pinpoint the host(s).

It is

There are 1k connections like this:

tcp      6 59 SYN_SENT src= dst= sport=59626 dport=35106 [UNREPLIED] src= dst= sport=35106 dport=59626 mark=0 use=1

With being that VM.

Nice! that makes sens as it scans all our IPs.

@BCornwall I think everything is completed here!

Thanks for handling that, @ayounsi!