Page MenuHomePhabricator

Update PDUs name-server config
Closed, ResolvedPublic

Description

Follow up from T239993#7501023
Some PDUs still use the old name-server config (208.80.153.254) which needs to be updated to 10.3.0.1.

Cf. https://wikitech.wikimedia.org/wiki/SRE/Dc-operations/Platform-specific_documentation/ServerTech#Setting_up_the_Configuration

So far, the ones that need fixing are:

  • ps1-a5-codfw
  • ps1-d3-codfw
  • ps1-b2-codfw
  • ps1-22-ulsfo
  • ps1-23-ulsfo - didn't list the old nameserver address, instead had no nameserver address and wasn't initially listed on this task.

Related Objects

Event Timeline

ayounsi created this task.
RobH subscribed.

So in the past I never set DNS nameservers on the PDU network settings. To that point, 22 has the old nameserver on it, but pdu 23 in ulsfo has the dns server entries blank.

Do we need to populate this across all PDUs? If so, we likely need to audit as ps1-23-ulsfo didn't make this list when it was blank (rather than wrong.)

For now I set both ps1-[22|23]-ulsfo to 10.3.0.1.

So I'm reassignign this back to Arzhel for his input on why PDUs need name resolution (for logging?) If they do, we should likely expand this task or create a new one to audit and check all PDU nameserver settings across all sites.

We usually use the FQDN for logging and NTP endpoints, see https://wikitech.wikimedia.org/wiki/SRE/Dc-operations/Platform-specific_documentation/ServerTech#Setting_up_the_Configuration

So I'd assume that if DNS is not configured, neither is NTP or syslog.

That said, they're there mostly as a "just in case" but I don't think we've ever looked at PDUs logs as LibreNMS shows all the relevant metrics.

Manually auditing all the PDUs is a pain, so maybe we can script it, check syslog to figure out if any PDU haven't send any logs in X amount of time or just leave them as it. Up to you!

Edit: not sure if it works for all the PDUs but this could be an options:
netmon1002:~$ snmpwalk -v2c -c <community> ps1-22-ulsfo.mgmt.ulsfo.wmnet 1.3.6.1.2.1.7.7.1 | grep 10.3.0.1
It checks the UDP counters for a reference to 10.3.0.1