Page MenuHomePhabricator

Review and fix PDU settings for syslog/ntp/email servers
Closed, ResolvedPublic

Description

While checking PDU readings for T171823 I noticed the settings for SNTP and syslog under /main.html?3,10 point to pmtpa. Both SNTP and syslog should be changed to point to eqiad and codfw, namely:

  • ntp.eqiad.wikimedia.org + ntp.codfw.wikimedia.org
  • syslog.eqiad.wmnet + syslog.codfw.wmnet

And verified that syslog/ntp can get out of mgmt network towards the respective servers too.

Ditto for email settings at /main.html?3,14, the feature is enabled and points to smtp.pmtpa.wmnet. Since we're monitoring via snmp and icinga for e.g. phase imbalance and logging to syslog for access audit I think we can just disable the email feature instead.

hostnameupgradedre-configured
ps1-a1-codfw.mgmt.codfw.wmnetyesyes
ps1-a2-codfw.mgmt.codfw.wmnetyesyes
ps1-a3-codfw.mgmt.codfw.wmnetyesyes
ps1-a4-codfw.mgmt.codfw.wmnetyesyes
ps1-a5-codfw.mgmt.codfw.wmnetyesyes
ps1-a6-codfw.mgmt.codfw.wmnetyesyes
ps1-a7-codfw.mgmt.codfw.wmnetyesyes
ps1-a8-codfw.mgmt.codfw.wmnetyesyes
ps1-b1-codfw.mgmt.codfw.wmnetyesyes
ps1-b2-codfw.mgmt.codfw.wmnetyesyes
ps1-b3-codfw.mgmt.codfw.wmnetyesyes
ps1-b4-codfw.mgmt.codfw.wmnetyesyes
ps1-b6-codfw.mgmt.codfw.wmnetyesyes
ps1-b7-codfw.mgmt.codfw.wmnetyesyes
ps1-b8-codfw.mgmt.codfw.wmnetyesyes
ps1-c1-codfw.mgmt.codfw.wmnetyesyes
ps1-c2-codfw.mgmt.codfw.wmnetyesyes
ps1-c3-codfw.mgmt.codfw.wmnetyesyes
ps1-c4-codfw.mgmt.codfw.wmnetyesyes
ps1-c5-codfw.mgmt.codfw.wmnetyesyes
ps1-c6-codfw.mgmt.codfw.wmnetyesyes
ps1-c7-codfw.mgmt.codfw.wmnetyesyes
ps1-c8-codfw.mgmt.codfw.wmnetyesyes
ps1-d1-codfw.mgmt.codfw.wmnetyesyes
ps1-d3-codfw.mgmt.codfw.wmnetyesyes
ps1-d4-codfw.mgmt.codfw.wmnetyesyes
ps1-d5-codfw.mgmt.codfw.wmnetyesyes
ps1-d6-codfw.mgmt.codfw.wmnetyesyes
ps1-d7-codfw.mgmt.codfw.wmnetyesyes
ps1-d8-codfw.mgmt.codfw.wmnetyesyes
ps1-d2-codfw.mgmt.codfw.wmnetyesyes
ps1-b5-codfw.mgmt.codfw.wmnetyesyes
ps1-a1-eqiad.mgmt.eqiad.wmnetyesyes
ps1-a2-eqiad.mgmt.eqiad.wmnetyesyes
ps1-a3-eqiad.mgmt.eqiad.wmnetyesyes
ps1-a4-eqiad.mgmt.eqiad.wmnetyesyes
ps1-a5-eqiad.mgmt.eqiad.wmnetyesyes
ps1-a6-eqiad.mgmt.eqiad.wmnetyesyes
ps1-a7-eqiad.mgmt.eqiad.wmnetyesyes
ps1-a8-eqiad.mgmt.eqiad.wmnetyesyes
ps1-b1-eqiad.mgmt.eqiad.wmnetyesyes
ps1-b2-eqiad.mgmt.eqiad.wmnetyesyes
ps1-b3-eqiad.mgmt.eqiad.wmnetyesyes
ps1-b4-eqiad.mgmt.eqiad.wmnetyesyes
ps1-b5-eqiad.mgmt.eqiad.wmnetyesyes
ps1-b6-eqiad.mgmt.eqiad.wmnetyesyes
ps1-b7-eqiad.mgmt.eqiad.wmnetyesyes
ps1-b8-eqiad.mgmt.eqiad.wmnetyesyes
ps1-c1-eqiad.mgmt.eqiad.wmnetyesyes
ps1-c2-eqiad.mgmt.eqiad.wmnetyesyes
ps1-c3-eqiad.mgmt.eqiad.wmnetyesyes
ps1-c4-eqiad.mgmt.eqiad.wmnetyesyes
ps1-c5-eqiad.mgmt.eqiad.wmnetyesyes
ps1-c6-eqiad.mgmt.eqiad.wmnetyesyes
ps1-c7-eqiad.mgmt.eqiad.wmnetyesyes
ps1-c8-eqiad.mgmt.eqiad.wmnetyesyes
ps1-d1-eqiad.mgmt.eqiad.wmnetyesyes
ps1-d2-eqiad.mgmt.eqiad.wmnetyesyes
ps1-d3-eqiad.mgmt.eqiad.wmnetyesyes
ps1-d4-eqiad.mgmt.eqiad.wmnetyesyes
ps1-d5-eqiad.mgmt.eqiad.wmnetyesyes
ps1-d6-eqiad.mgmt.eqiad.wmnetyesyes
ps1-d7-eqiad.mgmt.eqiad.wmnetyesyes
ps1-d8-eqiad.mgmt.eqiad.wmnetyesyes

Event Timeline

fgiunchedi renamed this task from Review and fix PDU settings for syslog/ntp servers to Review and fix PDU settings for syslog/ntp/email servers.Sep 8 2017, 9:43 AM
fgiunchedi updated the task description. (Show Details)

On ps1-d2-eqiad.mgmt.eqiad.wmnet:

  • FQDN set
  • NTP set (verified working)
  • DNS set
  • Syslog set to librenms (verified working) and syslog.eqiad.wmnet
  • Upgraded to most recent firmware
  • Email disabled

Please review the device's config, if all good, I'll do the same on the other devices.

Thanks @ayounsi ! Looks good to me, some things I found:

  • I can't reach its port 443 e.g. from bast3002 (connection refused) though ssh works and I'm seeing SSL: Enabled Port: 443
  • Only one DNS set 208.80.154.254, likely not a big deal but while we're at it might as well have two
ayounsi claimed this task.