Page MenuHomePhabricator

Some Observability clusters apparently do not support IPv6.
Open, LowPublic

Description

Greetings!

During importation of DNS into Netbox as part of the transition to automation, we discovered some clusters do not have IPv6 DNS entries, which we interpreted as intentional (given that this was the mechanism used to prevent potential clients from accessing the IPv6 interfaces on the machine, if a given service did not support IPv6), and prevented from being imported into automation.

We are now triaging these clusters for their potential at supporting IPv6 in the future, so below are hosts which were left out of IPv6 DNS which we think that your team is responsible for. If you could take some time to put any information you have about supporting IPv6 on these clusters, specific plans for doing so, or if it will not in the forseeable future be possible to do so, it would be greatly appreciated!

If any of these machines don't belong to you let us know on this ticket or the parent task (T253173), thanks!

  • centrallog2001.codfw.wmnet
  • centrallog1001.eqiad.wmnet
  • graphite2003.codfw.wmnet
  • graphite1004.eqiad.wmnet
  • logstash[2001-2003,2026-2029].codfw.wmnet
  • logstash[1010-1012,1026-1029].eqiad.wmnet
  • thanos-fe[2001-2003].codfw.wmnet
  • mwlog2001.codfw.wmnet
  • mwlog1001.eqiad.wmnet

Event Timeline

Hello,

Is there a specific timeline you'd like us to meet with this? Mainly the goal is to understand urgency for prioritization. Thanks!

Hi!

So the idea is we'd like overall for all of our clusters to have IPv6 reachability. This is not terribly urgent, just a state that has remained a long time and we'd like to rectify.

The request is along the lines of, what domain-specific knowledge do we need to support IPv6 on these clusters, if possible - really if anything bad will happen if we add IPv6 DNS

For the above clusters, it looks like some of the other hosts in them already have IPv6 DNS, so they may be trivial. There are cases when changes to configuration or FERM need to be made to support it, which largely is what this task is asking about.

crusnov added a subscriber: fgiunchedi.

A quick survey of the clusters above:

  • centrallog[12]001 - is handling anycast syslog, which appears to be being mediated on ipv4 by bird if I'm reading this right. It doesn't seem like adding the host dns for ipv6 would be a problem because of this.
  • graphite - Seems to be only listening on ipv4 for most of its services. There are a lot of things running, and save for a very few are only on ipv4.
  • logstash - All services except for envoy appear to be listening on ipv6
  • thanos-fe - all services except for envoy appear to be listening on ipv6
  • mwlog - The logging services do not appear to be listening on ipv6