Page MenuHomePhabricator

Move kafka200[123] to logstash202[012]
Closed, ResolvedPublic

Description

Hello, here is a task to track renaming hardware, switchports, etc. for codfw systems that were repurposed. The systems were renamed as follows:

kafka2001 -> logstash2020
kafka2002 -> logstash2021
kafka2003 -> logstash2022

Event Timeline

herron triaged this task as Medium priority.Oct 9 2019, 8:10 PM
herron created this task.
wiki_willy added subscribers: Papaul, wiki_willy.

Hi @Papaul - this task is relabel, update in Netbox, and update switchport descriptions to the newly renamed hostnames. Thanks, Willy

papaul@asw-a-codfw# show | compare 
[edit interfaces ge-5/0/9]
-   description kafka2001;
+   description logstash2020;
papaul@asw-b-codfw# show | compare 
[edit interfaces ge-5/0/33]
-   description kafka2002;
+   description logstash2021;
papaul@asw-c-codfw# show | compare 
[edit interfaces ge-5/0/16]
-   description kafka2003;
+   description logstash2022;

Please note that the hostname mismatch in netbox versus puppet was causing reporting errors on the puppetdb netbox report.

To fix this, I have done the following:

  • renamed kafka2001 to logstash2020 in netbox could not update switch port description as no port is labeled kafka2001 or logstash2020 on asw-a-codfw.
  • renamed kafka2002 to logstash2021 in netbox, could not update switch port description as no port is labeled kafka2002 or logstash2021 on asw-b-codfw.
  • renamed kafka2003 to logstash2022 in netbox, could not update switch port as there is already a port labeled logstash2022 on the switch stack, and no port labeled kafka2003.

End results:

on-site must still apply hostname labels to all 3 servers
on-site needs to trace out logstash1020 and logstash1021 as their switch port descriptions are not properly set, set correct descriptions. if @Papaul recalls renaming logstash2022 on the switch stack, great, otherwise may want to trace that out and confirm.

Switch port output demonstrating lack of port descriptions for first two servers:

robh@asw-a-codfw>       

{master:7}
robh@asw-a-codfw> show interfaces descriptions | grep kafka 
xe-4/0/19       up    up   kafka-main2001

{master:7}
robh@asw-a-codfw> show interfaces descriptions | grep logstash 
ge-5/0/9        up    up   logstash2000
ge-5/0/18       up    up   logstash2001

{master:7}
robh@asw-a-codfw> exit 

Connection to asw-a-codfw.mgmt.codfw.wmnet closed.
Killed by signal 1.

robh@gir:~/Documents/wikimedia/git/dns/templates (master)
15 $> ssh asw-b-codfw.mgmt.codfw.wmnet
--- JUNOS 14.1X53-D43.7 built 2017-04-28 02:42:05 UTC
{master:2}
robh@asw-b-codfw> show interfaces descriptions | grep kafka 
xe-4/0/2        up    up   kafka-main2002

{master:2}
robh@asw-b-codfw> show interfaces descriptions | grep logstash 
ge-5/0/33       up    up   logstash2001

{master:2}
robh@asw-b-codfw> exit 

Connection to asw-b-codfw.mgmt.codfw.wmnet closed.
Killed by signal 1.

robh@gir:~/Documents/wikimedia/git/dns/templates (master)
16 $> ssh asw-c-codfw.mgmt.codfw.wmnet
--- JUNOS 14.1X53-D43.7 built 2017-04-28 02:42:05 UTC
{master:2}
robh@asw-c-codfw> show interfaces descriptions | grep kafka 
xe-7/0/8        up    up   kafka-main2003

{master:2}
robh@asw-c-codfw> show interfaces descriptions | grep logstash 
ge-5/0/16       up    up   logstash2022
ge-5/0/29       up    up   logstash2002

{master:2}
robh@asw-c-codfw>

Mentioned in SAL (#wikimedia-operations) [2019-10-11T08:32:21Z] <moritzm> remove kafka1001-1003 from debmonitor DB (T235125)

Mentioned in SAL (#wikimedia-operations) [2019-10-11T08:34:04Z] <moritzm> remove kafka2001-2003 from debmonitor DB (T235125)

Volans added a subscriber: Volans.

Re-opening as the DNS name of the interfaces attached to those hosts have not been modified in Netbox.
Things like:

IP address: 10.193.1.23/16	
Parent: logstash2020
DNS name: kafka2001.mgmt.codfw.wmnet