Page MenuHomePhabricator

Bare metal cloud: management interfaces
Closed, ResolvedPublic0 Estimated Story Points

Description

As part of the long term goal of automating the management of bare metal, this Q goal will focus on the management interfaces:

  • Import existing management interfaces IPs into Netbox
  • Automate the assignment of new host's management interface IP
  • Automate the generation of management interface DNS records

Event Timeline

Volans triaged this task as Medium priority.Jul 18 2019, 9:31 AM
Volans created this task.

Change 532456 had a related patch set uploaded (by CRusnov; owner: CRusnov):
[operations/dns@master] Separate mgmt interface addresses into appropriately included files

https://gerrit.wikimedia.org/r/532456

For what it's worth the latest import of up-to-date dns repository, with the current state of things (dry-run):

2019-12-23 06:59:11,139 [INFO] [DRY RUN] would create mgmt interface on an-conf1001
2019-12-23 06:59:11,139 [INFO] [DRY RUN] would create management ip for an-conf1001
2019-12-23 06:59:11,224 [INFO] [DRY RUN] would create mgmt interface on an-conf1002
2019-12-23 06:59:11,224 [INFO] [DRY RUN] would create management ip for an-conf1002
2019-12-23 06:59:11,311 [INFO] [DRY RUN] would create mgmt interface on an-conf1003
2019-12-23 06:59:11,312 [INFO] [DRY RUN] would create management ip for an-conf1003
2019-12-23 06:59:20,966 [WARNING] No information for host atlas-codfw         
2019-12-23 06:59:20,966 [WARNING] No information for host atlas-eqiad         
2019-12-23 06:59:20,966 [WARNING] No information for host atlas-eqsin         
2019-12-23 06:59:20,966 [WARNING] No information for host atlas-esams         
2019-12-23 06:59:20,966 [WARNING] No information for host atlas-ulsfo         
2019-12-23 06:59:23,126 [INFO] [DRY RUN] would create management ip for bast3004
2019-12-23 06:59:24,296 [INFO] [DRY RUN] would create mgmt interface on censorship1001
2019-12-23 06:59:24,296 [INFO] [DRY RUN] would create management ip for censorship1001
2019-12-23 06:59:25,922 [INFO] [DRY RUN] would create mgmt interface on cloudcephmon1001
2019-12-23 06:59:25,922 [INFO] [DRY RUN] would create management ip for cloudcephmon1001
2019-12-23 06:59:26,011 [INFO] [DRY RUN] would create mgmt interface on cloudcephmon1002
2019-12-23 06:59:26,011 [INFO] [DRY RUN] would create management ip for cloudcephmon1002
2019-12-23 06:59:26,091 [INFO] [DRY RUN] would create mgmt interface on cloudcephmon1003
2019-12-23 06:59:26,091 [INFO] [DRY RUN] would create management ip for cloudcephmon1003
2019-12-23 06:59:26,183 [INFO] [DRY RUN] would create mgmt interface on cloudcephosd1001
019-12-23 06:59:26,183 [INFO] [DRY RUN] would create management ip for cloudcephosd1001                                                                                     
2019-12-23 06:59:26,278 [INFO] [DRY RUN] would create mgmt interface on cloudcephosd1002                                                                                     
2019-12-23 06:59:26,278 [INFO] [DRY RUN] would create management ip for cloudcephosd1002                                                                                     
2019-12-23 06:59:26,372 [INFO] [DRY RUN] would create mgmt interface on cloudcephosd1003                                                                                     
2019-12-23 06:59:26,372 [INFO] [DRY RUN] would create management ip for cloudcephosd1003                                                                                     
2019-12-23 06:59:29,109 [WARNING] No information for host cloudelastic1005
2019-12-23 06:59:29,109 [WARNING] No information for host cloudelastic1006
2019-12-23 06:59:41,445 [WARNING] No information for host cloudvirt-wdqs1001
2019-12-23 06:59:41,445 [WARNING] No information for host cloudvirt-wdqs1002
2019-12-23 06:59:41,445 [WARNING] No information for host cloudvirt-wdqs1003
2019-12-23 06:59:58,059 [INFO] [DRY RUN] would create management ip for cp3050
2019-12-23 06:59:58,272 [INFO] [DRY RUN] would create management ip for cp3051
2019-12-23 06:59:58,513 [INFO] [DRY RUN] would create management ip for cp3052
2019-12-23 06:59:58,768 [INFO] [DRY RUN] would create management ip for cp3053
2019-12-23 06:59:58,991 [INFO] [DRY RUN] would create management ip for cp3054
2019-12-23 06:59:59,219 [INFO] [DRY RUN] would create management ip for cp3055
2019-12-23 06:59:59,500 [INFO] [DRY RUN] would create management ip for cp3057
2019-12-23 06:59:59,781 [INFO] [DRY RUN] would create management ip for cp3058
2019-12-23 07:00:00,058 [INFO] [DRY RUN] would create management ip for cp3059     
2019-12-23 07:00:00,334 [INFO] [DRY RUN] would create management ip for cp3060     
2019-12-23 07:00:00,581 [INFO] [DRY RUN] would create management ip for cp3061     
2019-12-23 07:00:00,829 [INFO] [DRY RUN] would create management ip for cp3062     
2019-12-23 07:00:01,061 [INFO] [DRY RUN] would create management ip for cp3063     
2019-12-23 07:00:01,301 [INFO] [DRY RUN] would create management ip for cp3064     
2019-12-23 07:00:01,582 [INFO] [DRY RUN] would create management ip for cp3065
2019-12-23 07:00:46,696 [INFO] [DRY RUN] would create management ip for db2132
2019-12-23 07:00:46,917 [INFO] [DRY RUN] would create management ip for db2133
2019-12-23 07:00:47,158 [INFO] [DRY RUN] would create management ip for db2134
2019-12-23 07:00:47,362 [INFO] [DRY RUN] would create management ip for db2135
2019-12-23 07:00:56,752 [INFO] [DRY RUN] would create management ip for dns3001 
2019-12-23 07:00:56,972 [INFO] [DRY RUN] would create management ip for dns3002       
2019-12-23 07:01:00,120 [INFO] [DRY RUN] would create mgmt interface on dumpsdata1003 
2019-12-23 07:01:00,121 [INFO] [DRY RUN] would create management ip for dumpsdata1003   
2019-12-23 07:01:05,484 [INFO] [DRY RUN] would create mgmt interface on elastic1053     
2019-12-23 07:01:05,484 [INFO] [DRY RUN] would create management ip for elastic1053     
2019-12-23 07:01:05,556 [INFO] [DRY RUN] would create mgmt interface on elastic1054     
2019-12-23 07:01:05,556 [INFO] [DRY RUN] would create management ip for elastic1054     
2019-12-23 07:01:05,629 [INFO] [DRY RUN] would create mgmt interface on elastic1055     
2019-12-23 07:01:05,629 [INFO] [DRY RUN] would create management ip for elastic1055     
2019-12-23 07:01:05,698 [INFO] [DRY RUN] would create mgmt interface on elastic1056                                                                                          
2019-12-23 07:01:05,699 [INFO] [DRY RUN] would create management ip for elastic1056                                                                                          
2019-12-23 07:01:05,767 [INFO] [DRY RUN] would create mgmt interface on elastic1057                                                                                          
2019-12-23 07:01:05,767 [INFO] [DRY RUN] would create management ip for elastic1057                                                                                          
2019-12-23 07:01:05,852 [INFO] [DRY RUN] would create mgmt interface on elastic1058                                                                                          
2019-12-23 07:01:05,853 [INFO] [DRY RUN] would create management ip for elastic1058
2019-12-23 07:01:05,938 [INFO] [DRY RUN] would create mgmt interface on elastic1059
2019-12-23 07:01:05,938 [INFO] [DRY RUN] would create management ip for elastic1059
2019-12-23 07:01:06,014 [INFO] [DRY RUN] would create mgmt interface on elastic1060
2019-12-23 07:01:06,014 [INFO] [DRY RUN] would create management ip for elastic1060
2019-12-23 07:01:06,092 [INFO] [DRY RUN] would create mgmt interface on elastic1061
2019-12-23 07:01:06,092 [INFO] [DRY RUN] would create management ip for elastic1061
2019-12-23 07:01:06,164 [INFO] [DRY RUN] would create mgmt interface on elastic1062
2019-12-23 07:01:06,164 [INFO] [DRY RUN] would create management ip for elastic1062
2019-12-23 07:01:06,245 [INFO] [DRY RUN] would create mgmt interface on elastic1063
2019-12-23 07:01:06,245 [INFO] [DRY RUN] would create management ip for elastic1063
2019-12-23 07:01:06,318 [INFO] [DRY RUN] would create mgmt interface on elastic1064
2019-12-23 07:01:06,319 [INFO] [DRY RUN] would create management ip for elastic1064
2019-12-23 07:01:06,401 [INFO] [DRY RUN] would create mgmt interface on elastic1065
2019-12-23 07:01:06,402 [INFO] [DRY RUN] would create management ip for elastic1065
2019-12-23 07:01:06,482 [INFO] [DRY RUN] would create mgmt interface on elastic1066
2019-12-23 07:01:06,482 [INFO] [DRY RUN] would create management ip for elastic1066
2019-12-23 07:01:06,551 [INFO] [DRY RUN] would create mgmt interface on elastic1067
2019-12-23 07:01:06,551 [INFO] [DRY RUN] would create management ip for elastic1067
2019-12-23 07:01:16,466 [WARNING] No information for host es1020              
2019-12-23 07:01:16,466 [WARNING] No information for host es1021              
2019-12-23 07:01:16,466 [WARNING] No information for host es1022              
2019-12-23 07:01:16,466 [WARNING] No information for host es1023              
2019-12-23 07:01:16,466 [WARNING] No information for host es1024              
2019-12-23 07:01:16,466 [WARNING] No information for host es1025                
2019-12-23 07:01:21,888 [INFO] [DRY RUN] would create mgmt interface on frban1001     
2019-12-23 07:01:21,888 [INFO] [DRY RUN] would create management ip for frban1001     
2019-12-23 07:01:21,971 [INFO] [DRY RUN] would create mgmt interface on frban2001       
2019-12-23 07:01:21,971 [INFO] [DRY RUN] would create management ip for frban2001       
2019-12-23 07:01:23,534 [INFO] [DRY RUN] would create management ip for frdb1003        
2019-12-23 07:01:24,034 [INFO] [DRY RUN] would create management ip for frdb2002        
2019-12-23 07:01:25,190 [INFO] [DRY RUN] would create mgmt interface on frnetmon1001    
2019-12-23 07:01:25,190 [INFO] [DRY RUN] would create management ip for frnetmon1001    
2019-12-23 07:01:26,504 [INFO] [DRY RUN] would create mgmt interface on frqueue2001
2019-12-23 07:01:26,504 [INFO] [DRY RUN] would create management ip for frqueue2001
2019-12-23 07:01:28,849 [INFO] [DRY RUN] would create mgmt interface on ganeti1009
2019-12-23 07:01:28,849 [INFO] [DRY RUN] would create management ip for ganeti1009
2019-12-23 07:01:28,933 [INFO] [DRY RUN] would create mgmt interface on ganeti1010
2019-12-23 07:01:28,933 [INFO] [DRY RUN] would create management ip for ganeti1010
2019-12-23 07:01:29,009 [INFO] [DRY RUN] would create mgmt interface on ganeti1011
2019-12-23 07:01:29,009 [INFO] [DRY RUN] would create management ip for ganeti1011
2019-12-23 07:01:29,082 [INFO] [DRY RUN] would create mgmt interface on ganeti1012
2019-12-23 07:01:29,082 [INFO] [DRY RUN] would create management ip for ganeti1012
2019-12-23 07:01:29,168 [INFO] [DRY RUN] would create mgmt interface on ganeti1013
2019-12-23 07:01:29,168 [INFO] [DRY RUN] would create management ip for ganeti1013
2019-12-23 07:01:29,246 [INFO] [DRY RUN] would create mgmt interface on ganeti1014
2019-12-23 07:01:29,246 [INFO] [DRY RUN] would create management ip for ganeti1014
2019-12-23 07:01:29,326 [INFO] [DRY RUN] would create mgmt interface on ganeti1015
2019-12-23 07:01:29,326 [INFO] [DRY RUN] would create management ip for ganeti1015
2019-12-23 07:01:29,418 [INFO] [DRY RUN] would create mgmt interface on ganeti1016
2019-12-23 07:01:29,418 [INFO] [DRY RUN] would create management ip for ganeti1016
2019-12-23 07:01:29,487 [INFO] [DRY RUN] would create mgmt interface on ganeti1017
2019-12-23 07:01:29,487 [INFO] [DRY RUN] would create management ip for ganeti1017
2019-12-23 07:01:29,563 [INFO] [DRY RUN] would create mgmt interface on ganeti1018
2019-12-23 07:01:29,564 [INFO] [DRY RUN] would create management ip for ganeti1018
2019-12-23 07:01:29,640 [INFO] [DRY RUN] would create mgmt interface on ganeti1019
2019-12-23 07:01:29,640 [INFO] [DRY RUN] would create management ip for ganeti1019
2019-12-23 07:01:29,717 [INFO] [DRY RUN] would create mgmt interface on ganeti1020
2019-12-23 07:01:29,717 [INFO] [DRY RUN] would create management ip for ganeti1020
2019-12-23 07:01:29,797 [INFO] [DRY RUN] would create mgmt interface on ganeti1021
2019-12-23 07:01:29,797 [INFO] [DRY RUN] would create management ip for ganeti1021
2019-12-23 07:01:29,870 [INFO] [DRY RUN] would create mgmt interface on ganeti1022
2019-12-23 07:01:29,870 [INFO] [DRY RUN] would create management ip for ganeti1022
2019-12-23 07:01:34,869 [INFO] [DRY RUN] would create management ip for ganeti3001
2019-12-23 07:01:35,106 [INFO] [DRY RUN] would create management ip for ganeti3002
2019-12-23 07:01:35,346 [INFO] [DRY RUN] would create management ip for ganeti3003
2019-12-23 07:01:36,994 [INFO] [DRY RUN] would create mgmt interface on gerrit1001
2019-12-23 07:01:36,994 [INFO] [DRY RUN] would create management ip for gerrit1001
2019-12-23 07:01:43,871 [INFO] [DRY RUN] would create mgmt interface on krb1001
2019-12-23 07:01:43,871 [INFO] [DRY RUN] would create management ip for krb1001
2019-12-23 07:01:43,952 [INFO] [DRY RUN] would create mgmt interface on krb2001
2019-12-23 07:01:43,952 [INFO] [DRY RUN] would create management ip for krb2001     
2019-12-23 07:01:52,784 [WARNING] No information for host logstash1026             
2019-12-23 07:01:52,784 [WARNING] No information for host logstash1027             
2019-12-23 07:01:52,784 [WARNING] No information for host logstash1028            
2019-12-23 07:01:52,784 [WARNING] No information for host logstash1029            
2019-12-23 07:01:58,114 [INFO] [DRY RUN] would create management ip for lvs3005   
2019-12-23 07:01:58,335 [INFO] [DRY RUN] would create management ip for lvs3006   
2019-12-23 07:01:58,546 [INFO] [DRY RUN] would create management ip for lvs3007   
2019-12-23 07:02:22,613 [INFO] [DRY RUN] would create mgmt interface on ms-be1051 
2019-12-23 07:02:22,613 [INFO] [DRY RUN] would create management ip for ms-be1051 
2019-12-23 07:02:22,722 [INFO] [DRY RUN] would create mgmt interface on ms-be1052 
2019-12-23 07:02:22,722 [INFO] [DRY RUN] would create management ip for ms-be1052 
2019-12-23 07:02:22,823 [INFO] [DRY RUN] would create mgmt interface on ms-be1053 
2019-12-23 07:02:22,824 [INFO] [DRY RUN] would create management ip for ms-be1053 
2019-12-23 07:02:22,915 [INFO] [DRY RUN] would create mgmt interface on ms-be1054 
2019-12-23 07:02:22,915 [INFO] [DRY RUN] would create management ip for ms-be1054 
2019-12-23 07:02:23,004 [INFO] [DRY RUN] would create mgmt interface on ms-be1055 
2019-12-23 07:02:23,004 [INFO] [DRY RUN] would create management ip for ms-be1055 
2019-12-23 07:02:23,100 [INFO] [DRY RUN] would create mgmt interface on ms-be1056 
2019-12-23 07:02:23,100 [INFO] [DRY RUN] would create management ip for ms-be1056 
2019-12-23 07:02:23,199 [INFO] [DRY RUN] would create mgmt interface on ms-be1057 
2019-12-23 07:02:23,200 [INFO] [DRY RUN] would create management ip for ms-be1057 
2019-12-23 07:02:23,301 [INFO] [DRY RUN] would create mgmt interface on ms-be1058 
2019-12-23 07:02:23,301 [INFO] [DRY RUN] would create management ip for ms-be1058 
2019-12-23 07:02:23,400 [INFO] [DRY RUN] would create mgmt interface on ms-be1059 
2019-12-23 07:02:23,400 [INFO] [DRY RUN] would create management ip for ms-be1059 
2019-12-23 07:02:33,514 [INFO] [DRY RUN] would create mgmt interface on ms-be2051 
2019-12-23 07:02:33,514 [INFO] [DRY RUN] would create management ip for ms-be2051 
2019-12-23 07:02:33,608 [INFO] [DRY RUN] would create mgmt interface on ms-be2052 
2019-12-23 07:02:33,608 [INFO] [DRY RUN] would create management ip for ms-be2052 
2019-12-23 07:02:33,690 [INFO] [DRY RUN] would create mgmt interface on ms-be2053 
2019-12-23 07:02:33,690 [INFO] [DRY RUN] would create management ip for ms-be2053 
2019-12-23 07:02:33,773 [INFO] [DRY RUN] would create mgmt interface on ms-be2054 
2019-12-23 07:02:33,773 [INFO] [DRY RUN] would create management ip for ms-be2054 
2019-12-23 07:02:33,843 [INFO] [DRY RUN] would create mgmt interface on ms-be2055 
2019-12-23 07:02:33,843 [INFO] [DRY RUN] would create management ip for ms-be2055 
2019-12-23 07:02:33,916 [INFO] [DRY RUN] would create mgmt interface on ms-be2056
2019-12-23 07:02:33,916 [INFO] [DRY RUN] would create management ip for ms-be2056
2019-12-23 07:03:07,827 [WARNING] No information for host mw1349
2019-12-23 07:03:07,827 [WARNING] No information for host mw1350
2019-12-23 07:03:07,827 [WARNING] No information for host mw1351
2019-12-23 07:03:07,827 [WARNING] No information for host mw1352
2019-12-23 07:03:07,827 [WARNING] No information for host mw1353
2019-12-23 07:03:07,827 [WARNING] No information for host mw1354
2019-12-23 07:03:07,827 [WARNING] No information for host mw1355
2019-12-23 07:03:07,827 [WARNING] No information for host mw1356
2019-12-23 07:03:07,827 [WARNING] No information for host mw1357
2019-12-23 07:03:07,827 [WARNING] No information for host mw1358
2019-12-23 07:03:07,827 [WARNING] No information for host mw1359
2019-12-23 07:03:07,827 [WARNING] No information for host mw1360
2019-12-23 07:03:07,827 [WARNING] No information for host mw1361
2019-12-23 07:03:07,827 [WARNING] No information for host mw1362
2019-12-23 07:03:07,827 [WARNING] No information for host mw1363
2019-12-23 07:03:07,827 [WARNING] No information for host mw1364
2019-12-23 07:03:07,827 [WARNING] No information for host mw1365
2019-12-23 07:03:07,827 [WARNING] No information for host mw1366
2019-12-23 07:03:07,827 [WARNING] No information for host mw1367
2019-12-23 07:03:07,827 [WARNING] No information for host mw1368
2019-12-23 07:03:07,827 [WARNING] No information for host mw1369
2019-12-23 07:03:07,827 [WARNING] No information for host mw1370
2019-12-23 07:03:07,827 [WARNING] No information for host mw1371
2019-12-23 07:03:07,827 [WARNING] No information for host mw1372
2019-12-23 07:03:07,827 [WARNING] No information for host mw1373
2019-12-23 07:03:07,827 [WARNING] No information for host mw1374
2019-12-23 07:03:07,828 [WARNING] No information for host mw1375
2019-12-23 07:03:07,828 [WARNING] No information for host mw1376
2019-12-23 07:03:07,828 [WARNING] No information for host mw1377
2019-12-23 07:03:07,828 [WARNING] No information for host mw1378
2019-12-23 07:03:07,828 [WARNING] No information for host mw1379
2019-12-23 07:03:07,828 [WARNING] No information for host mw1380
2019-12-23 07:03:07,828 [WARNING] No information for host mw1381
2019-12-23 07:03:07,828 [WARNING] No information for host mw1382
2019-12-23 07:03:07,828 [WARNING] No information for host mw1383
2019-12-23 07:03:07,828 [WARNING] No information for host mw1384
2019-12-23 07:04:16,958 [WARNING] No information for host samarium
2019-12-23 07:04:29,100 [WARNING] No information for host wdqs1011
2019-12-23 07:04:29,100 [WARNING] No information for host wdqs1012
2019-12-23 07:04:29,100 [WARNING] No information for host wdqs1013

The "No information for host" means that it couldn't find that host in the DNS zone file 'wmnet' in one of the 'mgmt' origins. It is possible this is a flaw in the script or some other explanation.

@crusnov thanks for the dry-run run, here my comments:

  • atlas-* those are the RIPE Atlas anchors, we don't have mgmt for them, so they should be skipped for mgmt. It's easy because they are a different tenant (RIPE NCC) in Netbox. For real IPs it would be a bit trickier as we have them as ripe-atlas-*.wikimedia.org..
  • Some hosts are in planned state but not yet in the DNS, that's currently a possible situation (see https://wikitech.wikimedia.org/wiki/Server_Lifecycle#States ).
  • cp3056 is not in the list but should be. Is the script excluding devices in state failed in Netbox? Those are perfectly valid hosts that should have their records set.

All the would create lines are correct, of course the log doesn't tell us which IP they will add. If you can have a more detailed debug log just do some random check to ensure all is still working as expected like when we've run the script the first time.
That said ok for me to run it for real adding all the missing records.

Okay here's an updated log (I took out the No Information lines)

2019-12-23 18:10:43,464 [INFO] [DRY RUN] would create mgmt interface on analytics1068
2019-12-23 18:10:43,465 [INFO] [DRY RUN] would create management ip 10.65.4.106/16 for analytics1068
2019-12-23 18:10:46,046 [INFO] [DRY RUN] would create mgmt interface on an-conf1001
2019-12-23 18:10:46,046 [INFO] [DRY RUN] would create management ip 10.65.5.118/16 for an-conf1001
2019-12-23 18:10:46,114 [INFO] [DRY RUN] would create mgmt interface on an-conf1002
2019-12-23 18:10:46,114 [INFO] [DRY RUN] would create management ip 10.65.5.119/16 for an-conf1002
2019-12-23 18:10:46,186 [INFO] [DRY RUN] would create mgmt interface on an-conf1003
2019-12-23 18:10:46,186 [INFO] [DRY RUN] would create management ip 10.65.5.120/16 for an-conf1003
2019-12-23 18:10:55,574 [INFO] [DRY RUN] would create management ip 10.21.0.191/24 for bast3004
2019-12-23 18:10:56,756 [INFO] [DRY RUN] would create mgmt interface on censorship1001
2019-12-23 18:10:56,757 [INFO] [DRY RUN] would create management ip 10.65.3.100/16 for censorship1001
2019-12-23 18:10:58,586 [INFO] [DRY RUN] would create mgmt interface on cloudcephmon1001
2019-12-23 18:10:58,586 [INFO] [DRY RUN] would create management ip 10.65.3.125/16 for cloudcephmon1001
2019-12-23 18:10:58,658 [INFO] [DRY RUN] would create mgmt interface on cloudcephmon1002
2019-12-23 18:10:58,658 [INFO] [DRY RUN] would create management ip 10.65.3.126/16 for cloudcephmon1002
2019-12-23 18:10:58,736 [INFO] [DRY RUN] would create mgmt interface on cloudcephmon1003
2019-12-23 18:10:58,736 [INFO] [DRY RUN] would create management ip 10.65.3.127/16 for cloudcephmon1003
2019-12-23 18:10:58,815 [INFO] [DRY RUN] would create mgmt interface on cloudcephosd1001
2019-12-23 18:10:58,815 [INFO] [DRY RUN] would create management ip 10.65.2.177/16 for cloudcephosd1001
2019-12-23 18:10:58,896 [INFO] [DRY RUN] would create mgmt interface on cloudcephosd1002
2019-12-23 18:10:58,896 [INFO] [DRY RUN] would create management ip 10.65.2.178/16 for cloudcephosd1002
2019-12-23 18:10:58,962 [INFO] [DRY RUN] would create mgmt interface on cloudcephosd1003
2019-12-23 18:10:58,962 [INFO] [DRY RUN] would create management ip 10.65.2.179/16 for cloudcephosd1003
2019-12-23 18:11:27,974 [INFO] [DRY RUN] would create management ip 10.21.0.174/24 for cp3050
2019-12-23 18:11:28,187 [INFO] [DRY RUN] would create management ip 10.21.0.175/24 for cp3051
2019-12-23 18:11:28,411 [INFO] [DRY RUN] would create management ip 10.21.0.176/24 for cp3052
2019-12-23 18:11:28,633 [INFO] [DRY RUN] would create management ip 10.21.0.177/24 for cp3053
2019-12-23 18:11:28,897 [INFO] [DRY RUN] would create management ip 10.21.0.178/24 for cp3054
2019-12-23 18:11:29,115 [INFO] [DRY RUN] would create management ip 10.21.0.183/24 for cp3055
2019-12-23 18:11:29,335 [INFO] [DRY RUN] would create management ip 10.21.0.184/24 for cp3056
2019-12-23 18:11:29,531 [INFO] [DRY RUN] would create management ip 10.21.0.185/24 for cp3057
2019-12-23 18:11:29,708 [INFO] [DRY RUN] would create management ip 10.21.0.186/24 for cp3058
2019-12-23 18:11:29,923 [INFO] [DRY RUN] would create management ip 10.21.0.187/24 for cp3059
2019-12-23 18:11:30,129 [INFO] [DRY RUN] would create management ip 10.21.0.188/24 for cp3060
2019-12-23 18:11:30,319 [INFO] [DRY RUN] would create management ip 10.21.0.192/24 for cp3061
2019-12-23 18:11:30,494 [INFO] [DRY RUN] would create management ip 10.21.0.193/24 for cp3062
2019-12-23 18:11:30,694 [INFO] [DRY RUN] would create management ip 10.21.0.194/24 for cp3063
2019-12-23 18:11:30,890 [INFO] [DRY RUN] would create management ip 10.21.0.195/24 for cp3064
2019-12-23 18:11:31,076 [INFO] [DRY RUN] would create management ip 10.21.0.196/24 for cp3065
2019-12-23 18:12:06,457 [INFO] [DRY RUN] would create management ip 10.193.1.83/16 for db2132
2019-12-23 18:12:06,645 [INFO] [DRY RUN] would create management ip 10.193.1.84/16 for db2133
2019-12-23 18:12:06,843 [INFO] [DRY RUN] would create management ip 10.193.1.85/16 for db2134
2019-12-23 18:12:07,039 [INFO] [DRY RUN] would create management ip 10.193.1.86/16 for db2135
2019-12-23 18:12:14,384 [INFO] [DRY RUN] would create management ip 10.21.0.173/24 for dns3001
2019-12-23 18:12:14,559 [INFO] [DRY RUN] would create management ip 10.21.0.182/24 for dns3002
2019-12-23 18:12:17,342 [INFO] [DRY RUN] would create mgmt interface on dumpsdata1003
2019-12-23 18:12:17,342 [INFO] [DRY RUN] would create management ip 10.65.5.182/16 for dumpsdata1003
2019-12-23 18:12:21,881 [INFO] [DRY RUN] would create mgmt interface on elastic1053
2019-12-23 18:12:21,881 [INFO] [DRY RUN] would create management ip 10.65.7.94/16 for elastic1053
2019-12-23 18:12:21,958 [INFO] [DRY RUN] would create mgmt interface on elastic1054
2019-12-23 18:12:21,958 [INFO] [DRY RUN] would create management ip 10.65.7.95/16 for elastic1054
2019-12-23 18:12:22,036 [INFO] [DRY RUN] would create mgmt interface on elastic1055
2019-12-23 18:12:22,037 [INFO] [DRY RUN] would create management ip 10.65.7.96/16 for elastic1055
2019-12-23 18:12:22,103 [INFO] [DRY RUN] would create mgmt interface on elastic1056
2019-12-23 18:12:22,103 [INFO] [DRY RUN] would create management ip 10.65.7.97/16 for elastic1056
2019-12-23 18:12:22,186 [INFO] [DRY RUN] would create mgmt interface on elastic1057
2019-12-23 18:12:22,186 [INFO] [DRY RUN] would create management ip 10.65.7.98/16 for elastic1057
2019-12-23 18:12:22,258 [INFO] [DRY RUN] would create mgmt interface on elastic1058
2019-12-23 18:12:22,258 [INFO] [DRY RUN] would create management ip 10.65.7.99/16 for elastic1058
2019-12-23 18:12:22,326 [INFO] [DRY RUN] would create mgmt interface on elastic1059
2019-12-23 18:12:22,326 [INFO] [DRY RUN] would create management ip 10.65.7.100/16 for elastic1059
2019-12-23 18:12:22,398 [INFO] [DRY RUN] would create mgmt interface on elastic1060
2019-12-23 18:12:22,398 [INFO] [DRY RUN] would create management ip 10.65.7.101/16 for elastic1060
2019-12-23 18:12:22,467 [INFO] [DRY RUN] would create mgmt interface on elastic1061
2019-12-23 18:12:22,468 [INFO] [DRY RUN] would create management ip 10.65.7.102/16 for elastic1061
2019-12-23 18:12:22,535 [INFO] [DRY RUN] would create mgmt interface on elastic1062
2019-12-23 18:12:22,535 [INFO] [DRY RUN] would create management ip 10.65.7.103/16 for elastic1062
2019-12-23 18:12:22,599 [INFO] [DRY RUN] would create mgmt interface on elastic1063
2019-12-23 18:12:22,599 [INFO] [DRY RUN] would create management ip 10.65.7.104/16 for elastic1063
2019-12-23 18:12:22,675 [INFO] [DRY RUN] would create mgmt interface on elastic1064
2019-12-23 18:12:22,675 [INFO] [DRY RUN] would create management ip 10.65.7.105/16 for elastic1064
2019-12-23 18:12:22,745 [INFO] [DRY RUN] would create mgmt interface on elastic1065
2019-12-23 18:12:22,745 [INFO] [DRY RUN] would create management ip 10.65.7.106/16 for elastic1065
2019-12-23 18:12:22,817 [INFO] [DRY RUN] would create mgmt interface on elastic1066
2019-12-23 18:12:22,817 [INFO] [DRY RUN] would create management ip 10.65.7.107/16 for elastic1066
2019-12-23 18:12:22,898 [INFO] [DRY RUN] would create mgmt interface on elastic1067
2019-12-23 18:12:22,898 [INFO] [DRY RUN] would create management ip 10.65.7.108/16 for elastic1067
2019-12-23 18:12:37,449 [INFO] [DRY RUN] would create mgmt interface on frban1001
2019-12-23 18:12:37,450 [INFO] [DRY RUN] would create management ip 10.64.40.212/16 for frban1001
2019-12-23 18:12:37,538 [INFO] [DRY RUN] would create mgmt interface on frban2001
2019-12-23 18:12:37,539 [INFO] [DRY RUN] would create management ip 10.195.0.117/24 for frban2001
2019-12-23 18:12:38,910 [INFO] [DRY RUN] would create management ip 10.64.40.219/26 for frdb1003
2019-12-23 18:12:39,307 [INFO] [DRY RUN] would create management ip 10.195.0.105/24 for frdb2002
2019-12-23 18:12:40,695 [INFO] [DRY RUN] would create mgmt interface on frnetmon1001
2019-12-23 18:12:40,695 [INFO] [DRY RUN] would create management ip 10.64.40.207/16 for frnetmon1001
2019-12-23 18:12:42,040 [INFO] [DRY RUN] would create mgmt interface on frqueue2001
2019-12-23 18:12:42,041 [INFO] [DRY RUN] would create management ip 10.195.0.116/24 for frqueue2001
2019-12-23 18:12:44,101 [INFO] [DRY RUN] would create mgmt interface on ganeti1009
2019-12-23 18:12:44,101 [INFO] [DRY RUN] would create management ip 10.65.5.104/16 for ganeti1009
2019-12-23 18:12:44,169 [INFO] [DRY RUN] would create mgmt interface on ganeti1010
2019-12-23 18:12:44,169 [INFO] [DRY RUN] would create management ip 10.65.5.105/16 for ganeti1010
2019-12-23 18:12:44,241 [INFO] [DRY RUN] would create mgmt interface on ganeti1011
2019-12-23 18:12:44,241 [INFO] [DRY RUN] would create management ip 10.65.5.106/16 for ganeti1011
2019-12-23 18:12:44,303 [INFO] [DRY RUN] would create mgmt interface on ganeti1012
2019-12-23 18:12:44,304 [INFO] [DRY RUN] would create management ip 10.65.5.107/16 for ganeti1012
2019-12-23 18:12:44,363 [INFO] [DRY RUN] would create mgmt interface on ganeti1013
2019-12-23 18:12:44,363 [INFO] [DRY RUN] would create management ip 10.65.5.108/16 for ganeti1013
2019-12-23 18:12:44,429 [INFO] [DRY RUN] would create mgmt interface on ganeti1014
2019-12-23 18:12:44,429 [INFO] [DRY RUN] would create management ip 10.65.5.109/16 for ganeti1014
2019-12-23 18:12:44,495 [INFO] [DRY RUN] would create mgmt interface on ganeti1015
2019-12-23 18:12:44,495 [INFO] [DRY RUN] would create management ip 10.65.5.110/16 for ganeti1015
2019-12-23 18:12:44,567 [INFO] [DRY RUN] would create mgmt interface on ganeti1016
2019-12-23 18:12:44,567 [INFO] [DRY RUN] would create management ip 10.65.5.111/16 for ganeti1016
2019-12-23 18:12:44,638 [INFO] [DRY RUN] would create mgmt interface on ganeti1017
2019-12-23 18:12:44,638 [INFO] [DRY RUN] would create management ip 10.65.5.112/16 for ganeti1017
2019-12-23 18:12:44,718 [INFO] [DRY RUN] would create mgmt interface on ganeti1018
2019-12-23 18:12:44,718 [INFO] [DRY RUN] would create management ip 10.65.5.113/16 for ganeti1018
2019-12-23 18:12:44,802 [INFO] [DRY RUN] would create mgmt interface on ganeti1019
2019-12-23 18:12:44,803 [INFO] [DRY RUN] would create management ip 10.65.5.114/16 for ganeti1019
2019-12-23 18:12:44,890 [INFO] [DRY RUN] would create mgmt interface on ganeti1020
2019-12-23 18:12:44,890 [INFO] [DRY RUN] would create management ip 10.65.5.115/16 for ganeti1020
2019-12-23 18:12:44,971 [INFO] [DRY RUN] would create mgmt interface on ganeti1021
2019-12-23 18:12:44,971 [INFO] [DRY RUN] would create management ip 10.65.5.116/16 for ganeti1021
2019-12-23 18:12:45,051 [INFO] [DRY RUN] would create mgmt interface on ganeti1022
2019-12-23 18:12:45,052 [INFO] [DRY RUN] would create management ip 10.65.5.117/16 for ganeti1022
2019-12-23 18:12:49,258 [INFO] [DRY RUN] would create management ip 10.21.0.172/24 for ganeti3001
2019-12-23 18:12:49,452 [INFO] [DRY RUN] would create management ip 10.21.0.181/24 for ganeti3002
2019-12-23 18:12:49,777 [INFO] [DRY RUN] would create management ip 10.21.0.190/24 for ganeti3003
2019-12-23 18:12:51,031 [INFO] [DRY RUN] would create mgmt interface on gerrit1001
2019-12-23 18:12:51,031 [INFO] [DRY RUN] would create management ip 10.65.3.102/16 for gerrit1001
2019-12-23 18:12:56,708 [INFO] [DRY RUN] would create mgmt interface on krb1001
2019-12-23 18:12:56,709 [INFO] [DRY RUN] would create management ip 10.65.3.99/16 for krb1001
2019-12-23 18:12:56,780 [INFO] [DRY RUN] would create mgmt interface on krb2001
2019-12-23 18:12:51,031 [INFO] [DRY RUN] would create mgmt interface on gerrit1001
2019-12-23 18:12:51,031 [INFO] [DRY RUN] would create management ip 10.65.3.102/16 for gerrit1001
2019-12-23 18:12:56,708 [INFO] [DRY RUN] would create mgmt interface on krb1001
2019-12-23 18:12:56,709 [INFO] [DRY RUN] would create management ip 10.65.3.99/16 for krb1001
2019-12-23 18:12:56,780 [INFO] [DRY RUN] would create mgmt interface on krb2001
2019-12-23 18:12:56,780 [INFO] [DRY RUN] would create management ip 10.193.3.33/16 for krb2001
2019-12-23 18:13:08,659 [INFO] [DRY RUN] would create management ip 10.21.0.171/24 for lvs3005
2019-12-23 18:13:08,834 [INFO] [DRY RUN] would create management ip 10.21.0.180/24 for lvs3006
2019-12-23 18:13:09,010 [INFO] [DRY RUN] would create management ip 10.21.0.189/24 for lvs3007
2019-12-23 18:13:28,429 [INFO] [DRY RUN] would create mgmt interface on ms-be1051
2019-12-23 18:13:28,429 [INFO] [DRY RUN] would create management ip 10.65.5.10/16 for ms-be1051
2019-12-23 18:13:28,523 [INFO] [DRY RUN] would create mgmt interface on ms-be1052
2019-12-23 18:13:28,524 [INFO] [DRY RUN] would create management ip 10.65.5.11/16 for ms-be1052
2019-12-23 18:13:28,603 [INFO] [DRY RUN] would create mgmt interface on ms-be1053
2019-12-23 18:13:28,603 [INFO] [DRY RUN] would create management ip 10.65.5.12/16 for ms-be1053
2019-12-23 18:13:28,678 [INFO] [DRY RUN] would create mgmt interface on ms-be1054
2019-12-23 18:13:28,678 [INFO] [DRY RUN] would create management ip 10.65.5.13/16 for ms-be1054
2019-12-23 18:13:28,759 [INFO] [DRY RUN] would create mgmt interface on ms-be1055
2019-12-23 18:13:28,759 [INFO] [DRY RUN] would create management ip 10.65.5.14/16 for ms-be1055
2019-12-23 18:13:28,832 [INFO] [DRY RUN] would create mgmt interface on ms-be1056
2019-12-23 18:13:28,832 [INFO] [DRY RUN] would create management ip 10.65.5.15/16 for ms-be1056
2019-12-23 18:13:28,896 [INFO] [DRY RUN] would create mgmt interface on ms-be1057
2019-12-23 18:13:28,896 [INFO] [DRY RUN] would create management ip 10.65.5.16/16 for ms-be1057
2019-12-23 18:13:28,967 [INFO] [DRY RUN] would create mgmt interface on ms-be1058
2019-12-23 18:13:28,967 [INFO] [DRY RUN] would create management ip 10.65.5.17/16 for ms-be1058
2019-12-23 18:13:29,032 [INFO] [DRY RUN] would create mgmt interface on ms-be1059
2019-12-23 18:13:29,032 [INFO] [DRY RUN] would create management ip 10.65.5.18/16 for ms-be1059
2019-12-23 18:13:37,273 [INFO] [DRY RUN] would create mgmt interface on ms-be2051
2019-12-23 18:13:37,274 [INFO] [DRY RUN] would create management ip 10.193.3.72/16 for ms-be2051
2019-12-23 18:13:37,346 [INFO] [DRY RUN] would create mgmt interface on ms-be2052
2019-12-23 18:13:37,347 [INFO] [DRY RUN] would create management ip 10.193.3.73/16 for ms-be2052
2019-12-23 18:13:37,414 [INFO] [DRY RUN] would create mgmt interface on ms-be2053
2019-12-23 18:13:37,415 [INFO] [DRY RUN] would create management ip 10.193.3.74/16 for ms-be2053
2019-12-23 18:13:37,486 [INFO] [DRY RUN] would create mgmt interface on ms-be2054
2019-12-23 18:13:37,486 [INFO] [DRY RUN] would create management ip 10.193.3.75/16 for ms-be2054
2019-12-23 18:13:37,542 [INFO] [DRY RUN] would create mgmt interface on ms-be2055
2019-12-23 18:13:37,542 [INFO] [DRY RUN] would create management ip 10.193.3.76/16 for ms-be2055
2019-12-23 18:13:37,610 [INFO] [DRY RUN] would create mgmt interface on ms-be2056
2019-12-23 18:13:37,610 [INFO] [DRY RUN] would create management ip 10.193.3.77/16 for ms-be2056
2019-12-23 18:15:20,192 [INFO] [DRY RUN] would create mgmt interface on wmf7622
2019-12-23 18:15:20,192 [INFO] [DRY RUN] would create management ip 10.65.1.13/16 for wmf7622

Thanks, LGTM, feel free to proceed.

Some conflicts in importing DNS again because of offline devices. https://phabricator.wikimedia.org/P10013 is a list.

Mentioned in SAL (#wikimedia-operations) [2019-12-23T18:54:51Z] <chaomodus> Deleting mgmt IP addresses from Netbox that are connected to offline devices. T228387

After tidying, DNS has been reimported successfully.

Imported Decommissioning and Failed status DNS entries. See paste at: https://phabricator.wikimedia.org/P10026 for list.

Setup script to import PDUs too, here's what the results would be:
https://phabricator.wikimedia.org/P10027

LGTM, a couple bike-shed like comments:

Cool, I have executed the script which has imported them. I didn't change the interface type from 1G ones, so those are wrong, and tehy do have the mgmt_only bit set, so i'll just bulk fix that.

Alright I have adjusted PDUs to use 100M interface types (on all but esams) and unset the mgmt_only bit.

Change 532456 abandoned by CRusnov:
Separate mgmt interface addresses into appropriately included files

Reason:
This will be done in a piecemeal manner instead of all at once (plus this is outdated).

https://gerrit.wikimedia.org/r/532456

Volans updated the task description. (Show Details)

Resolving as the final steps are tracked in T233183