Eq: new Druid test VM for analytics
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	klausman
	Oct 29 2020, 10:51 AM

Description

Site/Location: EQ
Number of systems: 1
Service: an-test-druid1001.eqiad.wmnet
Networking Requirements: Analytics VLAN
Processor Requirements: 4 vcores
Memory: 8GB
Disks: 100G
Other Requirements:

Details

	Subject	Repo	Branch	Lines +/-
	install-server: Add DHCP entry for an-test-druid1001	operations/puppet	production	+6 -1

Customize query in gerrit

Event Timeline

klausman created this task.Oct 29 2020, 10:51 AM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptOct 29 2020, 10:51 AM

klausman claimed this task.Oct 29 2020, 10:51 AM

klausman renamed this task from Site: (QUANTITY) VM %request for SERVICE[S] to Eq: new Druid test VM for analytics.Oct 29 2020, 10:56 AM

root@cumin1001:~# cookbook sre.ganeti.makevm --vcpus 4 --memory 8 --disk 100  --network analytics eqiad_D an-test-druid1001.eqiad.wmnet
START - Cookbook sre.ganeti.makevm
Ready to create Ganeti VM an-test-druid1001.eqiad.wmnet in the ganeti01.svc.eqiad.wmnet cluster on row D with 4 vCPUs, 8GB of RAM, 100GB of disk in the analytics network.
Is this correct?
Type "done" to proceed
> done
Allocated IPv4 10.64.53.6/24
Set DNS name of IP 10.64.53.6/24 to an-test-druid1001.eqiad.wmnet
Allocated IPv6 2620:0:861:108:10:64:53:6/64 with DNS name an-test-druid1001.eqiad.wmnet
Generating the DNS records from Netbox data. It will take a couple of minutes.
2020-10-29 11:11:19,218 [INFO] Gathering devices, interfaces, addresses and prefixes from Netbox
2020-10-29 11:13:08,271 [INFO] Gathered 2182 devices from Netbox
2020-10-29 11:13:08,271 [INFO] Generating DNS records
2020-10-29 11:13:14,959 [INFO] Generated 12032 direct and reverse records (6016 each) in 26 direct zones and 168 reverse zones                                                                                    
2020-10-29 11:13:14,960 [INFO] Cloning /srv/netbox-exports/dns.git/ to /tmp/dns-c25pcHBldHM-jdruvzjh ...
2020-10-29 11:13:15,557 [INFO] Generating zonefile snippets to directory /tmp/dns-c25pcHBldHM-jdruvzjh
2020-10-29 11:13:16,180 [INFO] Committed changes: 1d04e945917ee0470dff1c49e67146287211b411
2020-10-29 11:13:16,202 [INFO] Validating generated data
2020-10-29 11:13:16,202 [INFO] Commit details: {'insertions': 4, 'deletions': 0, 'lines': 4, 'files': 3}
commit 1d04e945917ee0470dff1c49e67146287211b411
Author: generate-dns-snippets <noc@wikimedia.org>
Date:   Thu Oct 29 11:13:16 2020 +0000

    klausman@cumin1001: Created records for VM an-test-druid1001.eqiad.wmnet

diff --git a/53.64.10.in-addr.arpa b/53.64.10.in-addr.arpa
index a027e96..ca9eb1f 100644
--- a/53.64.10.in-addr.arpa
+++ b/53.64.10.in-addr.arpa
@@ -1,6 +1,7 @@
 1   1H IN PTR vrrp-gw-1023.eqiad.wmnet.
 2   1H IN PTR ae4-1023.cr1-eqiad.wikimedia.org.
 3   1H IN PTR ae4-1023.cr2-eqiad.wikimedia.org.
+6   1H IN PTR an-test-druid1001.eqiad.wmnet.
 21  1H IN PTR an-test-worker1003.eqiad.wmnet.
 22  1H IN PTR analytics1042.eqiad.wmnet.
 23  1H IN PTR analytics1043.eqiad.wmnet.
diff --git a/8.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa b/8.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa
index b2eb4ad..3d71f3b 100644
--- a/8.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa
+++ b/8.0.1.0.1.6.8.0.0.0.0.0.0.2.6.2.ip6.arpa
@@ -1,4 +1,5 @@
 1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0 1H IN PTR vrrp-gw-1023.eqiad.wmnet.
+6.0.0.0.3.5.0.0.4.6.0.0.0.1.0.0 1H IN PTR an-test-druid1001.eqiad.wmnet.
 1.2.0.0.3.5.0.0.4.6.0.0.0.1.0.0 1H IN PTR an-test-worker1003.eqiad.wmnet.
 2.2.0.0.3.5.0.0.4.6.0.0.0.1.0.0 1H IN PTR analytics1042.eqiad.wmnet.
 3.2.0.0.3.5.0.0.4.6.0.0.0.1.0.0 1H IN PTR analytics1043.eqiad.wmnet.
diff --git a/eqiad.wmnet b/eqiad.wmnet
index faf9e4e..8276670 100644
--- a/eqiad.wmnet
+++ b/eqiad.wmnet
@@ -38,6 +38,8 @@ an-test-client1001                       1H IN A 10.64.5.6
 an-test-client1001                       1H IN AAAA 2620:0:861:104:10:64:5:6
 an-test-coord1001                        1H IN A 10.64.53.41
 an-test-coord1001                        1H IN AAAA 2620:0:861:108:10:64:53:41
+an-test-druid1001                        1H IN A 10.64.53.6
+an-test-druid1001                        1H IN AAAA 2620:0:861:108:10:64:53:6
 an-test-master1001                       1H IN A 10.64.5.39
 an-test-master1001                       1H IN AAAA 2620:0:861:104:10:64:5:39
 an-test-master1002                       1H IN A 10.64.36.112
METADATA: {"path": "/tmp/dns-c25pcHBldHM-jdruvzjh", "sha1": "1d04e945917ee0470dff1c49e67146287211b411", "insertions": 4, "deletions": 0, "lines": 4, "files": 3}                                                  
Have you checked that the diff is OK?
Type "done" to proceed
> done
2020-10-29 11:13:38,287 [INFO] Pushed with bitflags 256: 3b5fcdc..1d04e94
2020-10-29 11:13:38,354 [INFO] Temporary directory /tmp/dns-c25pcHBldHM-jdruvzjh removed.
Updating the Netbox passive copies of the repository on netbox2001.wikimedia.org
Updating the authdns copies of the repository on authdns[1001,2001].wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002].wikimedia.org                                                            
Deploying the updated zonefiles on authdns[1001,2001].wikimedia.org,dns[1001-1002,2001-2002,3001-3002,4001-4002,5001-5002].wikimedia.org                                                                          
The Ganeti's command output will be printed at the end.
Creating VM an-test-druid1001.eqiad.wmnet in cluster ganeti01.svc.eqiad.wmnet with row=D vcpus=4 memory=8GB disk=100GB link=analytics. This may take a few minutes.                                               
Thu Oct 29 11:14:17 2020  - INFO: No-installation mode selected, disabling startup
Thu Oct 29 11:14:21 2020  - INFO: Selected nodes for instance an-test-druid1001.eqiad.wmnet via iallocator hail: ganeti1021.eqiad.wmnet, ganeti1020.eqiad.wmnet
Thu Oct 29 11:14:22 2020 * creating instance disks...
Thu Oct 29 11:14:27 2020 adding instance an-test-druid1001.eqiad.wmnet to cluster config
Thu Oct 29 11:14:27 2020 adding disks to cluster config
Thu Oct 29 11:14:27 2020  - INFO: Waiting for instance an-test-druid1001.eqiad.wmnet to sync disks
Thu Oct 29 11:14:27 2020  - INFO: - device disk/0:  0.10% done, 3h 28m 2s remaining (estimated)
Thu Oct 29 11:15:27 2020  - INFO: - device disk/0:  2.20% done, 45m 13s remaining (estimated)
Thu Oct 29 11:16:28 2020  - INFO: - device disk/0:  4.40% done, 42m 47s remaining (estimated)
Thu Oct 29 11:17:28 2020  - INFO: - device disk/0:  6.50% done, 42m 30s remaining (estimated)
Thu Oct 29 11:18:28 2020  - INFO: - device disk/0:  8.70% done, 41m 57s remaining (estimated)
Thu Oct 29 11:19:28 2020  - INFO: - device disk/0: 10.80% done, 39m 33s remaining (estimated)
Thu Oct 29 11:20:28 2020  - INFO: - device disk/0: 13.00% done, 39m 11s remaining (estimated)
Thu Oct 29 11:21:29 2020  - INFO: - device disk/0: 15.10% done, 38m 56s remaining (estimated)
Thu Oct 29 11:22:29 2020  - INFO: - device disk/0: 17.30% done, 38m 22s remaining (estimated)
Thu Oct 29 11:23:29 2020  - INFO: - device disk/0: 19.40% done, 36m 2s remaining (estimated)
Thu Oct 29 11:24:29 2020  - INFO: - device disk/0: 21.60% done, 35m 16s remaining (estimated)
Thu Oct 29 11:25:29 2020  - INFO: - device disk/0: 23.70% done, 35m 20s remaining (estimated)
Thu Oct 29 11:26:30 2020  - INFO: - device disk/0: 25.90% done, 32m 49s remaining (estimated)
Thu Oct 29 11:27:30 2020  - INFO: - device disk/0: 28.00% done, 32m 43s remaining (estimated)
Thu Oct 29 11:28:30 2020  - INFO: - device disk/0: 30.20% done, 32m 4s remaining (estimated)
Thu Oct 29 11:29:30 2020  - INFO: - device disk/0: 32.40% done, 30m 1s remaining (estimated)
Thu Oct 29 11:30:30 2020  - INFO: - device disk/0: 34.50% done, 29m 33s remaining (estimated)
Thu Oct 29 11:31:30 2020  - INFO: - device disk/0: 36.70% done, 28m 56s remaining (estimated)
Thu Oct 29 11:32:31 2020  - INFO: - device disk/0: 38.80% done, 27m 3s remaining (estimated)
Thu Oct 29 11:33:31 2020  - INFO: - device disk/0: 41.00% done, 26m 25s remaining (estimated)
Thu Oct 29 11:34:31 2020  - INFO: - device disk/0: 43.10% done, 26m 3s remaining (estimated)
Thu Oct 29 11:35:31 2020  - INFO: - device disk/0: 45.30% done, 25m 17s remaining (estimated)
Thu Oct 29 11:36:31 2020  - INFO: - device disk/0: 47.40% done, 23m 39s remaining (estimated)
Thu Oct 29 11:37:32 2020  - INFO: - device disk/0: 49.60% done, 22m 49s remaining (estimated)
Thu Oct 29 11:38:32 2020  - INFO: - device disk/0: 51.70% done, 22m 11s remaining (estimated)
Thu Oct 29 11:39:32 2020  - INFO: - device disk/0: 53.90% done, 21m 33s remaining (estimated)
Thu Oct 29 11:40:32 2020  - INFO: - device disk/0: 56.00% done, 19m 44s remaining (estimated)
Thu Oct 29 11:41:32 2020  - INFO: - device disk/0: 58.20% done, 19m 5s remaining (estimated)
Thu Oct 29 11:42:32 2020  - INFO: - device disk/0: 60.30% done, 18m 28s remaining (estimated)
Thu Oct 29 11:43:33 2020  - INFO: - device disk/0: 62.50% done, 16m 45s remaining (estimated)
Thu Oct 29 11:44:33 2020  - INFO: - device disk/0: 64.60% done, 16m 0s remaining (estimated)
Thu Oct 29 11:45:33 2020  - INFO: - device disk/0: 66.80% done, 15m 18s remaining (estimated)
Thu Oct 29 11:46:33 2020  - INFO: - device disk/0: 69.00% done, 13m 45s remaining (estimated)
Thu Oct 29 11:47:33 2020  - INFO: - device disk/0: 71.10% done, 13m 3s remaining (estimated)
Thu Oct 29 11:48:34 2020  - INFO: - device disk/0: 73.30% done, 12m 18s remaining (estimated)
Thu Oct 29 11:49:34 2020  - INFO: - device disk/0: 75.40% done, 10m 53s remaining (estimated)
Thu Oct 29 11:50:34 2020  - INFO: - device disk/0: 77.60% done, 10m 5s remaining (estimated)
Thu Oct 29 11:51:34 2020  - INFO: - device disk/0: 79.70% done, 9m 19s remaining (estimated)
Thu Oct 29 11:52:34 2020  - INFO: - device disk/0: 81.90% done, 8m 26s remaining (estimated)
Thu Oct 29 11:53:34 2020  - INFO: - device disk/0: 84.00% done, 7m 10s remaining (estimated)
Thu Oct 29 11:54:35 2020  - INFO: - device disk/0: 86.20% done, 6m 18s remaining (estimated)
Thu Oct 29 11:55:35 2020  - INFO: - device disk/0: 88.30% done, 5m 23s remaining (estimated)
Thu Oct 29 11:56:35 2020  - INFO: - device disk/0: 90.50% done, 4m 15s remaining (estimated)
Thu Oct 29 11:57:35 2020  - INFO: - device disk/0: 92.60% done, 3m 21s remaining (estimated)
Thu Oct 29 11:58:35 2020  - INFO: - device disk/0: 94.80% done, 2m 24s remaining (estimated)
Thu Oct 29 11:59:36 2020  - INFO: - device disk/0: 96.90% done, 1m 23s remaining (estimated)
Thu Oct 29 12:00:36 2020  - INFO: - device disk/0: 99.10% done, 25s remaining (estimated)
Thu Oct 29 12:01:01 2020  - INFO: - device disk/0: 100.00% done, 1s remaining (estimated)
Thu Oct 29 12:01:02 2020  - INFO: - device disk/0: 100.00% done, 0s remaining (estimated)
Thu Oct 29 12:01:02 2020  - INFO: - device disk/0: 100.00% done, 0s remaining (estimated)
Thu Oct 29 12:01:02 2020  - INFO: - device disk/0: 100.00% done, 0s remaining (estimated)
Thu Oct 29 12:01:02 2020  - INFO: - device disk/0: 100.00% done, 0s remaining (estimated)
Thu Oct 29 12:01:03 2020  - INFO: - device disk/0: 100.00% done, 0s remaining (estimated)
Thu Oct 29 12:01:03 2020  - INFO: Instance an-test-druid1001.eqiad.wmnet's disks are in sync
Thu Oct 29 12:01:03 2020  - INFO: Waiting for instance an-test-druid1001.eqiad.wmnet to sync disks
Thu Oct 29 12:01:03 2020  - INFO: Instance an-test-druid1001.eqiad.wmnet's disks are in sync
MAC address for an-test-druid1001.eqiad.wmnet is: aa:00:00:3c:aa:38
Syncing VMs in DC eqiad to Netbox
Created interface ##PRIMARY## on VM an-test-druid1001
Attached IPv4 10.64.53.6/24 and IPv6 2620:0:861:108:10:64:53:6/64 to VM an-test-druid1001 and marked as primary IPs
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0)
root@cumin1001:~#

Change 637504 had a related patch set uploaded (by Klausman; owner: Klausman):
[operations/puppet@production] install-server: Add DHCP entry for an-test-druid1001

https://gerrit.wikimedia.org/r/637504

gerritbot added a project: Patch-For-Review.Oct 29 2020, 2:45 PM

Change 637504 merged by Klausman:
[operations/puppet@production] install-server: Add DHCP entry for an-test-druid1001

https://gerrit.wikimedia.org/r/637504

Maintenance_bot removed a project: Patch-For-Review.Oct 29 2020, 3:10 PM

Ottomata added a project: Analytics-Clusters.Jan 25 2021, 4:41 PM

Ottomata moved this task from Backlog to Done on the Analytics-Clusters board.