Page MenuHomePhabricator

Create a VM for matomo1002 (eqiad)
Closed, ResolvedPublic

Description

Hi!

As part of the migration to Debian Buster, I'd need a new vm to replace matomo1001:

Given the current usage, I'd need something like:

  • 4 vcores
  • 50/100G of disk space usage - We are using only 12G currently, but usage varied a lot in the past, so: 50G would be fine, but with 100G I'd be more happy.
  • 8G of memory
  • Debian Buster
  • no public IP, no Analytics vlan

Event Timeline

Marostegui added a project: serviceops.
Marostegui added a subscriber: Dzahn.
elukey@ganeti1003:~$   sudo gnt-group list
Group Nodes Instances AllocPolicy NDParams
row_A     4        44 preferred   ovs=False, ssh_port=22, ovs_link=, spindle_count=1, exclusive_storage=False, cpu_speed=1, ovs_name=switch1, oob_program=
row_C     4        42 preferred   ovs=False, ssh_port=22, ovs_link=, spindle_count=1, exclusive_storage=False, cpu_speed=1, ovs_name=switch1, oob_program=

Change 597266 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/dns@master] Add A/AAAA/PTR records for matomo1002

https://gerrit.wikimedia.org/r/597266

Change 597266 merged by Elukey:
[operations/dns@master] Add A/AAAA/PTR records for matomo1002

https://gerrit.wikimedia.org/r/597266

elukey changed the task status from Open to Stalled.May 19 2020, 1:49 PM

This is currently blocked due to resource constraints in row_c eqiad for Ganeti, see https://wikitech.wikimedia.org/wiki/Ganeti#Verify_cluster_resource_availability

We can wait or create the VM in codfw.

elukey@cumin1001:~$ sudo cookbook sre.ganeti.makevm eqiad_C matomo1002.eqiad.wmnet --vcpus 4 --memory 8 --disk 50
START - Cookbook sre.ganeti.makevm
Ready to create Ganeti VM matomo1002.eqiad.wmnet in the ganeti01.svc.eqiad.wmnet cluster on row C with 4 vCPUs, 8GB of RAM, 50GB of disk in the private network.
Is this correct?
Type "done" to proceed
> done
The command output will be printed at the end.
Creating VM matomo1002.eqiad.wmnet in cluster ganeti01.svc.eqiad.wmnet with row=C vcpus=4 memory=8GB disk=50GB link=private. This may take a few minutes.
Fri Jun  5 09:46:49 2020  - INFO: No-installation mode selected, disabling startup
Fri Jun  5 09:48:48 2020  - INFO: Selected nodes for instance matomo1002.eqiad.wmnet via iallocator hail: ganeti1010.eqiad.wmnet, ganeti1009.eqiad.wmnet
Fri Jun  5 09:48:49 2020 * creating instance disks...
Fri Jun  5 09:48:54 2020 adding instance matomo1002.eqiad.wmnet to cluster config
Fri Jun  5 09:48:54 2020 adding disks to cluster config
Fri Jun  5 09:48:55 2020  - INFO: Waiting for instance matomo1002.eqiad.wmnet to sync disks
Fri Jun  5 09:48:55 2020  - INFO: - device disk/0:  0.10% done, 1h 4m 14s remaining (estimated)
Fri Jun  5 09:49:55 2020  - INFO: - device disk/0:  4.40% done, 22m 26s remaining (estimated)
Fri Jun  5 09:50:55 2020  - INFO: - device disk/0:  8.70% done, 21m 17s remaining (estimated)
Fri Jun  5 09:51:55 2020  - INFO: - device disk/0: 12.90% done, 19m 27s remaining (estimated)
Fri Jun  5 09:52:56 2020  - INFO: - device disk/0: 17.20% done, 18m 57s remaining (estimated)
Fri Jun  5 09:53:56 2020  - INFO: - device disk/0: 21.50% done, 18m 14s remaining (estimated)
Fri Jun  5 09:54:56 2020  - INFO: - device disk/0: 25.80% done, 16m 32s remaining (estimated)
Fri Jun  5 09:55:56 2020  - INFO: - device disk/0: 30.10% done, 15m 51s remaining (estimated)
Fri Jun  5 09:56:57 2020  - INFO: - device disk/0: 34.20% done, 15m 40s remaining (estimated)
Fri Jun  5 09:57:57 2020  - INFO: - device disk/0: 38.70% done, 13m 29s remaining (estimated)
Fri Jun  5 09:58:57 2020  - INFO: - device disk/0: 43.20% done, 12m 44s remaining (estimated)
Fri Jun  5 09:59:57 2020  - INFO: - device disk/0: 47.70% done, 10m 59s remaining (estimated)
Fri Jun  5 10:00:58 2020  - INFO: - device disk/0: 52.20% done, 10m 9s remaining (estimated)
Fri Jun  5 10:01:58 2020  - INFO: - device disk/0: 56.70% done, 9m 18s remaining (estimated)
Fri Jun  5 10:02:58 2020  - INFO: - device disk/0: 61.20% done, 8m 44s remaining (estimated)
Fri Jun  5 10:03:58 2020  - INFO: - device disk/0: 65.70% done, 7m 18s remaining (estimated)
Fri Jun  5 10:04:58 2020  - INFO: - device disk/0: 70.20% done, 6m 24s remaining (estimated)
Fri Jun  5 10:05:59 2020  - INFO: - device disk/0: 74.70% done, 5m 43s remaining (estimated)
Fri Jun  5 10:06:59 2020  - INFO: - device disk/0: 79.10% done, 4m 54s remaining (estimated)
Fri Jun  5 10:07:59 2020  - INFO: - device disk/0: 83.50% done, 3m 39s remaining (estimated)
Fri Jun  5 10:08:59 2020  - INFO: - device disk/0: 87.90% done, 2m 35s remaining (estimated)
Fri Jun  5 10:10:00 2020  - INFO: - device disk/0: 92.10% done, 2m 16s remaining (estimated)
Fri Jun  5 10:11:00 2020  - INFO: - device disk/0: 96.60% done, 45s remaining (estimated)
Fri Jun  5 10:11:45 2020  - INFO: - device disk/0: 99.90% done, 1s remaining (estimated)
Fri Jun  5 10:11:46 2020  - INFO: - device disk/0: 100.00% done, 0s remaining (estimated)
Fri Jun  5 10:11:46 2020  - INFO: Instance matomo1002.eqiad.wmnet's disks are in sync
Fri Jun  5 10:11:46 2020  - INFO: Waiting for instance matomo1002.eqiad.wmnet to sync disks
Fri Jun  5 10:11:47 2020  - INFO: Instance matomo1002.eqiad.wmnet's disks are in sync
MAC address for matomo1002.eqiad.wmnet is: aa:00:00:21:0c:ea
END (PASS) - Cookbook sre.ganeti.makevm (exit_code=0)
elukey changed the task status from Stalled to Open.Jun 5 2020, 10:15 AM

Change 602637 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] Add puppet configuration for matomo1002

https://gerrit.wikimedia.org/r/602637

Change 602637 merged by Elukey:
[operations/puppet@production] Add puppet configuration for matomo1002

https://gerrit.wikimedia.org/r/602637

Change 602742 had a related patch set uploaded (by Elukey; owner: Elukey):
[operations/puppet@production] profile::piwik::webserver: support Buster

https://gerrit.wikimedia.org/r/602742

Change 602742 merged by Elukey:
[operations/puppet@production] profile::piwik::webserver: support Buster

https://gerrit.wikimedia.org/r/602742

elukey claimed this task.