Page MenuHomePhabricator

Create schema[12]00[12] (schema.svc.{eqiad,codfw}.wmnet)
Closed, ResolvedPublic5 Story Points

Description

These will be very simple machines that will use lvs (and discovery?) for reliability. They will host a simple http server serving readonly static files cloned from git schema repositories.

These are simple webservers hosting static content, so the resource requirements are very low. Allocating:

  • 2 vCPU
  • 2G RAM
  • 10G disk

Event Timeline

Ottomata created this task.Mar 28 2019, 8:33 PM
Restricted Application added a project: Operations. · View Herald TranscriptMar 28 2019, 8:33 PM
Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Milimetric triaged this task as High priority.Apr 1 2019, 3:39 PM
Milimetric moved this task from Incoming to Modern Event Platform on the Analytics board.

@akosiaris, I'd love to do this sooner rather than later. It'd make some configuration/deployment stuff in the Hive/Hadoop world much easier.

Am I allowed to follow https://wikitech.wikimedia.org/wiki/Ganeti#Create_a_VM and do this myself?

@akosiaris, I'd love to do this sooner rather than later. It'd make some configuration/deployment stuff in the Hive/Hadoop world much easier.
Am I allowed to follow https://wikitech.wikimedia.org/wiki/Ganeti#Create_a_VM and do this myself?

Sure, I don't see why not. Just make sure to fill in the vm-requests task (I guess this task will do) with the requested resources/network configs so we have some papertrail and so that people can comment.

Change 502866 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/dns@master] Add DNS entries for schema[12]00[12] ganeti VMs

https://gerrit.wikimedia.org/r/502866

Change 502866 merged by Ottomata:
[operations/dns@master] Add DNS entries for schema[12]00[12] ganeti VMs

https://gerrit.wikimedia.org/r/502866

Ottomata updated the task description. (Show Details)Apr 10 2019, 7:38 PM

On ganeti1003:

sudo gnt-instance add -t drbd -I hail --net 0:link=private --hypervisor-parameters=kvm:boot_order=network -o debootstrap+default --no-install -g row_A -B vcpus=2,memory=2g --disk 0:size=10g schema1001.eqiad.wmnet
sudo gnt-instance info schema1001.eqiad.wmnet | grep MAC
      MAC: aa:00:00:d0:90:00

sudo gnt-instance add -t drbd -I hail --net 0:link=private --hypervisor-parameters=kvm:boot_order=network -o debootstrap+default --no-install -g row_C -B vcpus=2,memory=2g --disk 0:size=10g schema1002.eqiad.wmnet
sudo gnt-instance info schema1002.eqiad.wmnet | grep MAC
      MAC: aa:00:00:a0:5c:cf

On ganeti2003:

sudo gnt-instance add -t drbd -I hail --net 0:link=private --hypervisor-parameters=kvm:boot_order=network -o debootstrap+default --no-install -g row_A -B vcpus=2,memory=2g --disk 0:size=10g schema2001.codfw.wmnet
sudo gnt-instance info schema2001.codfw.wmnet | grep MAC
      MAC: aa:00:00:25:b5:b3

sudo gnt-instance add -t drbd -I hail --net 0:link=private --hypervisor-parameters=kvm:boot_order=network -o debootstrap+default --no-install -g row_B -B vcpus=2,memory=2g --disk 0:size=10g schema2002.codfw.wmnet
sudo gnt-instance info schema2002.codfw.wmnet | grep MAC
      MAC: aa:00:00:f6:d7:82

Change 503079 had a related patch set uploaded (by CDanis; owner: CDanis):
[operations/puppet@production] kubelet operational latencies: increase thresholds by 10x

https://gerrit.wikimedia.org/r/503079

Change 504646 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add linux-host-entries and netboot/partman for schemaa[12]00[12]

https://gerrit.wikimedia.org/r/504646

Ottomata set the point value for this task to 5.

Change 504646 merged by Ottomata:
[operations/puppet@production] Add linux-host-entries and netboot/partman for schema[12]00[12]

https://gerrit.wikimedia.org/r/504646

ganeti1001, 1002 and 2001 have been installed. I dunno what's up with ganeti2002. gnt-instance console schema2002.codfw.wmnet never shows anything.

Change 504845 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/puppet@production] Fix typo with schema2002.codfw.wmnet

https://gerrit.wikimedia.org/r/504845

Change 504845 merged by Alexandros Kosiaris:
[operations/puppet@production] Fix typo with schema2002.codfw.wmnet

https://gerrit.wikimedia.org/r/504845

ganeti1001, 1002 and 2001 have been installed. I dunno what's up with ganeti2002. gnt-instance console schema2002.codfw.wmnet never shows anything.

Typo. fixed in https://gerrit.wikimedia.org/r/504845, console shows up normally now and VM is being installed. Leaving it to you to finish the puppet stuff

Change 503079 abandoned by Alexandros Kosiaris:
kubelet operational latencies: increase thresholds by 10x

Reason:
Per comments above

https://gerrit.wikimedia.org/r/503079

Ottomata moved this task from Next Up to Done on the Analytics-Kanban board.Apr 19 2019, 4:02 PM
Ottomata moved this task from Backlog to Done on the EventBus board.Apr 23 2019, 3:13 PM
Dzahn added a subscriber: Dzahn.Apr 30 2019, 1:22 AM

When adding a new type of server name please add them in these 2 places:

  • cumin aliases (currently we are getting mail with "DC aliases do not cover all hosts: schema[2001-2002].codfw.wmnet,schema[1001-1002].eqiad.wmnet")

(modules/profile/templates/cumin/aliases.yaml.erb , https://wikitech.wikimedia.org/wiki/Cumin#Global_grammar_host_selection)

Change 507312 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Add cumin aliasaes for schema*

https://gerrit.wikimedia.org/r/507312

Done, thanks. Also edited https://wikitech.wikimedia.org/wiki/Ganeti#Assign_a_hostname%2FIP with instructions for future mes :)

Change 507353 had a related patch set uploaded (by Ottomata; owner: Ottomata):
[operations/puppet@production] Include profile::standard and base::firewall in role::eventschemas::service

https://gerrit.wikimedia.org/r/507353

Change 507353 merged by Ottomata:
[operations/puppet@production] Include profile::standard and base::firewall in role::eventschemas::service

https://gerrit.wikimedia.org/r/507353

Change 507312 merged by Ottomata:
[operations/puppet@production] Add cumin aliasaes for schema*

https://gerrit.wikimedia.org/r/507312

Nuria closed this task as Resolved.May 14 2019, 9:13 PM