Page MenuHomePhabricator
Create Task
Maniphest T164011

codfw: ganeti2007-ganeti2008 racking and onsite setup task
Closed, ResolvedPublic
Actions

Description

This task will outline where the new ganeti systems (ordered on T161701) should be racked and setup.

Existing ganeti racking location

Row B rack B1
ganeti2001
ganeti2002

Row B rack B5
ganeti2003
ganeti2004

Row B rack B8
ganeti2005
ganeti2006

New ganeti racking schema

Row A rack A5
ganeti2007

Row C rack C5
ganeti2008

Please approve or modify new ganeti racking schema

  • racking schema approve ?
  • receive in and attach packing slip to parent task T161701
  • rack systems, update racktables
  • create mgmt dns entries (both asset tag and hostname)
  • create production dns entries (internal vlan)
  • update/create sub task with network port info for all new hosts
  • install_server module update (mac address and partitioning info, partition = ganeti.cfg)

Please provide HW RAID level

  • RAID 0
  • RAID 1
  • RAID 5
  • install os
  • puppet/salt accept
  • hand off to @akosiaris for service implementation.

Details

SubjectRepoBranchLines +/-
Assign IPs for ganeti2007, ganeti2008operations/dnsmaster+4 -0
DNS: Add mgmt dns entries for ganeti200[7-8]operations/dnsmaster+8 -0
Assign new IPs to ganeti2005, ganeti2006operations/dnsmaster+5 -4
Customize query in gerrit

Related Objects
Search...

Event Timeline

Papaul created this task.Apr 27 2017, 4:16 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 27 2017, 4:16 PM
Papaul renamed this task from codfw: ganeti2007-ganeti2008 racking and onsite setup tasks to codfw: ganeti2007-ganeti2008 racking and onsite setup task.Apr 27 2017, 4:16 PM
RobH reassigned this task from Papaul to akosiaris.Apr 27 2017, 4:48 PM

So @akosiaris should review and approve of the racking layout, since he is the ganeti expert!

Alex: On these new nodes, do you want them distributed equally and thus put in row A and C as @Papaul suggests, or do these need to in the same row as the existing nodes? When checking eqiad or codfw, its unclear, since all of eqiads gaenti are in row c, and all of codfw's are in row b, so currently all ganeti nodes in a site are in the same row. Wasn't sure if that was intentional or merely happenstance.

Please advise, so @Papaul can move forward with racking, thanks! Also please reassign from @akosiaris to @Papaul once commented.

Stashbot subscribed.Apr 28 2017, 10:47 AM

Mentioned in SAL (#wikimedia-operations) [2017-04-28T10:47:30Z] <akosiaris> migrate/evacuate ganeti2005, ganeti2006 for T164011

akosiaris added a comment.Apr 28 2017, 2:02 PM

No, that rack schema wont work.

What we want to do is have 4 boxes per rack row. I am already emptying ganeti2005, ganeti2006. Those 2, alongside ganeti2007 and ganeti2008 should be put in a different rack row than row B. Row A would probably be a good idea. A quick look at racktables says that A2 and A4 (2 boxes per rack) looks like a good match. Alternatively if we don't want row A for some reason, C5 and C7 look like a good match as well

@RobH, @Papaul how does A2, A4 sound ?

akosiaris added a comment.Apr 28 2017, 2:26 PM

I 've fully removed ganeti2005, ganeti2006 from the cluster and downtimed them in icinga for 2 months, then turned them off. Feel free to unrack them whenever you want.

Dzahn subscribed.Apr 28 2017, 4:11 PM

We have lots of room in A2 and A4 and we can move into A4, but we can't move into A2 because there is a 10G switch and the server just has 1G nic cards (would need adapter).

Dzahn added a comment.Apr 28 2017, 4:30 PM

ganeti2005 has been moved to A4 @ 16. switch: asw-a4-codfw: port ge-4/0/16 please configure switch

Dzahn added a comment.Apr 28 2017, 4:51 PM

ganeti2006 has been moved to A4 @ 17. switch: asw-a4-codfw: port ge-4/0/17 please configure switch

Dzahn added a comment.Apr 28 2017, 5:24 PM

2007 and 2008 should go into A5 then (still has to happen)

Stashbot added a comment.May 2 2017, 9:24 AM

Mentioned in SAL (#wikimedia-operations) [2017-05-02T09:24:02Z] <akosiaris> remove configuration from ge-8/0/0, ge-8/0/3 from asw-b-codfw for ganeti2005, ganeti2006 move to row A. T164011

Stashbot added a comment.May 2 2017, 9:27 AM

Mentioned in SAL (#wikimedia-operations) [2017-05-02T09:27:00Z] <akosiaris> create interface range ganeti on asw-a-codfw. T164011

Stashbot added a comment.May 2 2017, 9:29 AM

Mentioned in SAL (#wikimedia-operations) [2017-05-02T09:29:01Z] <akosiaris> Set description for ganeti2005, ganeti2006 on asw-a-codfw. T164011

gerritbot subscribed.May 2 2017, 9:36 AM

Change 351275 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/dns@master] Assign new IPs to ganeti2005, ganeti2006

https://gerrit.wikimedia.org/r/351275

gerritbot added a project: Patch-For-Review.May 2 2017, 9:36 AM
akosiaris added a comment.May 2 2017, 10:20 AM
In T164011#3221336, @Dzahn wrote:

We have lots of room in A2 and A4 and we can move into A4, but we can't move into A2 because there is a 10G switch and the server just has 1G nic cards (would need adapter).

Ah , indeed I forgot about that. Thanks.

In T164011#3221615, @Dzahn wrote:

2007 and 2008 should go into A5 then (still has to happen)

Sounds fine.

In T164011#3221482, @Dzahn wrote:

ganeti2005 has been moved to A4 @ 16. switch: asw-a4-codfw: port ge-4/0/16 please configure switch

Note that this is actually ge-4/0/15, not ge-4/0/16. Port configured

In T164011#3221526, @Dzahn wrote:

ganeti2006 has been moved to A4 @ 17. switch: asw-a4-codfw: port ge-4/0/17 please configure switch

Note that this is actually ge-4/0/16, not ge-4/0/17. Port configured

gerritbot added a comment.May 2 2017, 10:20 AM

Change 351275 merged by Alexandros Kosiaris:
[operations/dns@master] Assign new IPs to ganeti2005, ganeti2006

https://gerrit.wikimedia.org/r/351275

Stashbot added a comment.May 2 2017, 10:44 AM

Mentioned in SAL (#wikimedia-operations) [2017-05-02T10:44:26Z] <akosiaris> create new ganeti nodegroup called row_A holding ganeti2005, ganeti2006. Renamed the default nodegroup to row_B. T164011

gerritbot added a comment.May 2 2017, 12:22 PM

Change 351303 had a related patch set uploaded (by Alexandros Kosiaris; owner: Alexandros Kosiaris):
[operations/dns@master] Assign IPs for ganeti2007, ganeti2008

https://gerrit.wikimedia.org/r/351303

Papaul claimed this task.May 4 2017, 3:36 PM
Papaul triaged this task as Medium priority.
Papaul updated the task description. (Show Details)May 4 2017, 3:45 PM
Papaul added a comment.May 4 2017, 3:48 PM

@akosiaris what HW RAID type I have to use to ganeti200[7-8]?

gerritbot added a comment.May 4 2017, 4:09 PM

Change 351877 had a related patch set uploaded (by Papaul; owner: Papaul):
[operations/dns@master] DNS: Add mgmt dns entries for ganeti200[7-8]

https://gerrit.wikimedia.org/r/351877

Papaul updated the task description. (Show Details)May 4 2017, 4:24 PM
gerritbot added a comment.May 4 2017, 5:17 PM

Change 351877 merged by Dzahn:
[operations/dns@master] DNS: Add mgmt dns entries for ganeti200[7-8]

https://gerrit.wikimedia.org/r/351877

akosiaris added a parent task: T122134: url-downloader should be set up more redundantly.May 5 2017, 7:29 AM
Papaul created subtask T164594: codfw: ganeti2007-ganeti2008 switch port configuration .May 5 2017, 3:56 PM
Papaul updated the task description. (Show Details)May 5 2017, 4:03 PM
gerritbot added a comment.May 8 2017, 11:42 AM

Change 351303 merged by Alexandros Kosiaris:
[operations/dns@master] Assign IPs for ganeti2007, ganeti2008

https://gerrit.wikimedia.org/r/351303

akosiaris closed subtask T164594: codfw: ganeti2007-ganeti2008 switch port configuration as Resolved.May 8 2017, 11:48 AM
Papaul reassigned this task from Papaul to akosiaris.May 8 2017, 4:23 PM
Papaul updated the task description. (Show Details)

@akosiaris This is complete on my side you can take over.

akosiaris closed this task as Resolved.May 10 2017, 2:35 PM

ganeti2007, ganeti2008 are installed, fully updated (along with ganeti2005, ganeti2006) and part of the cluster. I 'll resolve this. Thanks @Papaul

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL