Page MenuHomePhabricator

attach furud's new arrays (furud-array[3-7])
Closed, ResolvedPublic


This task will track the racking and wiring of 5 additional shelves to the furud storage array.

Please use the following checklist:

  • - receive in the 5 shelves via T184625. This includes 4 2M and 1 14M SAS cable.
  • - rack the new arrays in the spots listed below. Dell advises only attaching half of the arrays to port 1 (so array1-4) and array5-7 to port2 of furud's external raid controller

Related Objects


Event Timeline

RobH triaged this task as High priority.Jan 17 2018, 10:41 PM
RobH created this task.

Please note I've assigned this task to @faidon to review and approve, since he has been point person on this system's project.

Ideally, once we have furud's existing 2 shelves, then one of us can ssh in and poll the raid controller on what kind of raid is present. Since they aren't attached, I'm not certain what is setup. We should be able to simply append more shelves in, and add them to the raid array(s) and grow the LVM across them.

Please comment and append/change steps as needed, and assign to @Papaul for processing when the shelves arrive. Thanks!

RobH mentioned this in Unknown Object (Task).Jan 17 2018, 10:45 PM

Let's keep the existing arrays (array1 & array2) offline, and just connect all of the new ones.

Also, no RAID config needed on the BIOS; we'll do the config by hand using megacli later, they're just in JBOD mode anyway.

Papaul updated the task description. (Show Details)

Thanks for taking care of this before your trip! I checked this out last week, and it seemed then (and now that I double-checked it) that only three shelves (36 disks) are visible, rather than 5 (array3-7).

Yes that is correct we have only 3 shelves 36 disks visible the reason being that we have 3 shelves (36 disks) connected to port 2 and 4 shelves (48 disks) connected to port 1 (port1 is not plugged in for now only port 2) according to the racking schema provided by Rob. See image below for the racking . (multi-chain configuration)

dell-storage-md1400-page2.png (705×954 px, 79 KB)

I'd like all the 5 shelves (array3-7) connected to furud, but not the 2 old ones (array1-2) until further notice. Can we just bypass array1-2 by disconnecting them entirely, and creating a chain with just array3-7?

I rebooted furud and is not booting right now, saying:

The total number of enclosures connected to connector 01, has exceeded
the maximum allowable limit of 4 enclosures. Please remove the extra enclosures
and then restart your system.

@RobH could you perhaps help out with the topology here?

@Papaul/@RobH, if in doubt, let's talk on IRC for more. Assume this to have the highest priority of all the codfw on-site tasks.

@faidon i have 4 arrays attached to port 1 according to the message you are having we can not connect more than 4 arrays on port 1.
so if you want all 5 new arrays to be on line here is a suggestion

  • Connect 3 on port 0 and 2 on port 1

If it works for you let me know.

Figured this out with @Papaul on IRC (thanks!).

We have now:

  • connector0 connected to arrays 3, 4, 5
  • connector1 connected to arrays 6, 7

megacli shows 6 arrays with 12 disks each, with a bit of an odd numbering:

root@furud:~# megacli -PDList -a0 | grep '^Enclosure Device ID' | uniq -c
     12 Enclosure Device ID: 39
     12 Enclosure Device ID: 53
     12 Enclosure Device ID: 66
     12 Enclosure Device ID: 78
     12 Enclosure Device ID: 92

We went through each of these with @Papaul, turning on and off their ID LED one by one, with:

megacli -PDLocate -PhysDrv '[XX:0]' -a0
megacli -PDLocate -stop -PhysDrv '[XX:0]' -a0

and we ended up with:

Device IDArray #

These are now attached and configured, resolving.