This task will track the racking and wiring of 5 additional shelves to the furud storage array.
Please use the following checklist:
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Unknown Object (Task) | |||||
| Resolved | Papaul | T185153 attach furud's new arrays (furud-array[3-7]) |
Please note I've assigned this task to @faidon to review and approve, since he has been point person on this system's project.
Ideally, once we have furud's existing 2 shelves, then one of us can ssh in and poll the raid controller on what kind of raid is present. Since they aren't attached, I'm not certain what is setup. We should be able to simply append more shelves in, and add them to the raid array(s) and grow the LVM across them.
Please comment and append/change steps as needed, and assign to @Papaul for processing when the shelves arrive. Thanks!
Let's keep the existing arrays (array1 & array2) offline, and just connect all of the new ones.
Also, no RAID config needed on the BIOS; we'll do the config by hand using megacli later, they're just in JBOD mode anyway.
Thanks for taking care of this before your trip! I checked this out last week, and it seemed then (and now that I double-checked it) that only three shelves (36 disks) are visible, rather than 5 (array3-7).
Yes that is correct we have only 3 shelves 36 disks visible the reason being that we have 3 shelves (36 disks) connected to port 2 and 4 shelves (48 disks) connected to port 1 (port1 is not plugged in for now only port 2) according to the racking schema provided by Rob. See image below for the racking . (multi-chain configuration)
I'd like all the 5 shelves (array3-7) connected to furud, but not the 2 old ones (array1-2) until further notice. Can we just bypass array1-2 by disconnecting them entirely, and creating a chain with just array3-7?
I rebooted furud and is not booting right now, saying:
The total number of enclosures connected to connector 01, has exceeded the maximum allowable limit of 4 enclosures. Please remove the extra enclosures and then restart your system.
@RobH could you perhaps help out with the topology here?
@Papaul/@RobH, if in doubt, let's talk on IRC for more. Assume this to have the highest priority of all the codfw on-site tasks.
@faidon i have 4 arrays attached to port 1 according to the message you are having we can not connect more than 4 arrays on port 1.
so if you want all 5 new arrays to be on line here is a suggestion
If it works for you let me know.
Figured this out with @Papaul on IRC (thanks!).
We have now:
megacli shows 6 arrays with 12 disks each, with a bit of an odd numbering:
root@furud:~# megacli -PDList -a0 | grep '^Enclosure Device ID' | uniq -c
12 Enclosure Device ID: 39
12 Enclosure Device ID: 53
12 Enclosure Device ID: 66
12 Enclosure Device ID: 78
12 Enclosure Device ID: 92We went through each of these with @Papaul, turning on and off their ID LED one by one, with:
megacli -PDLocate -PhysDrv '[XX:0]' -a0 megacli -PDLocate -stop -PhysDrv '[XX:0]' -a0
and we ended up with:
| Device ID | Array # |
| 39 | 5 |
| 53 | 6 |
| 66 | 7 |
| 78 | 3 |
| 92 | 4 |