Page MenuHomePhabricator

lvs2009/lvs2010 with no RAID configured
Closed, ResolvedPublic

Description

As reported by @faidon in T136562, lvs2009 and lvs2010 lack proper RAID configuration:

lvs2009
vgutierrez@lvs2009:~$ sudo fdisk -l
Disk /dev/sda: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0xf47ed367

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sda1  *         2048 466862079 466860032 222.6G 83 Linux
/dev/sda2       466864126 468860927   1996802   975M  5 Extended
/dev/sda5       466864128 468860927   1996800   975M 82 Linux swap / Solaris

Partition 2 does not start on physical sector boundary.


Disk /dev/sdb: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 847DAEB4-B413-48A4-8059-EFEA1A07EC78

Device     Start       End   Sectors   Size Type
/dev/sdb1   2048 468860927 468858880 223.6G Microsoft basic data
lvs2010
vgutierrez@lvs2010:~$ sudo fdisk -l
Disk /dev/sda: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0xefe402d4

Device     Boot     Start       End   Sectors   Size Id Type
/dev/sda1  *         2048 466862079 466860032 222.6G 83 Linux
/dev/sda2       466864126 468860927   1996802   975M  5 Extended
/dev/sda5       466864128 468860927   1996800   975M 82 Linux swap / Solaris

Partition 2 does not start on physical sector boundary.


Disk /dev/sdb: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 1DDD8559-9054-491B-A640-9FFEDA68E0B5

Device     Start       End   Sectors   Size Type
/dev/sdb1   2048 468860927 468858880 223.6G Microsoft basic data

Event Timeline

Vgutierrez created this task.
Vgutierrez moved this task from Backlog to Hardware on the Traffic board.

Looks like the culprit is https://github.com/wikimedia/puppet/blob/production/modules/install_server/files/autoinstall/netboot.cfg#L126-L127:

lvs100[7-9]|lvs101[012]|lvs2*) echo partman/flat.cfg ;; \
lvs100[1-6]|lvs101[3-6]|lvs[345]*|lvs2009|lvs2010) echo partman/raid1-lvm.cfg ;; \

i would say this is part of T196560 which is still open. i left a comment there.

@Vgutierrez does this mean reinstalling both LVS servers (lvs2009 and lvs2010) if yes please elaborate how you want to approach this .

Thanks.

@Papaul that's right, reinstalling both servers would be the fastest/safest approach :)

@Vgutierrez let me know when I have green light to start working on this.

Change 464156 had a related patch set uploaded (by Vgutierrez; owner: Vgutierrez):
[operations/puppet@production] install_server: limit flat.cfg to lvs200[1-6] letting lvs20[09-10] out

https://gerrit.wikimedia.org/r/464156

Change 464156 merged by Dzahn:
[operations/puppet@production] install_server: limit flat.cfg to lvs200[1-6] letting lvs20[09-10] out

https://gerrit.wikimedia.org/r/464156

puppet certs for lvs2009 and lvs2010 delete from master for OS reinstall

The reason the OS reinstall is taking long is that when all 4 NIC's are plugged in, the server can not auto configure the first NIC so I have to go in the BIOS and disable the other 3 NIC's. The OS is installing now.

LVS 2009

root@lvs2009:~# fdisk -l
Disk /dev/sda: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x2c601a25

Device     Boot    Start       End   Sectors  Size Id Type
/dev/sda1  *        2048  97656831  97654784 46.6G fd Linux raid autodetect
/dev/sda2       97656832 468860927 371204096  177G fd Linux raid autodetect


Disk /dev/sdb: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x46a3abf2

Device     Boot    Start       End   Sectors  Size Id Type
/dev/sdb1           2048  97656831  97654784 46.6G fd Linux raid autodetect
/dev/sdb2       97656832 468860927 371204096  177G fd Linux raid autodetect


Disk /dev/md1: 176.9 GiB, 189922279424 bytes, 370941952 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/md0: 46.5 GiB, 49965694976 bytes, 97589248 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/lvs2009--vg-swap: 952 MiB, 998244352 bytes, 1949696 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/lvs2009--vg-_placeholder: 176 GiB, 188919840768 bytes, 368984064 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
  • Re-enable the other 3 NIC's
  • First puppet run complete

LVS2009 is ready

lvs2010

root@lvs2010:~# fdisk -l
Disk /dev/sda: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0x8a6958cc

Device     Boot    Start       End   Sectors  Size Id Type
/dev/sda1  *        2048  97656831  97654784 46.6G fd Linux raid autodetect
/dev/sda2       97656832 468860927 371204096  177G fd Linux raid autodetect


Disk /dev/sdb: 223.6 GiB, 240057409536 bytes, 468862128 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: dos
Disk identifier: 0xdd5dea9a

Device     Boot    Start       End   Sectors  Size Id Type
/dev/sdb1           2048  97656831  97654784 46.6G fd Linux raid autodetect
/dev/sdb2       97656832 468860927 371204096  177G fd Linux raid autodetect


Disk /dev/md0: 46.5 GiB, 49965694976 bytes, 97589248 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/md1: 176.9 GiB, 189922279424 bytes, 370941952 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/lvs2010--vg-swap: 952 MiB, 998244352 bytes, 1949696 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes


Disk /dev/mapper/lvs2010--vg-_placeholder: 176 GiB, 188919840768 bytes, 368984064 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes

Lvs2010 is ready.

Vgutierrez assigned this task to Papaul.
Vgutierrez removed a project: Patch-For-Review.