reporting from IRC
17:51 <godog> cmjohnson1: it is easier to think about it if they are more spread equally among rows, 3x per row should do it, it matters less 10G vs 1G in this case ``
I've fixed the raid on the machines (cfr https://wikitech.wikimedia.org/wiki/Platform-specific_documentation/HP_DL3N0_Gen9#ms-be_RAID0_config) that were able to get to debian-installer via PXE and these are now reinstalling:
=== ms-be1028 === ms-be1029 === ms-be1030 === ms-be1035 === ms-be1037 === ms-be1038
These don't seem to successfully PXE boot into debian-installer, could you take a look why is that?
=== ms-be1031 === ms-be1032 === ms-be1033 === ms-be1034
These I don't seem to be able to reach the console
=== ms-be1036 === ms-be1039
1031's port was also a member of labs-instances vlan, removed the port from there and disabled/enabled the port and now 1031 can pxe-boot.
1031 / 1032 / 1033 still had their 10G interfaces enabled and thus 1G interfaces would show up starting from eth2. I've disabled the 10G interfaces on all three and machines are now installing
Just powercycled ms-be1016 that was stuck in console (pingable but no ssh available):
[11674384.225319] BUG: soft lockup - CPU#12 stuck for 22s! [migration/12:149] in ms-be1016's console
When I powercycled I saw:
error: diskfilter writes are not supported. Press any key to continue...