Page MenuHomePhabricator
Create Task
Maniphest T296856

Installation issues on PowerEdge R440 Ganeti servers with buster / firmware update needed
Closed, ResolvedPublic
Actions

Description

I tried to reinstall ganeti2010 with buster, but I'm getting a strange installer failure; over the serial console I can see that the d-i image and the initrd are being served (and it's also showing up in the logs of the install* server), but after that the system reboots into the installed system.

This might be an issue with the new kernel and outdated firmware. Could you please upgrade firmware on this host (and if possible also NIC firmware)? The server currently has no instance and can be powered down any time.

  • ganeti2007
  • ganeti2008
  • ganeti2009
  • ganeti2010
  • ganeti2011
  • ganeti2012
  • ganeti2013
  • ganeti2014
  • ganeti2015
  • ganeti2016
  • ganeti2017
  • ganeti2018

Related Objects

Event Timeline

MoritzMuehlenhoff created this task.Dec 1 2021, 3:40 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptDec 1 2021, 3:40 PM
Volans subscribed.Dec 1 2021, 3:42 PM
Papaul triaged this task as Medium priority.Dec 1 2021, 3:52 PM
BIOS Version 	1.7.0
iDRAC Firmware Version 	3.30.30.30
Papaul closed this task as Resolved.Dec 1 2021, 6:41 PM

this is complete

BIOS Version 	2.12.2
iDRAC Firmware Version 	5.00.10.20
MoritzMuehlenhoff added a comment.Dec 1 2021, 6:44 PM

Thanks for the quick turnaround, much appreciated! I'll kick off another reimage attempt tomorrow.

MoritzMuehlenhoff added a comment.Dec 2 2021, 8:17 AM

I have good and bad news :-) The good news is that the firmware update made the Buster installer work \o/. The bad news is that we have nine more servers of that batch, so those will likely all need a firmware update as well. Do you prefer to reuse/extend this task or should I rather create new ones as I run into them?

Volans mentioned this in T296906: reimage/pxe boot failing on cloudvirt1028.Dec 2 2021, 9:30 AM
MoritzMuehlenhoff reopened this task as Open.Dec 2 2021, 12:11 PM
In T296856#7542925, @MoritzMuehlenhoff wrote:

I have good and bad news :-) The good news is that the firmware update made the Buster installer work \o/. The bad news is that we have nine more servers of that batch, so those will likely all need a firmware update as well. Do you prefer to reuse/extend this task or should I rather create new ones as I run into them?

Unfortunately I was right and I ran into the same issue with ganeti2009 as well. Can you please also update it? Or alternatively I can also open a new task. The server can be powered off any time.

MoritzMuehlenhoff renamed this task from Installation issues on ganeti2010 with buster / firmware update to Installation issues on ganeti2009/2010 with buster / firmware update needed.Dec 2 2021, 12:11 PM
Papaul updated the task description. (Show Details)Dec 2 2021, 12:48 PM

I prefer to just reuse/extent the task by adding the nodes in the description like i did for ganeti2009 and 2010 so we keep better tack.

Thanks

MoritzMuehlenhoff added a comment.Dec 2 2021, 12:58 PM
In T296856#7543451, @Papaul wrote:

I prefer to just reuse/extent the task by adding the nodes in the description like i did for ganeti2009 and 2010 so we keep better tack.

Thanks

Sounds good to me, I'll extend the task description as I run into further affected servers.

RhinosF1 subscribed.Dec 2 2021, 5:18 PM
Papaul updated the task description. (Show Details)Dec 2 2021, 9:26 PM
MoritzMuehlenhoff renamed this task from Installation issues on ganeti2009/2010 with buster / firmware update needed to Installation issues on PowerEdge R440 Ganeti servers with buster / firmware update needed.Dec 3 2021, 11:21 AM
MoritzMuehlenhoff updated the task description. (Show Details)

And one more, added to the task description and ready to power off any time. I should have made an advent calendar out of these, one surprise server for each day of December so far :-)

Papaul updated the task description. (Show Details)Dec 3 2021, 2:26 PM
Papaul moved this task from Backlog to Hardware Failure / Troubleshoot on the ops-codfw board.Dec 3 2021, 4:31 PM
MoritzMuehlenhoff updated the task description. (Show Details)Dec 6 2021, 2:17 PM

One more; ganeti2016. Ready to be powered off any time.

MoritzMuehlenhoff updated the task description. (Show Details)Dec 6 2021, 3:58 PM

One more; ganeti2012. Ready to be powered off any time.

Papaul updated the task description. (Show Details)Dec 6 2021, 5:08 PM
Papaul updated the task description. (Show Details)Dec 6 2021, 7:59 PM
MoritzMuehlenhoff updated the task description. (Show Details)Dec 7 2021, 10:17 AM

One more; ganeti2013. Ready to be powered off any time.

MoritzMuehlenhoff updated the task description. (Show Details)Dec 7 2021, 1:08 PM

One more; ganeti2014. Ready to be powered off any time.

Papaul updated the task description. (Show Details)Dec 7 2021, 6:55 PM
Papaul updated the task description. (Show Details)Dec 7 2021, 7:37 PM
elukey mentioned this in T297422: Installation issues on PowerEdge R440 Kafka main codfw servers with buster / firmware update needed.Dec 9 2021, 6:19 PM
MoritzMuehlenhoff updated the task description. (Show Details)Dec 10 2021, 2:52 PM

One more; ganeti2008. Ready to be powered off any time.

MoritzMuehlenhoff updated the task description. (Show Details)Dec 10 2021, 6:12 PM

One more; ganeti2017. Ready to be powered off any time.

Papaul updated the task description. (Show Details)Dec 13 2021, 3:58 PM
Papaul updated the task description. (Show Details)Dec 13 2021, 5:09 PM
MoritzMuehlenhoff updated the task description. (Show Details)Dec 14 2021, 3:48 PM

One more; ganeti2018. Ready to be powered off any time.

Papaul updated the task description. (Show Details)Dec 14 2021, 10:37 PM
MoritzMuehlenhoff updated the task description. (Show Details)Dec 16 2021, 9:40 AM

One more; ganeti2015. Ready to be powered off any time.

Papaul updated the task description. (Show Details)Dec 16 2021, 3:59 PM
MoritzMuehlenhoff updated the task description. (Show Details)Dec 16 2021, 10:02 PM

One more; ganeti2007. Ready to be powered off any time. It's the last one \o/

Papaul closed this task as Resolved.Dec 20 2021, 10:17 PM
Papaul updated the task description. (Show Details)

@MoritzMuehlenhoff complete

MoritzMuehlenhoff reopened this task as Open.Dec 20 2021, 10:19 PM

Thanks Papaul for the swift turnarounds, much appreciated.

Papaul closed this task as Resolved.Dec 21 2021, 2:29 PM

@MoritzMuehlenhoff no problem closing this task now

MoritzMuehlenhoff mentioned this in T299527: Installation issues on PowerEdge R440 eqiad Ganeti servers with buster / firmware update needed .Jan 19 2022, 2:44 PM
hnowlan mentioned this in T299652: Installation issues on PowerEdge R440 Restbase servers with buster / firmware update needed.Jan 20 2022, 3:42 PM
MoritzMuehlenhoff mentioned this in T303776: cloudvirt1025 and cloudvirt1026 fail to pxe boot.Mar 14 2022, 10:46 PM
MoritzMuehlenhoff mentioned this in T303773: cloudvirt1024.eqiad.wmnet DHCP problems.Mar 15 2022, 9:48 AM
MoritzMuehlenhoff mentioned this in T329363: Upgrade Hadoop test cluster to Bullseye.Jun 28 2023, 12:34 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL