Replace a faulty boot media without losing ESXi configuration

One of the ESXi’s I have in the lab started reporting bootbank errors.

Simultaneously I noticed that power operations, VM snapshots and some other tasks stopped going through. This is when I knew that it was time to replace a decade old USB stick that this host was booting from.

Issue detected on in Datacenter : Bootbank cannot be found at path '/bootbank' (2022-02-19T02:07:40.880Z cpu1:744685)

After a reboot, host would usually work normally for at least a day so it was enough to back up its configuration via PowerCLI:

My servers don’t have any monitors or peripherals plugged in, so I wanted to make it work without having to look around for cables. I performed all the steps below on a different ESXi but it should be possible to use the same host with minimal adaptations.

So the next step is to create a VM, insert a new USB stick in the server and pass it through to the VM:

Next we boot from ESXi installation ISO which has the same build number as the original host and go through installation steps:

The issue now that there doesn’t seems to be a native way to boot a VM from a pass-through USB device. So the workaround is to use Plop boot manager as intermediary. Which is basically a bootable CD that allows you to select a USB device to continue the boot with. The latest available version is 5.0.15 that was released almost a decade ago.

Nested ESXi successfully booted and picked an IP via DHCP. Next step is to apply the configuration that has been extracted from the original host:

Set-VMHost -VMHost -State ‘Maintenance’
Set-VMHostFirmware -VMHost -Restore -SourcePath "C:/configBundle-"

PowerCLI warned me about some mismatch between the original host and the new one but ultimately applied the configuration:

Set-VMHostFirmware : 2022-02-19 3:59:00 PM	Set-VMHostFirmware		fault.MismatchedBundle.summary	
At line:1 char:1
VMHost               UploadUrl                                                   
------               ---------                                            

I removed the original ESXi from vSphere inventory, replaced its faulty USB drive with the one that we just prepared and booted the server. Then I re-added the host in the inventory and it has been automatically placed in its original cluster. Local datastores and all other configurations were restored to their original values.

2022-03-08 Update: Looks like a “missing bootbank” error is actually a known issue that has been resolved in ESXi 7 Update 2c.

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.