Regain control of a rogue XCP Server

In the past year, I have constructed a small private cloud from four Core-i7 machines and Xen Cloud Provider.  Recently, one of my servers suffered filesystem corruption such that somehow its root password was mangled.  If you are lucky enough to have physical access to the machine, then you can try this procedure, which I must note is not much different than that for any normal Linux box, except in the specifics of the bootloader.

1. Boot your server as usual, but you need to get to the SYSLINUX prompt by pressing the <shift> key after BIOS POST.


2. You will then see the Syslinux boot: prompt which is asking you for which kernel it should boot.   You need to then type menu.c32 <enter>.  Be warned that a hidden boot timer is ticking at this prompt, but it's reset every time you type a key.


3. You will be presented with a menu for booting into e.g., regular XCP, safe mode, etc.  Select the first entry (xe) with the cursor keys (it should start on that entry anyway), then press <Tab> to edit the boot parameters.


4. You should see something like the following:
   
   append /boot/xen.gz dom0_mem=752M lowmem_emergency_pool=1M crashkernel=64M@32M console= vga=mode-0x0311 --- /boot/vmlinuz-2.6-xen root=LABEL=root-mlbdhhzz ro xencons=hvc console=hvc0 console=tty0 quiet vga=785 splash --- /boot/initrd-2.6-xen.img


5. You want to insert the single keyword for the dom0 kernel like so:
   
   append /boot/xen.gz dom0_mem=752M lowmem_emergency_pool=1M crashkernel=64M@32M console= vga=mode-0x0311 --- /boot/vmlinuz-2.6-xen single root=LABEL=root-mlbdhhzz ro xencons=hvc console=hvc0 console=tty0 quiet vga=785 splash --- /boot/initrd-2.6-xen.img
   
   
   I also removed the quiet and splash options so that I can see if anything strange is happening during the boot process.

This should drop you at a single-user root prompt from which to do your analysis and repair.

YMMV!