Consider a headless server like this: A typical x86 box at a remote location, which you can remotely initialize with a stock - say - Ubuntu image. After it is initialized you only can login via ssh - or remotely reset it, i.e. you can't access the BIOS or the boot manager prompt (say Grub 1).
Perhaps some kind of KVM is available, but the use of KVM is very expensive and you have to book it on a hourly basis.
Given this scenario, one can get paranoid about boot problems. For example:
- What if a kernel upgrade fails?
- what about a fsck-prompt in the early boot process? Probably, ssh is not available yet ...
Are there other gotchas to watch out for?
For kernel upgrades I configure grub (the legacy one) such that the menu.lst preamble contains
default 0saved
fallback 2 # counts from 0
and the first entry ends with:
savedefault fallback
The first grub entry is the upgraded kernel, and the third is a known working one. See also the grub manual section on fallback boot.
I changed the startup script /etc/rc.local (on a Debian-like system) to the effect that the default-entry setting is reset in case of a successful boot:
grub-set-default 0
This grub-setup works, but e.g. on Ubuntu this is not the default and one has to manually adjust the menu.lst after each kernel update.
I supply
panic=60
as kernel parameter such that, e.g. in case of a wrong root= parameter or broken kernel, the system automatically reboots in case of an error.
About the fsck issue I am not sure what the best way is. On Debian-like systems you can set
FSCKFIX=yes
in /etc/default/rcS, which tells fsck to auto-repair by default.
But if the auto-repair fails, perhaps I still get a prompt I can't access remotely?
Alternatively I could just disable fsck-checking via a zero in the sixth column of /etc/fstab - in case of a fs-error would then just re-initialize the system and restore the backups - thus avoiding all the fsck trouble?
