At work almost everything I run boots from SAN. This allows flexability when hardware problems arise. It does cause one issue, when moving storage systems how do you move your boot from san luns. Well there is the old dd trick to do a block level copy between two similar sized luns. This alone will not work, because the wwid of your boot from san lun will change. When this happens the following all become issues:
1. You need to change your bios settings for boot from san wwid (This is a trick provided by modern HBA’s and trick the BIOS into thinking a single lun is a local disk so it can be booted from and is normally done in the HBA BIOS at least for qlogic)
2. You need to identify the lun to your multipathing software (so it’s redundant I will use Linux device mapper which is handled by a kernel driver and the multipath.conf file)
Both of those are straight forward and easy but when you change them it just does not work, why not? Well it’s because of how linux boots and handles multipath devices.
You boot loader has not kernel and as such cannot load data off a multipathed device (it needs a kernel driver to do it) so how does it solve this issue? Well it compiles both the driver and multipath.conf into the initrd. Yep you heard me right your multipath.conf which you have been changing for years without an mkinitrd has a copy of multipath.conf. How does it work?
1. Grub stage 1 hands off to the hard drive partition (single path of your /boot partition)
2. Grub 1.5 loads the initrd which contains the multipath driver and multipath.conf for at least the boot from san lun.
3. Grub 2 loads the normal kernel which then reloads the multipath.conf from disk enabling all other disks listed in there
So to solve this issue when you change the WWID of your boot from SAN lun or optionally any lun you can remake you initrd with the following command:
mkinitrd /boot/initrd-new-boot.img your-kernel-version
For some reason on RHEL when I issue this command with the name of my current initrd and a -f to force it multipath does not get added like it should so use a different name then it works. Then just change your /boot/grub/menu.1st to boot the new initrd and your all set..
How do I tell whats in my initrd :
cd /tmp
mkdir initrd
zcat /boot/initrd-version.img | cpio -id
and you will have an open initrd that you can browse… try looking for /etc/multipath.conf.
You can recompile it with
cd initdir
find . | cpio -o -H newc | gzip -9 > ../initrd.gz
Enjoy
This article taught me that multipath.conf is stored in initial ram disk image. I was facing issue in reverting back multipathing for boot LUN after OVM server upgrade. When I created initial ramdisk (mkinitrd –add-multipath), /etc/multipath.conf was having blacklist entries for boot LUN. So my OVM server could use only single patch device (sd) for booting.
From this article, I understood that I need to create a new initial ramdisk after modifying /etc/multipath.conf. This fixed my problem. Thank you 🙂
Subin,
I am glad it helped you solve a problem.
Thanks,
Joseph