...
If the md device provides the file system including /boot or is an LVM PV (physical volume) then /etc/mdadm/mdadm.conf must be re-populated as shown below and the initramfs re-generated by:
update-initramfs -u
EFI System partition (ESP)
EFI only recognizes FAT16/FAT32, so according to most sources, RAID 1 is not possible. However, that is not true: RAID 1 is possible as long as metadata version 0.9 or 1.0 is used. The reason it works is because metadata 0.9 or 1.0 is stored at the end of the partition, which doesn't interfere with EFI, whereas metadata 1.2 does because it's stored at the beginning.
To convert the EFI System partition device to md RAID ...
Back up the current ESP files and unmount the file system: cd /boot && tar -cvzf efi.tar efi umount /boot/efi
Create a RAID array: mdadm --create --metadata=1.0 --verbose /dev/md1 --level=1 --raid-devices=2 /dev/sda1 missing
"missing" allows you to specify that a second device isn't present yet .
Format the partition (you might need to install dosfsutils package first to get the command):
mkfs.vfat /dev/md1
Update /etc/fstab's /boot/efi line with the new file system's UUID:
blkid | grep md1
... and edit /etc/fstab, inserting the UUID into the /boot/efi line.
Restore the ESP files:
cd /boot && tar -xvzf efi.tar
Removal
For example:
mdadm --manage /dev/md2 --fail /dev/sd[ab]1 mdadm --manage /dev/md2 --remove /dev/sd[ab]1 mdadm --manage /dev/md2 --stop mdadm --zero-superblock /dev/sd[ab]1
...
mdadm --zero-superblock /dev/sda2
TODO: is this next step necessary or even possible (it was not in PTDC-361)
Finally remove the device:
mdadm --remove /dev/md1
Booting from an md RAID 1 device
...
- Before Jessie (TODO: Trusty same?): use kenel parameter bootdegraded=true
- Jessie (TODO: Trusty same?): create /etc/initramfs-tools/conf.d/mdadm containing "BOOT_DEGRADED=true" and rebuild the initrd with updatecopy /usr/share/initramfs-tools/scripts/local-top/mdadm to /etc/initramfs-tools/scripts/local-top and patch it as described in https://jira.bluelightav.org/browse/FINANCESRV-368?focusedCommentId=53091 then rebuild the initramfs with
update-initramfs -u
When a RAID 1 md device is used for /boot, GRUB should be installed to both underlying devices, not to the md device. This is done using
dpkg-reconfigure grub-pc-grub
GRUB2 can be configured to include md support so can assemble the /boot md before loading initrd (or the kernel directly) but Blue Light accepts what the distro delivers.
...
/usr/share/mdadm/mkconf > /etc/mdadm/mdadm.conf
It is prudent to rebuild the initramfs after changing mdadm.conf:
update-initramfs -u
mdadm syntax table
In case mdadm --help and the mdam man page are not what is needed.
...
Mode | device before options | Options | Notes |
|---|---|---|---|
| --assemble | Y | bitmap, uuid, super-minor, name, config, scan, run, force, update, no-degraded | |
| --build | Y | ||
| --create | Y | ||
| --grow | N | ||
| --incremental | No options | ||
| --misc | N | ||
| --manage | Y | add, remove, fail, set-faulty, run, stop, readonly, readwrite | --manage may be omitted |
| --monitor | No device |
Troubleshooting
Degraded array report
The system my detect a problem with the array and drop an underlying device. This is normally reported by mail and by Nagios.
It can be confirmed and the excluded underlying device (partition) identified by cat /proc/mdstat
Defects in the physical device may be identified in kern.log and by smartctl -a /dev/sd<letter>
In case smartctl output shows extended tests are not being run (old build computers), update /etc/smarttd.conf from git and edit to suit.
In case no defects are found, there is no choice but to add the excluded underlying device back into the md device as documented under "Replacing an underlying device" above.
If defects are found, ensure the underlying device is not used again by zeroing its md superblock as documented under "Remove an underlying device (md device not in use)" above.
In one case (bafi), during boot an array was assembled from a defective underlying device and the good underlying device excluded.
initrd fails to assemble md devices
...
Populate /etc/mdadm/mdadm.conf as described above.
nvidia: wrong # of devices in RAID set "nvidia_gbbagfad"
This is normal on some motherboards with an Nvidia chipset with BIOS-provided RAID which, even when disabled, seems to lead to these messages. We have not found a way to disable them.
RebuildFinished event detected on md device *, component device mismatches found: * (on raid level *)
...
- Overview: md man page
- mdadm man page
- mdadm.conf man page
- kernel.org WIKI: https://raid.wiki.kernel.org/index.php/RAID_setup
- Wikiversity (nicw nice summary): https://en.wikiversity.org/wiki/Linux/mdadm