Introduction
md is a Linux kernel facility with user space supporting tools and files.
md was originally "mirror disk". It was renamed "multiple devices" when functionality was extended to more than mirrored disks. Functionality now includes non-mirrored RAID and multipathing.
md assembles multiple devices into a single virtual devive.
The multiple devices are variously called "underlying devices" and "RAID devices" in the documentation. On this WIKI page they are always "underlying devices".
At the time of writing, Blue Light has:
- Only used the Linux kernel facility md to create RAID 1 devices.
- Has preferred the traditional /dev/md[[:digit:]]* device file names.
TODO: better we move with the times and start using /dev/md/* device names as the installer does?
RAID 1
Creation
Create the block devices to be used as underlying devices. Typically these are HDD partitions but any block devices can be used. In the following example, /dev/sdb1 /dev/sdc1 are used.
mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
The md device may now be used like any other block device except when:
- Used for /boot The section "Booting from an md device" below applies.
- Used as an LVM PV The section "LVM PVs on md devices" below applies.
Replacing an underlying device
Typically this is done when an HDD has failed.
In case the replacement HDD is the same size and has the same partitions as a working HDD, the partitions can be created and GRUB installed in a single command. Assuming sda is the working HDD and sdb the replacement:
dd if=/dev/sda of=/dev/sdb bs=512 count=1
In other cases the block device for the new underlying device must be created as appropriate.
Regardless of how the underlying device was created, it can now be added to the md device. Assuming the underlying device is sdb1 and the md device is md0:
mdadm --manage --add /dev/md0 /dev/sdb1
Remove an md device
Removing /dev/md1 as an example ...
Find which block devices the md device is built on
mdadm --detail /dev/md1
Stop the device:
mdadm --stop /dev/md1
For each block device the device is built on (here /dev/sda2):
mdadm --zero-superblock /dev/sda2
Finally remove the device:
mdadm --remove /dev/md1
Booting from an md device
TODO: confirm and clarify this section.
Assuming a RAID 1 md device is being used for /boot, the only mechanism available before GRUB2 was for the boot loader to load initrd (or the kernel directly) from one of the underlying block devices in the normal way and then assemble the /boot md.
GRUB2 can be configured to include md support so can assemble the /boot md before loading initrd (or the kernel directly).
Start all md devices
Some linux system integrated an mdadm tool called mdadm-startall. You can install it if not already done.
It doesn't require any argument and will assemble all the md devices found.
mdadm-startall
Troubleshooting
initrd fails to assemble md devices
When initrd is responsible for assembling the md devices, it needs to be rebuilt when the md devices (not their underlying devices) change, for example when an md is removed and a new one created with the same name.
If this has not been done and the md devices were required to boot, a busybox prompt is presented. mdadm --assemble --scan was not effective. The md device had to be created by running mdadm --assemble /dev/md1 /dev/sda2 /dev/sdb2
When the system has been booted, the damage can be fixed by:
- Updating /etc/mdadm/mdadm.conf to include current md UUIDs (updated lines can be generated by running
mdadm --detail --scan). - Updating the current initrd by running
update-initramfs -u
LVM PVs on missing md devices
After fixing the missing md devices as above, the VGs and LVs can be enabled by running the busybox command lvm and then using its help to find the commands to do so (cut down versions of the GNU Linux equivalents).
References
- Overview: md man page
- mdadm man page
- mdadm.conf man page