Software RAID with mdadm utility

Intro

Configuration

• OS : Debian Jessie 8.1

Commands

Reassemble RAID from a LiveCD or after reinstall

• With a two RAID1 disks : sda1 and sdb1 :

root@host:~# mdadm --assemble /dev/md0 /dev/sda1 /dev/sdb1

Populate /etc/mdadm/mdadm.conf file

root@host:~# mdadm --detail --scan >> /etc/mdadm.conf

Remove device permanently

root@host:~# mdadm --zero-superblock /dev/sda1

Check RAID status

root@host:~# cat /proc/mdstat

Run data scrubbing to check for and fix errors

root@host:~# echo check > /sys/class/block/md0/md/sync_action

Modify rebuild speed limits

root@host:~# echo 20000 > /proc/sys/dev/raid/{speed_limit_max,speed_limit_min}

Print details of one or more md devices

root@host:~# mdadm -D /dev/md0

Return status :

• 0 no problem
• 1 error
• 2 complete failure

How To

Extend a RAID5 with the addition of a disk

• With a RAID5 array composed of 3 physical disks : sda1, sdb1 and sdc1 where we would like to add sdd1.
• First thing to do is a complete backup!
• Then unmount (optional, depending of the filesystem used) partition :

root@host:~# umount /dev/md0

• We copy the partition table from an existent disk to the new one (new size disk must be equal or bigger) :

root@host:~# sfdisk -d /dev/sdb | sfdisk --force /dev/sdd

• We add our new disk inside the array :

root@host:~# mdadm --manage /dev/md0 --add /dev/sdd1

• We grow the array size (use of --backup-file is needed and will help in case of failure). Depending of the disk (size and performance) this operation could take a lot of time. We could check the progress with cat /proc/mdastat) :

root@host:~# mdadm --grow /dev/md0 --raid-devices=4 --backup-file=/root/raid5.backup.file

• Then we extend filesystem (works with ext3/4 filesystem.)

root@host:~# resize2fs /dev/mapper/md0

Extend an encrypted RAID (convert RAID1 to RAID5)

• With a RAID1 array composed of 2 physical disks : sda1 and sdb1 where we would like to add sc1 :

• First thing to do is a complete backup!
• Then we do a backup of luks header :

root@host:~# cryptsetup luksHeaderBackup /dev/md0 --header-backup-file path/to/backup.img

• or manually by getting the header size "Payload offset", then save it using the dd command :

root@host:~# cryptsetup luksDump /dev/md0 | grep "Payload offset"

root@host:~# dd if=/dev/md0 of=chemin/vers/backup.img bs=512 count=4040

• For information we can restore header using one of this two commands :

root@host:~# cryptsetup luksHeaderRestore /dev/md0 --header-backup-file chemin/vers/backup.img

root@host:~# dd if=./backup.img of=/dev/md0 bs=512 count=4040

• Stop the RAID (for safety) :

root@host:~# mdadm --stop /dev/md0

• We copy the partition table from an existent disk to the new one (new size disk must be equal or bigger) :

root@host:~# sfdisk -d /dev/sdb | sfdisk --force /dev/sdc

• We convert RAID1 to RAID5 :

root@host:~# mdadm --create /dev/md0 --level=5 -n 2 /dev/sda1 /dev/sdb1

• Then we add the 3rd disk to the array :

root@host:~# mdadm --add /dev/md0 /dev/sdc1

• We grow the array size (use of --backup-file is needed and will help in case of failure). Depending of the disk (size and performance) this operation could take a lot of time. We could check the progress with cat /proc/mdastat) :

root@host:~# mdadm --grow /dev/md0 --raid-disks=3 --backup-file=/root/raid1-5.backup.file

• We resizes the active mapping

root@host:~# cryptsetup resize /dev/mapper/md_crypt

• We can do a filesystem check :

root@host:~# e2fsck -f /dev/mapper/md_crypt

• Then we extend filesystem (works with ext3/4 filesystem.)

root@host:~# resize2fs /dev/mapper/md_crypt