Tag Archives: Replace Faulty HDD in Raid1

Replacing A Failed Hard Drive In A Software RAID1 Array

Note: In this example I have two hard drives, /dev/sda and /dev/sdb with partitions /dev/sda1 and /dev/sdb1 as well as /dev/sda3 and /dev/sdb3.

1. Check raid health by typing this command “cat proc/mdstat” in terminal.

[root@linuxbox /]# cat proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda3[0]
459025152 blocks [2/1] [U_]

md0 : active raid1 sdb1[1] sda1[0]
20972736 blocks [2/2] [UU]

In the result above, your will noticed there is “_” or underline which means, sdb2 partition failed at md1.
In this case we have to change sdb device.

2. Install hdparm to look for the serial number of the faulty disk.

[root@linuxbox /]# yum install hdparm -y

[root@linuxbox /]# hdparm -i /dev/sdb

/dev/sdb:

Model=ST3500320NS   , FwRev=SN05    , SerialNo=   123HF45

 

Note: Take note on the serial number (mine is 123HF45), you need this serial to determine which HDD should be take off later.

In the sticker of the HDD there will be a serial number indicated. This is to make sure that you have taken out the correct faulty HDD.

3. We will set /dev/sdb device to fail. Follow the command below.

[root@linuxbox /]# mdadm –manage /dev/md0 –fail /dev/sdb1
mdadm: set /dev/sdb1 faulty in /dev/md0

[root@linuxbox /]# mdadm –manage /dev/md1 –fail /dev/sdb3
mdadm: set device faulty failed for /dev/sdb3:  No such device #In this case it will show a message “No such device” cause /dev/sdb3 failed already.

The output should be the same as below.

[root@linuxbox /]# cat /proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda3[0]
459025152 blocks [2/1] [U_]

md0 : active raid1 sdb1[2](F) sda1[0]
20972736 blocks [2/1] [U_]

unused devices: <none>
You have new mail in /var/spool/mail/root

4. We will then now set /dev/sdb1 and /dev/sdb3 to remove from the raid arrays.

[root@linuxbox /]# mdadm –manage /dev/md0 –remove /dev/sdb1
mdadm: hot removed /dev/sdb1

[root@linuxbox /]# mdadm –manage /dev/md1 –remove /dev/sdb3
mdadm: hot removed /dev/sdb3

[root@linuxbox /]# cat proc/mdstat
Personalities : [raid1]
md1 : active raid1 sda3[0]
md0 : active raid1 sda1[0]

5. Shutdown the server and add the new hard disk.

Adding The New Hard Disk

The first thing we must do now is to create the exact same partitioning as on /dev/sda.

[root@linuxbox /]# sfdisk -d /dev/sda | sfdisk /dev/sdb

After run the command below to check if both hard drives have the same partitioning now.

[root@linuxbox /]# fdisk -l

If both disk have same partition then you may proceed to step 5.

6. Next is to add /dev/sdb1 to /dev/md0 and /dev/sdb3 to /dev/md1.

root@linuxbox /]# mdadm –manage /dev/md0 –add /dev/sdb1
mdadm: re-added /dev/sdb1

root@linuxbox /]# mdadm –manage /dev/md1 –add /dev/sdb3
mdadm: re-added /dev/sdb2

It will automatically sync, you may type this command cat /proc/mdstat and you will see the output below.

That would be all, you have successfully replace /dev/sdb!