LVM Shrink to remove a device

Question

I have Centos 7

I have a 7 drive RAID6 Array created using mdadm. (This is composed on WD 4TB Red drives)

I have setup the RAID array as a LVM PV - all this has been working well for ages.

I have almost maxed out the drive space on the LVM - what i would like to do is to add additional space as follows

1) Purchase 2 x WD RED 6TB drives, create a RAID1 set from them under mdadm 2) Present that RAID set to LVM by creating a PV on it 3) Use that additional 6TB of disk space to extend the LV

I do not however want to make the LV 6TB bigger, the end result i want to achieve is to remove one of the 4TB drives from the RAID6 array, thus effectively increasing the total disk space by 2TB.

I believe the high level steps i need to take are as follows:-

1) Create a RAID1 array (call it /dev/md1 e.g.) composed of the two 6TB drives 2) partition the 6TB to have a LVM partition 3) create a PV with the 6TB of space 4) Add the PV to the existing VG 5) Move an appropriate number of extents to the 6TB PV to free up 4TB of extents in the RAID6 VG

This is now where i start to get a bit vague - i THINK i would

6) Shrink the RAID6 VG to ensure that 4TB of disk space is available 7) Reduce the number of disks in the RAID6 array 8) Shutdown the box and remove the retired drive ?

I need to flesh out 6), 7), 8) Please and let me know if i am on the right track

The RAID6 array is healthy - i will move the retired drive across to a secondary box that performs backup functions.

Answer 1

Your description is more or less correct.

However, this is quite a process: you have to pvmove extents currently located in the last 4TB of the PV, then reshape the entire RAID6 array (note there just was a RAID6 corruption bug in kernel 5.1.x prior to 5.1.5), and cross fingers and hope nothing will go wrong anywhere.

the end result i want to achieve is to remove one of the 4TB drives from the RAID6 array, thus effectively increasing the total disk space by 2TB.

Unless you have plans to get rid of all 4TB drives, I would do it a different way: partition the 6TB drives to participate in the 4TB RAID6 array.

Since you want to remove one 4TB drive, use mdadm --replace to replace it with (a 4TB partition on) one of your 6TB drives. Then you're free to move the 4TB drive to another box, the size of the RAID6 array stays the same.

Then you mdadm --grow --raid-devices=8 --add (a 4TB partition on) the other 6TB drive. Your RAID6 PV grows by 4TB in this step.

At this point only 4TB are used on your 6TB drives. The remaining 2TB partitions can be used to build a RAID1 array, adding another 2TB of useable storage.

Thus effectively increasing the total disk space by 6TB, and 4TB of those are covered by 2-way redundancy.

All my RAID arrays are built like this. I partition all my drives into same-sized segments so I can mix and match drive sizes and maximize storage efficiency without sacrificing redundancy; it also allows me to run dangerous reshape operations on a small segment of data first instead of risking it all in one shot; and if one drive goes bad in one place, only the affected segment degrades while other segments stay on full redundancy until you --replace the offending drive.