Several years after I wrote about software RAID1 and RAID5 I must say that I'm happily surprised how much software and hardware support for software RAID under Linux improved. My articles are more or less obsolete now, but this is short story of my RAID5 array for home.
I have Compaq brand-name desktop PC (refurbished, good buy) with one 160Gb SATA disk. I wanted to add three more 500Gb disks in RAID5 array to create ~1TB storage with redundancy.
Motherboard has just two SATA channels and form talking with a lot of people, it seems to me that SATA devices can't be chained (please correct me if I'm wrong). I somehow assumed that I can connect more than one disk on one SATA channel, partly from SCSI world, partly from old IDE (err, PATA) master/slave relationship. So, additional controller was in order, and only one which was available (with two SATA ports and in PCI-X variant) was no-name Sil 3132 based one:
20:00.0 RAID bus controller: Silicon Image, Inc. SiI 3132 Serial ATA Raid II Controller (rev 01)
I googled a bit and found some horror stories (including binary driver on Silicon Image's site for RHEL kernels), but latest Debian kernel 2.6.23-1-686 just worked. I wanted to use it just as SATA controller (with RAID part), so I was all set. It doesn't have RAID5 anyway.
Now, let's take a short de-tour and explain why do I want to have RAID5? Isn't RAID1 better/faster/right stuff for system?
I know only one use for RAID1: booting. And that's if you don't have hardware RAID5 which shows you RAID arrays as one disk anyway. If you want to boot Linux from software RAID5 you are out of luck.
But there is simply no reason to install system on hardware RAID1 as opposed to hardware RAID5. If you have enough disks (five for example) worst configuration is 2*RAID1 + 3*RAID5 since you get maximum of 2 disks accumulated performace. Much better performace (accumulated 4 disks) is just 5*RAID5.
Having said this rand for RAID5, I also didn't want to boot from it (since I have system on 160Gb disk), and I just need reliable and fast disk. Since I could fit just 4 disks in case, I will get 2 disks of accumulated performance and reliability (third disk).
Why not simple striping over three disks (hack, I would get 500Gb of storage more)?
I don't really believe in desktop-grade disks, (even SATA). Those disks are designed to have 8 hours/day life cycle and will probably die in two years. And I like my data...
Let's see some performance, first single disk:
root@brr:~# hdparm -tT /dev/sda /dev/sda: Timing cached reads: 2066 MB in 2.00 seconds = 1033.49 MB/sec Timing buffered disk reads: 232 MB in 3.00 seconds = 77.22 MB/sec
and RAID5 array:
root@brr:~# hdparm -tT /dev/md0 /dev/md0: Timing cached reads: 2030 MB in 2.00 seconds = 1014.76 MB/sec Timing buffered disk reads: 444 MB in 3.01 seconds = 147.49 MB/sec
After several hours of uptime, they are not very hot (order is same as in case):
root@brr:~# uptime
21:53:24 up 5:57, 1 user, load average: 0.00, 0.00, 0.00
root@brr:~# echo -e 'c\nd\na\nb' | xargs -i hddtemp /dev/sd{}
/dev/sdc: WDC WD1600JS-60MHB1: 46°C
/dev/sdd: WDC WD5000AAKS-00YGA0: 45°C
/dev/sda: WDC WD5000AAKS-00YGA0: 48°C
/dev/sdb: WDC WD5000AAKS-00YGA0: 44°C
But they do go up to 55°C under heavy load (specially unhappy sda which is always a bit warmer than other disks).
As you can also see, it moved my old 160Gb disk to sdc. Hack, I'm quite sure that I could force bios into re-ordering this (or turn off RAID BIOS on controller), but I just don't have time.
And lastly how does it look?
root@brr:~# vgdisplay --- Volume group --- VG Name raid5 System ID Format lvm2 Metadata Areas 1 Metadata Sequence No 5 VG Access read/write VG Status resizable MAX LV 0 Cur LV 1 Open LV 1 Max PV 0 Cur PV 1 Act PV 1 VG Size 931.52 GB PE Size 4.00 MB Total PE 238468 Alloc PE / Size 76800 / 300.00 GB Free PE / Size 161668 / 631.52 GB VG UUID onzKEw-TaRF-JBe1-9YWN-D6aJ-FIuY-1VRmLo
I didn't wrote exact commands to perform creation of array, but madm --create --help was basically enough. And that I waited for two hours for RAID array to sync...
There is one more useful tit-bit: if you have ext3 filesystem formated with resize_inode option you can do on-line resize of ext3 filesystem with resize2fs. Debian already does this by default (since etch, I beleve), but you can check with:
root@brr:~# tune2fs -l /dev/raid5/rest | grep resize_inode Filesystem features: has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file
So, if you installed LVM, you can on-line resize your logical volume while coping data to it. I did that while coping from old PATA and USB disks at same time and it works well... It does take much longer (and does more disk I/O) than resize_reiserfs, however.
And don't forget noatime for ext3!