RAID5 + hot spare or RAID6?

[Xaar]

I've recently built a 15tb archive server using an Areca 1170 24 port sata raid card, which supports RAID5 and 6.

A quick google revealed that raid6 suffers a significant performance hit compared to raid5, so I decided to go with RAID5 with a hotspare.

Unfortunately, I've already experienced 2 disk failures in the space of 2 weeks, and with a rebuild time of time 10hrs+ between failures, i'm starting to feel nervous. Did I make the wrong choice?

Performance is an important issue.

Has anyone here any experience with a RAID6 setup?

I've googled around, but cant find many specific details like max sequential read/write times.

Would I be better off rebuilding it as a RAID6? Bearing in mind I currently have ~7TB of data on there, which i'd have to move elsewhere and back between rebuilding

[arthurleung]

I've recently built a 15tb archive server using an Areca 1170 24 port sata raid card, which supports RAID5 and 6.

A quick google revealed that raid6 suffers a significant performance hit compared to raid5, so I decided to go with RAID5 with a hotspare.

Unfortunately, I've already experienced 2 disk failures in the space of 2 weeks, and with a rebuild time of time 10hrs+ between failures, i'm starting to feel nervous. Did I make the wrong choice?

Performance is an important issue.

Has anyone here any experience with a RAID6 setup?

I've googled around, but cant find many specific details like max sequential read/write times.

Would I be better off rebuilding it as a RAID6? Bearing in mind I currently have ~7TB of data on there, which i'd have to move elsewhere and back between rebuilding

The disk configuration really depends on you. If it consists of more than 6 disks, it is much safer to use RAID6, as RAID5's protection diminishes towards RAID0 as you increase the number of disks.

Just judging from your quoted value I assume you're running 2x 8x750GB + 1x 7x750GB RAID5 + 1 Hot spare which gives 15TB. In my opinion 4x6x750G RAID5 with no hot spare, perhaps with 2~4 cold spare would give a slightly better performance, where you can soft-RAID0 two (or 4) of the RAID5 array to get a decent performance without the overhead of RAID6. Or alternatively just use 3x8x750G RAID6. Even more redunduncy but slower performance.

If speed is really such a concern to you, then you should have bought two of the 12 (or 16) port cards which gives a faster combined CPU speed)

About your disk failure, it is quite usual to get that even for single drives, some drives die shortly after being used, basically in the past 4 years I've build half a dozen of 4~6 (new) Disks RAID5 and 50% of the time there is a bad disk die about 24 hours into operation or even dead on arrival. Once these disks are replaced, the array doesn't fails even after 2~3 years.

If you want some review as regarding to speed, I remember there are plenty of RAID card benchmark made by users in 2cpu forum, and to a lesser extent storagereview forum. As far as I remember the speed penalty is not large. These Areca cards have very powerful CPU so it is far more than enough to saturate dual GbE even in RAID6. For write performance, max out the cache and you'll not find any trouble. Remember these cards can put out 1GB/s of RAID5 performance if you give it some Raptors alike disk, and RAID6 will still be plenty fast even with 50% performance penalty.

Back during installation stage you really should have tried both RAID5 and RAID6 speed, and make sure the disks are working properly before you put it into real use. Usually it will take 3~10 days (especially with your huge array).

The rebuild time you've quoted is normal as RAID card disable write cache during rebuild stage to ensure data integrity, I doubt you could do much about it.

Though I have to say I've never tried any Areca RAID (Price too high), but I have 3Ware RAID cards running in a couple of servers (7xxx IDE, 8xxx SATA, 95xx SATA, 96xx SATAII). So far all running RAID5 with no more than 6 disks in the same array. All have been running perfectly fine, with occasionally disk dropout about once a year but work fine after rebuilding.

[Xaar]

I'm actually running it as a single raid5 (21 x 750gb + 2 hot spare).

Unfortunately, I had time constraints, so didn't get the chance to run multiple tests.

I had considered doing it as 22 x 750 raid6 + 1 hot spare, but was put off after reading a few reviews of raid6 performance.

The server is connected to an infiniband switch at 20gbit, and serves as a temporary archive store, prior to being laid off to lto3's.

Is there a disadvantage to running it as a single large array? Apart from the obvious chance of failure