Large/scaleable NAS recommendations

[Olimain]

Hi all,

I need to implement a NAS system for our growing video library and am looking for recommendations please. I'd probably look to start with around 12TB of storage but want to be able to scale that up to 100TB+ in the future so ideally looking for a system that can be daisychained by adding similar units. For now we'd use GbE for the in-house network but would want to be able to upgrade to 10GbE and would also require an eSATA port (and it has to be rack-mountable). I was planning on running it as RAID10 for performance and redundancy.

The average file size of our archive is 4.5GB and there could be up to 20,000 files so I'm not sure if that makes a difference to the RAID system we should be running compared with if we had millions of tiny files. It's just for storage rather than for shared editing so we don't need a SAN/fibre (for now!). I've been looking at these:

http://www.scan.co.uk/products/qnap-...-gigabit-rj-45

But can't find out if they can be daisychained. Does anyone here have much experience of this sort of thing?

Thanks in advance as always!

Oli

[Moby-Dick]

whats your budget?

[Olimain]

I was hoping to do it for around £2k+VAT

[Moby-Dick]

I'm not sure you'll get something off the shelf for that - though I suspect if you are prepared to spend a bit of time rolling your own, it could be possible.

As a little inspiration , have a look at the kind of stuff the backblaze guys have done. ( I'm not suggesting you design your own case , but a supermicro box with some disks in it running a NAS distro could be a pretty good start )

[shaithis]

http://www.scan.co.uk/products/qnap-...-gigabit-rj-45

But can't find out if they can be daisychained. Does anyone here have much experience of this sort of thing?

I've had a look at them (I deal with storage a lot at work: QNAP, Synology and Equillogic mainly)....it doesn't look like they daisy-chain as they don't have the port for it that older QNAPs do, although perhaps the software allows it via the NICs (2 slots for additional NICs on those units).

The one thing I will says is that I would never purchase or recommend a QNAP again after the issues I have had with RAID rebuilds after a failed drive. The Synologies and Equillogics (Dell) are considerably more robust from personal experience, Thecus might be worth a look as well, although they are less likely to have chainable units.

[Olimain]

Thanks for the replies. I don't mind spending a bit more than that if necessary to get something robust that we can expand.

Thanks shaithis for the recommendations to avoid Qnap - that sounds like a nightmare we really don't want! Would you be more inclined to go for something like the Synology RS2212RP+ then as that seems you can add a 12-bay unit on top in theory giving you 22x4TB drives?

[DanceswithUnix]

RAID 10 sounds a bit expensive.

100TB in 3TB drives means ~33 hard drives. Lets be dumb and stripe the lot for raid 0 performance (should be quick!), now we need another 33 for a raid 1 duplication. ~66 drives in total.

If you go with raid 5 you might get away with ~40 drives, plus you want a few spares for when they die.

In your basic setup, a raid 5 controller with 5 x 3TB drives would get you 12TB out of the 15TB total. Some would say you want raid 6 which adds another disk to make it more reliable, that is your call as to how much you value your data.

Then there is the problem of how do you back up even just 12TB. A second NAS with replication can work, but instantly doubles the cost.

Can't really recommend a specific setup though, I work for Hitachi and whilst it is excellent kit I think we are rather out of your budget

[wasabi]

RAID6 with a hot spare is probably saner in a home environment. I'd buy a big old fileserver chassis personally and stick a linux fileserver distro on it along with a bucnh of sata addin cards.

[Olimain]

Yes I see what you mean about RAID10, I'd always assumed I'd use that to give both performance and backup so hadn't really looked in much detail at the other options. I need to have a closer look at the speeds attainable with RAID5/6. As I say - it's just for storage rather than constant multi-user access though we will want to be able to copy a ~10GB file quickly to another machine as and when we want to transcode it/edit it.

We have the data backed up on video tape in a different building. Do the drives go regularly in RAID5/6 then? Regularly enough to need plenty on the shelf ready to replace ?

[DanceswithUnix]

It isn't that drives are all rubbish and die after five minutes, just a side effect of the scale involved.

If you have a million drives, then a million to one chance starts looking rather like a certainty

Drive fails should be OK as long as you replace them within their rated lifetime, but guessing a 2% chance per drive with 40+ drives it would seem good to have a spare or two.

As for performance, my cheap Linux box at home can max out a gigabit ethernet port from a single drive. Performance is far more of a concern if the files are small or you have lots of users pulling files at the same time.

Are you intending to pull lots of streams from this machine, or is it more of an archive?

[Olimain]

It's definitely more of an archive and we won't be either streaming files directly from the NAS nor will we have more than a couple of users copying a file from/to it at a time. I am however also hoping to use it for our FTP server (currently the folder structure for this is on a cheap 4 bay array).

We hold video libraries for our clients and for reliability and durability the masters are kept on tape, however 9 jobs out of 10 require some kind of digital file which involves us encoding from the master tape and uploading to a third party. Due to our digital storage capacity we periodically delete these but really it makes sense to keep this digital copy rather than remove it after each use as it will save capturing from tape in real time each time a video is required. We will still be keeping the masters on tape however we can quickly and easily build a digital library for quick access this way.

[shaithis]

Thanks shaithis for the recommendations to avoid Qnap - that sounds like a nightmare we really don't want! Would you be more inclined to go for something like the Synology RS2212RP+ then as that seems you can add a 12-bay unit on top in theory giving you 22x4TB drives?

That Synology is a nice unit, I have used a few of them and they are problem-free with good performance. Nothing spectacular but then for spectacular you need to spend silly money.

Still, at ~£1700 with NO drives.......I do wonder as to it's value in a home environment and would be inclined to make my own.

[Olimain]

I take your point and we almost always build our own workstations in order to get exactly the spec we're after but really on this occasion we don't mind spending more to get something that will work almost straight out of the box, with a warranty and that we can easily add to - this seems to fit the bill.

Do you know if we can half-populate them and add more drives as we need storage? Say we went RAID6 and opted for 6 x 4TB drives, giving us 16TB raw space with 2 drives for parity and 4 slots free. Can I simply use the NAS software to expand that and add another 4 drives/16TB in a few months' time without any data loss?

[shaithis]

Yes, they support just about every case of raid migration and expansion possible.

[Olimain]

Thank you for your help, much appreciated.

[abaxas]

You need a maid. Not the frilly knicker type but a massive arrary of inactive disks.

Get someone in to talk about this. This aint a hexus question.