What is the best way to set up network storage?

[ik9000]

After seeing a few friends with centralised storage and music streaming my wife has come round to the idea of setting this up in our house. Particularly so she can access her photos from her laptop if I'm working on the main computer. Can anyone give any pointers on what we need, and the best way of doing this?

If brands are important our router is netgear, as are the switches we have ready to install. We would cable to the router, then presumably use its wifi provision rather than needing it built-in.

Data protection is key - don't want to lose information.
Performance also important.

I've looked at few NAS boxes, but most only offer 4 bays, and IIRC for good redundancy isn't 6 the better bet?

Which drives should we look at? Will HDD suffice or should we just get SSD these days?

[Guy]

For NAS boxes, I'd say Synology hands down all day everyday.

4 bays seem to be the sweet spot still but they offer many models above that. For added redundancy provided you've got a reasonable connection can always backup to Azure/S3/Glacier (insert service of your choice) or replicate to local storage.
For basic storage I can't see a need to go SSD. Just a good HDD that's rated for NAS/enterprise use.

I've got a DS415+ at home and have been using 916+ models for some of our offices.

[MrComputerSaint]

SSD vs HDD, depends on space requirements and budget.
Some manufacturers make specific drives for NAS arrays.

If you're streaming a lot of stuff to several devices then SSD would be good and a gigabit network would be best.
If you're just looking for photo storage I'd imagine HDD would be cheaper cost.
I think for data recovery, Raid 5 would be good as it allows 1 drive failure and I can't imagine the photos being a massive size.
If 1 drive fails you'll have to organise another drive to replace it (If budget allows I'd buy the spare at the same time as you buy the other drives, so you have it on premise). You need at least 3 disks. So a 4 bay NAS would be a good idea.
You could of course go for a 2 bay NAS and RAID1 the 2 disks, so 2 x 2TB drives (eg). You'll have 2 TB of space, but an exact copy of the other drive.

I'd potentially look at a cloud back-up for the data, or some sort of off site backup. As then if there is a natural disaster, fire, act of god, meteorite etc then your photos are backed up.

[Biscuit]

Hardware ex. Hard Drives
As far as base level hardware goes, IMO there are 3 slightly different options worth considering:
1. Pre-built home-NAS. Examples can be found from companies such as Netgear, D-Link, Synology, Western Digital etc.
2. Off the shelf PC hardware designed for NAS. The most well known example being the HP Microserver.
3. Custom built server. NAS hardware in general is just typical PC hardware, same kinda CPUS, RAM, motherboard etc. Obviously you can buy enterprise grade stuff and get extra levels of data protection, or use old stuff lying around to save costs.

The solution you choose very much depends on how much money you want to spend, what you're comfortable working with, what features you require of the device, and how much of a project you want to undertake.

Option 1 is the least effort, as generally you only need to buy the box, the drives and then run through a number of pre-defined menus or wizards to get the system set up the way you want it.
The downside(if limited feature set isn't what you're specifically looking for) is lack of flexibility, as you will often be limited by the pre-defined features the manufacturers selects, although some do come with full versions of operating systems (Linux, BSD, Windows Home Server).
On a bang for buck scale, typically this will be fairly low down. If you want to high performance, big feature set and low cost, you're probably out of luck.
You can find basic boxes with moderate performance and enough features to do the job and there are also options available that come pre-supplied with hard-drives and/or upgradable hardware, but these are usually on the higher end of the market.

Option 2 is probably the best option for the tentative PC enthusiast. If you know you're way around computers and operating systems, but don't want to undertake a big project, then its definitely an option worth investigating. Typically the systems come prebuilt with all internal components, but with no OS or storage devices, so you still have to decide which OS to use (FreeNAS, unRAID, OpenMediaVault etc). Compatibility should always be checked, but typically the popular systems are well supported.
The flexibility and bang for buck propositions are generally very good. There are often great deals on the home server hardware and many of them available on the second hand market. The internal hardware is typically of pretty high quality (often small business grade) and upgradable to a reasonable degree.
The free choice of operating system allows you to decide whether the device is just a network storage, or a full blown mediaserver (or potentially more).

Option 3 is the hardest option to pin down with regards to value, as you can go as far as you want to. Obviously if you use bits and bobs kicking around the house, you save money, but you can also (what I did) scour ebay for enterprise-grade gear thats a couple of generations old, or if you really want too, buy totally new high end supermicro/xeon stuff.
Pretty much everything that is true of Option 2 is true of Option 1, with the proviso that not all hardware is equally supported by operating systems "out of the box" and could take a bit more effort to get working the way you want it too.

Selecting Drives
As far as SSDs go, unless silence is important, I would say its not worth it. A modern spinning disk (or array of them) should offer speeds that will pretty much saturate the bandwidth available on a standard 1gbps consumer grade network switch, so you wont really reap the benefits of an SSD. Certainly for typical media consumption, you don't even need that kind of bandwidth, 100mbps is probably still sufficient even though I would recommend 1gpbs every day of the week.

Cost of capacity is also very much on the side of the spinner still.

Its also worth checking compatibility, as some solutions will not support SSDs as data pool drives.

Most manufacturers these days offer specialised NAS drives, designed to run 24/7. I cant honestly say with any confidence that the drives are different beyond the claims of the manufacturers them self, but I have opted for them when upgrading my server.
I also still run 2TB Samsung Spinpoints and Seagate drives 24/7. These are from long before specialised consumer NAS drives started appearing, and they still get through every single health test without issue.

Redundancy and Backup
Drive Redundancy for the home is something that is, IMO, very much misunderstood.
A lot of people consider redundancy in an array as an alternative to backup, this isn't the case. The redundancy in the array will offer some protection from drive failure, but is NOT an alternative to backup.
Instead of considering the most elaborate 6 drive RAID array possible, I would recommend going for a simple 3 or 4 drive RAID5 array, a 2 drive RAID1 array or no array at all(just present the drives individually on the network) with either an automatic data transfer of vital files to another system, or a very simple data dump onto an external "cold" storage solution to back-up data. Cloud storage is the other option, but "security" is always a bit of a concern here.

To understand the function of drive redundancy, you need to get you're head around the basics of RAID.
As a quick summery,
RAID 0 will stripe data across 2+ drives to offer a single pool storage with potential for increased performance.
RAID 1 will copy the data to 2+ drives, so you have 2+ copies of identical information but you "lose" increased capacity from however many drives you have.
RAID 3-5 will stripe the data across the disks installed (minimum of 3) but one drives worth of data is used as a "parity" to offer the affor mentioned protection incase a drive fails. This gives you the total storage pool size of:
(number of disks)x(capacity of disks)-(single disk capacity) = usable space
The operation between 3,4 and 5 is subtly different and im not going to go into detail here, but essentially you're getting some of the benefits of 0 (increased pool size and performance) and some of the benefits of 1 (data protection).
RAID 6 is similar to 3-5, but with double parity meaning the minimum drives to make it worthwhile would be 6 (IMO) as you lose 2 x disk capacity but gain extra resiliency for a double drive failure.
There is a couple of other variants that combine the above systems in different ways... but you don't need to worry about them.

Typically a RAID controller will not look at the structure of the data it's striping, it just splits it up and puts it onto the disks. This means you are reliant on basically all the hardware in the array to ensure the integrity of the data. The parity information that is stored protects you from a single loss of a disk (or 2 disks for RAID6), as that information, along with the remaining data from the surviving disks can be used to rebuild the bits that were on the broken disk.
Hopefully this explains the reason we see RAID type storage in the home, what the function of the redundancy is and why its potentially not the best way for people to go.

Personally, I opt for unRAID, as this does not stripe the data across the disks, but maintains the individual file integrity and splits it across the installed disks, but still gives you parity. The advantage here is that if you pull a disk out of the array and plug it into another computer, you can still access everything on it, so if I was to experience multiple drive failure, I would still be able to access the files on the surviving drives.
You don't, however, reap any speed benefits that are possible with RAID 3-6 but it does have an option to install an SSD cache drive to increase performance of uploading and downloading of recent or commonly used files.

[AETAaAS]

Cheap and simple ideas here since the above posts have gone into more detail about more robust home solutions.

How much stuff are you planning to store? And what is the usage scenario?

Because if it is for photos and music you mentioned, I'd probably be thinking of cloud storage if you aren't part of Saracen's Tin Foil Hat Posse. If you are planning on putting your stuff on a 24/7 internet enabled system, I'm guessing you're not anyway.

If you have Amazon Prime, they offer unlimited photo storage at no extra charge. With access from PCs and mobiles, it gives you some options, and you can upload from your phone which may be your primary photography device. So it could be more convenient for you and your wife to keep a big gallery. Keep a copy locally always, but Amazon does manage redundancy well. Google Photo is another option, they are unlimited too but limit image/video resolution IIRC.

I use Google Play Music for that side, it allows you to upload then stream.

As you may gather I do rather like cloud storage for some things, especially when access anywhere, anytime is something you might want for the specific file/object. It has saved my bacon when I forgot to bring an updated presentation with me.

I'm wondering what router you have? Cheapo solution if your router supports it is to buy a RAID box, and plug mirrored RAID into your router's USB.

[ik9000]

Haven't been able to read everything yet - still at work, but brief answer to AETAaAS' queries:

Netgear Nighthawk R7500

We have prime and may use some of their cloud stuff - that or google drive or whatever, but we want to have our own store and archive where we can save photos after shoots, relieve the burden on laptop storage drives, and both do stuff without needing to clog up the desktop just to transfer photos off the memory card.

Photo files are large - shooting in RAW and HL JPEG. Trip to Africa = Over 100GB just on that one trip. That's before any photoshop editing and mastering and saving all the PSD files etc.

We'd prob want 3TB for photos, 1TB for files (future proofing), 1TB for videos (don't take that many) and 1TB for music - multi-track cubase sound editing stuff, not my MP3 collection. So 6TB of storage min (before any redundancy). Ideally more than that so it will last a good few years while we save to replace the HDD in 5 years time.

Enterprise HDD was my gut feel, as good 1TB SSDs still aren't cheap enough. As long as that will stream music ok and allow multiple users (2-3 at once) - Alexa just isn't stable enough on our VM modem (puma chipset) so we're having to find our own solution. If it works well it would be nice to be able to link the PVR to it and free up some of the HDD space on there - and so we can view the content on a different machine than just the TV.

We're in process of installing LAN -cat6 (so 1GB)

Biscuit - not read everything yet, but intrigued by last bit, what is unRAID? Is that a brand name?

[Biscuit]

Biscuit - not read everything yet, but intrigued by last bit, what is unRAID? Is that a brand name?

unRAID is an OS based on Linux specifically for the typical "home server" type applications. I'm very happy with it and the only issues I've had have been down to the fact I built a server and the components werent compatible with each other.

You can try it for free but if you do want to go to the kind of scale you're talking about, you're gunna need a license. I went for the $89 "plus" license which allows 12 storage devices (exc. the OS on a USB flash stick)

Here is their website:
https://lime-technology.com/

[jimborae]

In my home setup I use a Synology DS216+ (Raid1 5Tb backup from the server), a Dell T20 running WHS 2011 (2xRaid1 5Tb arrays) as the general server for the house, storing movies, TV, photos general docs etc and then an old HP Micro server (Win 8.1) thats the auto download box. Amazon Fire TV's around the house running Kodi then access the server via NFS share to stream server content. All this is hanging off an Asus RT-AC87u, HP Pro Curve 1800-24G switch and various unmanaged Netgear GS308 switches dotted around the house where needed. Only phones and laptops use the wireless everything else is RJ45. For a network numpty like me this serves the household well.

[peterb]

I use a 'PC' approach running a Linux distro to provide file shareing, mail server and web server (and streaming via a DLNA server) which was fun to build and I learned a lot doing it.

But if you're not interested in the nuts and bolts, the Synology approach is probably the way to go; it will do all the things I have described and a lot more and will almost work out of the box.

I use RAID 1 for my data storage and I backup to an Ultrium 4 tape drive - which has a SCSI interface. tape storage is cheap, but the drives are not, so you may not want to go that route!

Cloud storage is an option, but you are then into a subscription model, and (to my mind) you lose absolute control of your data. Cloud goes down, so does your data.

[AETAaAS]

ik9000, I'll take back my cloud storage suggestion. With that amount of data, it would be pretty inconvenient.

Have you considered Biscuit's suggestions? If you've no spare hardware sitting around to do a storage server with, as Guy and MrComputerSaint said earlier, a pre-built 4 bay NAS is the way to go for ease of use, and having a tidy little box. (Again Synology has my vote for [usually] well thought out systems and well featured software). HDDs are preferable for price/GB and the use case sounds pretty light. Two 8TB drives in mirrored RAID, with 2 bays to spare for future expansion.

If you have a spare PC, the sky could well be the limit.

[Spreadie]

I have this chassis:



...running Windows 7 a fairly low power Intel mITX setup, with two 2TB WD Reds in RAID 1, which backup to another 2TB in bay three. This is backed up periodically to a 2TB USB HDD and another encrypted drive I keep in the car as an off site backup.

If you want to keep the cost down, you can use just about any old PC and install XPEnology on it, which pretty much turns it into a Synology box. Your greatest outlay will be for the HDDs.

[ik9000]

old pcs that we have are:

HP P4 2.8GHz (S478) with 2GB DDR400

Dell Inspiron 530 core2duo DDR2 not sure how much

And a Pentium D + Mobo (without any chassis) and 1GB DDR2

I don't any of them would cut it, and I'm reluctant to sacrifice the Dell at this point - wife's a bit possessive of it. It doesn't have enough mobo connectors for a bank of drives either. Thrifty dell custom job.

[peterb]

If you have a spare PC, the sky could well be the limit.

Or depth of pocket for the drives!

Im using WD reds in my set up - although I built one previously using HGST drives - which have been running pretty much 24x7 for 3.5 yrs without missing a beat.

You don't need a lot of grunt to serve files, especially using a *nix box. You can always start with one of the old PCs - if t is too slow, you can aways move the drives to another platform - you may want to consider a small SSD (or HDD) for the system drive, and keep dedicated drives for data. An SSD isn't really much advantage in terms of boot speed, if you are going to run it permanently - it will reduce power consumption though.

[AETAaAS]

That Pentium 4 sounds a little old, looking around, it is a Socket 478 part and that was the days of PATA, AGP and PCI. Working with that could be more trouble than it's worth. The Dell and the Pentium D sounds more promising.

@peterb, you must be one of the few guys using a tape drive at home! Good point about the drive cost, how are the files stored now? Perhaps a fair amount of the cost is already paid for.

[Biscuit]

http://www.hotukdeals.com/deals/hpe-...ebuyer-2598231

Not exactly the best form factor for the job, but specs and price is pretty much on the mark.

[jimborae]

http://www.hotukdeals.com/deals/hpe-...ebuyer-2598231

Not exactly the best form factor for the job, but specs and price is pretty much on the mark.

Christ I thought my Dell T20 seemed hard to setup, this looks like an absolute pig! Doesn't anyone make a small server aimed at the home-user or small business anymore like the the old Gen 7 N45 & 54L HP Micro Servers?