Hard drive failed, reconfigure storage?

[Percy1983]

First things, I run backups and a UPS for the below setup to work safely, so here is my current setup. I am a photographer/videographer for refence. The failed drive is a 2014 drive so not bad going

1, 16GB ram cache, caching the below>

2,500gb NVME PCIE3 boot drive, 250gb NVME PCIE3 cache on the below.

3, 4TB raid 0 array (2x2TB HDs) current work in progress
4TB raid 0 array (2x2TB HDs) storage, games, completed work (one drive failed here).

Hopefully that makes sense and in some ways this has been a historic and evolving setup to which I wouldn't build this if starting all new now.

Plans to fix.

1, get another 2TB HD rebuild/restore continue (not doing this as other drive is similar age), get a new 4tb HD and carry one. around £80.

2, replace with 2x2tb sata ssds in raid, Crucial MX500s currently £127.05 so £254.10 in cost or single 4Tb MX500 for 256.06.

3, replace boot drive with 4TB nvme drive, Crucial P3 plus available for £330 seems a good deal (I understand the limitations on sustained writes).and use other nvme to still cache remaining storage.

In both 2 and 3 I will then have faster/slower storage so will move things to fit better (Ie current work and games on faster with completed work and storage on the slower).

I guess 3 would be good for direct storage in games if/when that happens.

So in summary its £80 with no more performance, £250 for a good kick in performance, or £330 for a rock on performance.

Any input or even other ideas?

[Saracen999]

First, and quick 'off to of head' thought - if one faied drive is 2014 it's likely its RAID partner is too, and very possibly, all four. Therefore, all four are of an age where a failure is getting more likely.

At that point, I'd consider ditching the failed drive (obviously) and retiring the other one in that RAID to be a spare to swap into the other RAID if/when one dies in that. It would seem a good bet for getting maximum life out of those 4 2TB drives.

Then, think through what you need today, and for the foreseeable future. Is your current capacity enough for needs going into the next few years? The 'sweet spot' for drive capacity v price has certainly gone up in 8 years, since 2014. Do you need bigger drives? Is going to NVMe SSD the best use of budget? Is that £330 the limit (-ish) or is there more available provided the benefit justfies the cost?

Two things :-

1) you say you "backup", but don't mention how. Some people regard RAID 0 as backup but it really isn't. What hapens is something happens on the RAID hardware/software, not a single drive failure?

Example - years ago, I had a 4-drive ATA RAID card with 4 separate HDs in it. Something (I suspect either mains fluctuation or lightning strike) blew the PC's PSU and it fried that RAID board and all four drives.

Yes, I saw you have a UPS and it might save you from that specific failure, but a software glitch on the RAID, or even an internal PSU failure won't be helped by a UPS. RAID is about resilience, about uptime, not an alternative to backup.

"Backup" usually means copies/backups on at teast three different media, two different places, and ideally, one off-site.

Backup is (IMHO) always a balance of cost versus level of safety, with a large dollop of convenience thrown in. Other questions to ask yourself are :-

a) What will it cost me if I lose data? Do a shoot for a client, say a wedding, and something fails .... can't reshoot it, can you.

b) What sort of deadlines do you operate on? As a journalist (back then) sometimes doing my own photography, most of the time I had days or weeks of leeway, but more than a few times I was up 'til stupid o'clock because insert long list of improbable coincidenes and it was 5AM with a 9AM deadline. And one of the first rules of backups is that when something goes wrong, it will do so at absolutely THE most inopportune moment.

Therefore, how fast might you need to recover from a backup?

c) Consider the breakdown between data you definitely can't afford to lose, data that would be a pain (like game saves) but not a disaster, and data where you might want an archive copy, but don't need to be able to do rapid restore.


That "archive" category might include old shoots that you want to keep in case you ever get asked for reprints (or might not, depending on what you shoot). In my case, i digitised all my CDs and LPs, Sure, I could do it all over but oh, the time it would take? Don't much fancy that job, a second time.

So think about what data goes where, and what type of backup it might need, if any. You might, for instance, put some on optical media (though be VERY careful with choice of media). I'm still using old magneto-optical (like RVD-RAM) and PD drives for that. Not super high capsacity and I doubt I'd suggest that route now but I have the drives (including a couple of backup drives, sitting in a cupboard, which I switch out every year or so. I use them for "archive" type data where I want longevity but immediate availability isn't necessary.

But I also back up NAS to NAS, NAS to external HD, and other NAS to a different external HD. The latter doesn't live here.

You may already have most of this stuff sorted, but if you're having to spend some money anyway, now is a good time to re-assess what needs to change, and if just going from 2TB RAIDs to 4TB RAIDs still cuts it? It might, but if not ....

[[GSV]Trig]

I've not been a fan of RAID for a while, I much prefer the route that Drivepool goes down where you can have any disk type or size and add them to a 'pool' and have the OS see that pool as a single drive, the drives are all NTFS so if something goes pop you don't lose data in the same way you would in a RAID fail, you can set the duplication of data in the pool on a folder by folder level where you can replicate data in said folder(s) over multiple drives to suit your needs.
Add to that the Scanner software and you can automate the evacuation of drives if they start to throw errors up.

Downside, it runs on Windows, for some that's a downside anyway, not everyone see's it that way.


Obviously, if you have a proper actual backup system then it doesn't matter what you have as your primary storage right?

[kalniel]

Looking at your future plans, there's very little reason to consider SSDs in RAID. I assume you're talking RAID 0 since you mention performance a lot, but larger drives often have higher performance anyway, and reduced overheads compared to a pair.

Then the rest of the conversation is really about performance, not backup since that's why you've configured it as you have and you've said backup is handled elsewhere.

Have you analysed how often you are moving out of your various caches? There's even less reason to worry about squeezing every last drop of performance out of storage or work in progress layers if they're never rate limiting. I'd certainly be tempted to move work in progress and games to an SSD though.

If designing it from scratch I'd probably run something like:
1) Fastest write NVME: Boot, Cache, Work in progress
2) Second fastest NVME: Games
3) Large reliable mechanical: Storage
and I'd leave RAM caching to windows.

So I guess that's most similar to your option 3 - buy a new boot drive and use the existing one as the Game disk (depending on games/how many you store locally vs download as needed).

[Percy1983]

Just updating here.
I have order a crucial P3 Plus 4TB for £281 (direct from crucial).

So this will be my new boot drive + current work + some games

A 4TB raid array will remain with finalised work, documents/pictures/videos/downloads etc and some games.

I will leave the 250gb nvme as cache on the raid array.

As per another post I am upgrading my processor and my old one is going to me son, to which he can now have the 500gb nvme for booting.

To clarify my backups (just for info). I have an 8TB external drive which backs up everything, and a 2TB and 3TB external drives which back up completed work and work in progress. The work in progress drive travels with me if working at my studio and I work directly on it and sync data back to my main desktop when back.

I use freefilesync so it only tracks and makes changes, and do yearly full data comparisons which I do spot the occasional bit rot corruption. So everything is on 2 drives and important stuff on 3 drives.

[Iota]

I am a photographer/videographer for refence.

A belated small thought, wouldn't Raid 5 be a more suitable option if you're using 4 HDD? If for example you're doing photo / video work which is naturally larger data sets, you gain the increase in I/O transfer speeds but also gain in data and parity redundancy (in addition to any backups you may also have). You could probably use an NVMe drive as cache to help, but I'd be seriously concerned about the longevity of using that medium as a cache, when additional RAM would possibly be of more benefit.

[Percy1983]

I have considered raid 5 but the main drawback is the loss of capacity, the main reason I need up with the current setup has been due to an evolution but yes if I planned it form the start I may have done things differently.

Looking at longevity of the NVME cache my 256GB cache drive has just short of 2x the reads/writes of my boot NVME so yes it will certainly die sooner but at a replacement cost of £35(ish) I am not too concerned, meanwhile the boost it adds is noticeable so I plan to keep using it on any mechanical storage for the future.