N7700 Data Failure - PLEASE HELP!!!

[glotgering]

Hello everyone,

I really need some advice on this one, so please chime in. Last night one of my drives failed (Drive 5)....no real worries because I always keep a spare handy. So I replaced the failed drive (Drive 5) and the RAID begain to reuild. Problem Solved Right? Well it would have been except that during the rebuild another drive failed (Drive 3). I did not have another spare handy and even if I did I would be concerned that if I replaced the second drive (Drive 3) without the first drive (Drive 5) rebuilding that I would lose the entire RAID. So I rebooted the Thecus and the RAID starting rebuilding again and I was able to access my data, at least for a while until the second drive failed again (Drive 3).

I know the best solutiong is to HOPE the RAID rebuilds completely and then change the second drive but if the second drive does not stay up long enough what would happen if I change the second drive (Drive 3) with only a partial rebuild on the array.

Please HELP!!!!

[philhu]

I am assuming you have a RAID5. So you must letthe first finish, stay off the raid, until your drive 5 finishes rebuild.

Then replace drive 3 and convert the drive to raid 6

If you lose 2 drive before it is done, you are finished.

If you can, after the reboot and the drive is rebuilding, QUICKLY pull the most important data off the drive. If it is failing quickly, you are screwed.


If you loose another, you might as well pull the plug and start drinking.

[PeteSmith]

I would start drinking now just in case - forward planning. I guess in most cases people will create the raid set with new disks purchased at the same time. You would probably expect the drives to need replacing within a couple of months of each other at the end of there expected lifespan - however it's damn unfortunate if they fail within a couple of hours of each other. What are the chances of that happening?

If you do manage to rebuild the raid array, i would thank yourself lucky and promptly replace each of the remaining drives. I know i would be mortified if i lost all the data on my NAS.

[rhumba]

Hello glotgering -
It might not be your drive, but the N7700 itself!

My N7700 drive #4 failed. With plenty of drives available, I replaced drive #4 with a brand new drive. As soon as I inserted #4 into the stack, a notice went out to me that my N7700 lost drive #3.

Well, fine, I thought, because I have have RAID6 *AND* thought maybe the way I inserted #4 into the stack somehow un-seated my #3. So I replaced my #3.

At this point, my inbox start to received emails by the thousand. Seriously, I had to spent a lot of time just removing my emails. I ended up turned off notification on the NAS.

Waited for three days without seeing any raid return to the NAS, I reset the machine. ... Bad move!

Now my raid is gone. I opened a support ticket with Thecus and there's no result yet.

[glotgering]

Thanks for your input...Here is the latest

I have not tried to power on the Thecus after the last failure. I bought a second Thecus and had it fedex'd in. My plan is to move the data off the old Thecus to the new Thecus.

The problem again is that after replacing drive 5 the raid starts to rebuild and then drive 3 fails before the RAID is rebuilt. I have to rebuild the RAID twice but I am convince that drive 3 will fail again before the raid is rebuilt. So I need to find a way to keep the Thecus up to get my 10.5 TB (9TB data) off (Yes it is full). The new Thecus has 14TB (12TB usable space).

So I have two questions

1) Should I remove the first drive again (Drive 5) and hopefully the RAID will boot in degraded mode giving me a chance to get my data off

or

2) Leave Drive 5 in and start moving as much data before Drive 3 fails and hopefully I can move enough data to allow the RAID to rebuld.

My theory is this. Perhaps the rebulding the RAID is causing drive 3 to fail because it is using some bad and perhaps by moving data to the new Thecus will help.

I have over 4,000 movies, family pics etc...basically all my data and losing this would cause me to jump off the nearest bridge. So please advise which option above is best.

[glotgering]

Well I decided to leave out the first drive that failed (Drive 5) to see if it would come up in degraded mode and it did. I am now moving my data to the new Thecus and hope that I can get enough moved before (Drive 3) fails again. This is going to take over a week due to the amount of data. I am still wondering however if I should re-insert (Drive 5) to see if the raid will rebuild without making (Drive 3) fail or perhaps it is best to leave things alone as long as it is working.

Perhaps now that I have moved nearly a TB of data the rebuild would work. Any thoughts?

[rhumba]

From a bad experience I'm struggling with, it's better off to finish the transfer of your data before doing any physical change to your problematic NAS.

I trusted this thing too much and now might have to face with a lost RAID. Grrrrrrr

[glotgering]

Well after 4 days of feverishly moving data I was able move all my data from the old thecus to the new thecus. I only lost 1 movie "Hancock" in the process. This was the single file that prevented the RAID from rebuilding and was causing drive 3 to fail. Now I will replace the 2 bad disks and then mirror my RAID 5 making it a poor man's RAID 50.

To say that I am relieved is the understatement of the century. On the bright side I now have 2 Thecus's and have 24.5 TB's of storage but if mirror the data I will have considerble less.

Thanks everyone who chimed in. I guess the lesson learned here is there is no substitute for a complete backup and it is important not to panic in the event of a RAID problem. I stayed relatively calm and tried to make good technical decisions and not emotional ones.

YIIIPPEEEE!!! I am so happy!!!