Large raid 5 storage needed.

[peterb]

Most has been said before - but if it is a legal requirement for compliance etc - then doing it cheap isn't an option (unless your director wants to end up with a big fine when his cheapo solution doesn't produce the goods)

To restate - RAID is fine for online resilience - but you need to have a proper backup regime - and you need to consider backup frequency - you might think monthly is OK - but depending on the volume of documents, weekly might be an option. Your backup needs to be done twice - once for on site retrieval, and again for offsite archive - offsite can be another office - if you have one - preferably in a fire proof safe. Your onsirte backups should ideally be in a fireproof container too.

This is not a trivial subject - there is a world of difference between backing up a commercial enterprise where you have legal obligations, and a backup system for a home system.

Oh yes - tape is really the only way to go - stored properly, a tape should last 20 years - DVDs may not last more than 5.

One other option might be to microfilm the documents although even microfilm is bulky. However it is durable! As for tape - probably some form of LTO system - Ultrium 2, 3 or 4 - each tape will store 200, 400 or 800 Gb uncompressed - they have hardware compression built in, so you can pretty much double that capacity.

Think about the on costs too - an Ultrium 2 tape costs between £15 and £20 - and you need one for each backup (don't reuse them or use one for multiple backups)

You wil also need to keep an index (probably manual in a book) so you can retrieve the tape when you need to access the documents.

Remember - if you are backing up weekly, you are looking at 50 or so tapes a year - 1000 over 20 years - and that too will be bulky. Backing up monthly might be more realistic.

But you need to sit down with your director and identify exactly what the requirement is - and then build a solution to meet that requirement.

[Saracen]

I go along with peterb - this is not something to do without careful thought, because there are a number of issues involved. It's generally about a whole series of issues, not least of which are availability and compliance requirements, but you may (depending on your exact situation) also need to be VERY careful about audit trails.

If you have a legal duty (as it seems you do) to archive certain documents, are you under any legal obligation as to the nature of the archive? Some requirements stipulate that copies must be on an uneditable media, so you may be looking at something like WORM optical or tape storage.

In any event, it sounds like you need a hierarchical system that will provide by availability (and that may indeed be a RAID box) but with that backed up to something like tape or optical that gives long-term storage, but you may also need to be able to track documents through that system using some form of audit trail. On top of that, you need to think about security to ensure that only those that should be able to access the archives can, as well as management software to ensure that you can find stuff on the archive when you need to. You perhaps could do worse than looking at someone like Plasmon as a starting point. But £1000? I doubt it.

[b0ned0me]

Have you thought of just outsourcing the whole gubbins? There are companies you can send your documents to, they'll scan them for you and send them back on DVD or whatever and keep a copy on tape. Or alternatively they'll just let you access the softcopy documents via secure webpage.

That way you can have the PDFs on your LAN server or whatever for convenience, and palm off the legal responibility for the archived copies to someone who specialises in this sort of thing. Building a whole secure document archiving system is probably a bit much when by the sound of things you should be putting your energies into establishing a proper backup strategy for your PCs & servers - I'm assuming there isn't one.
Once you have your whole daily/weekly/monthly/offsite backups with DR working properly and your day-to-day business is protected against fires/theft/lightning strikes/the electricity company, you can probably just insert the additional terabyte or so of PDFs into that process without too much difficulty.

[tfboy]

Another question you need to ask yourself apart from the physical storage of all the data, how are you going to reference it?

If it's 20 years worth and you reckon it's going to be around 1TB of data, that'll be several thousand documents. How do you plan to index / categorise all that so that when 5 years down the line you need to dig out a specific document, you'll know exactly how to find it.

The decision of how you categorise it will be impacted on the physical platform you use to store the data.

As you've said it's for legal reasons, as others have said, a simple home RAID solution isn't at all what's required. You need a professional solution to keep all the data backed up, and a professional way of accessing it.

[Fatboy40]

It sounds like you'd be well served with a Hewlett Packard DL360 G5 server + an MSA-20 array + an LTO4 drive (that can do WORM).

Should get you 1TB of storage for no more than 10k, if you're a business that has a turnover of 5 million or more than you need to go down this route and not spend just 1k on a solution, you need to slap your cheapskate M.D. round the head with kipper

[peterb]

Following on from Saracen's comment about legal requirements for archived data (not being editable) - as Fatboy40 says, Ultrium 4 has a WORM capability - you might also consider password protecting the final PDF files, or perhaps digitally signing them at some point so you can show that they haven't been tampered with. Another possibility is to scan the documents as images and store them as jpegs - again uneditable, but you have the added hassle of scanning them.

You may decide that this is unnecessary, but all options to consider in formulating the final plan.

[chavo]

WOW thats alot of great info.... and some not so great! lol
basically it is a family run business with franchisee's so yes we have 20 offices but we cant really store there. as for indexing etc we have a system in place as any company does! we have a server already and we outsource the server management
.
the majority of our new business is via the web and the info is stored on our website. this is hosted out of manchester on there very expensive RAID servers that are backed up by them, however i back them up to an FTP also daily keeping a 2 day trail.

As for legal implications of paper trails there are none in our industry but we need to be able to produce evidence of our findings for i think 6 years. at the moment this is a whole room of folders. we can refer to old info in a few minutes when required.

The decision has been made anyway and it is this. the server will remain doing what it has always done, we have hired a guy on min wage to scan, we reckon it will take him a year to scan them all (12k) they will then be stored on our new Lacie 2tb raid drive (600 quid) when he has finished they will be either burnt to blu ray or we will mirror it. the paper copies will be put in to storage.

Thanks for all your info gents. overwhelmed!

[Lucio]

Thanks for all your info gents. overwhelmed!

Best data backup method known to man, the humble piece of paper kept in a dry, warm environment in a fire safe and is even known to be future compatable with most generations of Human