Copy Contents of HDD with File Integrity Check Proggy

[sleepyhead]

Just after a simple program (if there is one) that can copy the contents from HDDs upto 300GB to another HDD with some form of file integrity check (either a simple CRC32 check or md5sum check).

IE I can just set it going, "copy contents from drive x, y and z to drive k" (for example) rather than manually sit there, copy the files, compare CRC values and rebuild files if required using zidrav.

I'm lazy but want the added security knowing the files are identical.

Oh and if there is such a program, it being free would be a bonus as it is really only one time use.

[Agent]

Acronis will verify files written / read to / from a drive if you request it as part of the process

Not free though.

[peterb]

Comes back to a linux program for me! Am I right in thinking you want to clone/image the disk (which is not the same as just copying files from the file system)

I have used the Linux dd command to do this - works a treat - fast and free! Verifying the image before and afterwards is not quite so simple. You can calculate the md5 hash of the source drive but the target drive is unlikely to be the same size, so unless you do the md5 hash onn the same number of sectors, you will get a different result.

You might also want to look at robocopy

http://www.ss64.com/nt/robocopy.html

[Splash]

Robocopy for the copying, then Beyond Compare (not free, but at $30 USD it's well worth buying because you WILL use it again if you have it. they also have a 30 day trial, but I'm unsure if it'#s restricted in any way).

[sleepyhead]

Comes back to a linux program for me! Am I right in thinking you want to clone/image the disk (which is not the same as just copying files from the file system)

...

Heh, not clone, move the files

There is just alot and last time I attempted something this grandiose, it failed drastically using the simple windows "copy/paste" system. That was back in the Windows 98 days though and was only moving 20 gigs (unlike today's movement of around maybe 500GB...alot of it is collected junk but I am a hoarder so nothing gets deleted...lol).

[sleepyhead]

Quick question about Robocopy:

I have a folder on Drive E, say called Misc, and that already has data in it and I also have a folder on Drive G called Misc but with different data within. I want to backup folder Misc on Drive G to the folder called Misc on Drive E. Would Robocopy just append the different data from Drive G onto Drive E or would it overwrite it?

Also I am using the Robotcopy GUI as I am useless with command lines.

Sorry, one more question:
If I use /L under Logging Options will that generate a text files of the folders and files of the source location to the the target location?

[Splash]

I have to state that I've never used the Robocopy GUI, so I can't really comment on that. As for your copying to a folder with the same name on a different drive it rather depends - will there be files with the same name in both locations? If so the default behaviour would be to overwrite the file, but there are plenty of switches that can be used to change this.

the /L switch generates a list of files that would be copied, but doesn't actually copy anything which is good for planning, less good as a scheduled backup. I think the switch you're looking for is /LOG:%file% where %file% is the name of the logfile. I believe unless you specify otherwise that the log file will be created in the directory that you run the Robocopy job from.

As I'm pretty bored at the mo if you can give me some more details I can probably knock up a quick script for you - pm me the details if you like and I'll see what I can do.

EDIT - actually it seems that most of your requirements are already there, something like

robocopy g:\Misc e:\Misc /e /LOG:robocopy_log.txt

should do everything that you need, unless you need to mask so that certain files aren't copied. The /e switch will copy folders even if they're empty, so if you have no subfolders you can drop that.

[sleepyhead]

There will be different files in G:\Misc and E:\Misc. I want to make sure the contents of the target location isn't deleted when the folder names are the same and instead adds the files from the source location to the target location.

The GUI is really easy for me, click the switches I need (such as /e, /copyall, /move) and hit run

Also, thanks for helping me realise what I am doing wrong with trying to make a log file (forgetting to actually give the log file a name so it saved to nothing...ie nothing got saved. lol.)

[Splash]

No probs, but I do advise that you read up and fully understand the /move switch before using it...

The contents of the target location would only be overwritten (nothing would be deleted, unless you use the /mir switch) in the event that there was a file with the same name in the originating location (and even then only if it were newer, though this is the default behaviour again and can be changed)

Does the GUI generate a batch file or script that you can then run at a later date without using the GUI? I'm not sure if this is something that you only need to run once, but if it's something to be done on a regular basis then it might be worth writing your own...

As always, if you get stuck let me know and I'll be happy to help.

[sleepyhead]

I've only used the /move on one occasion and that was to move a folder I had already copied off the old HDD; I put it in the wrong location. Everything else has been a copy of the other drives.

This should be a one time thing, but as I am copying data from my 200GB PATA and 300GB External HDD and plonking it onto my new spangly HDD, I am re-organising too which makes backing everything up a simple case using Robocopy. Well in the end anyhow. There is also a 120GB PATA and 500GB SATA drive to loot as well...all to fit into a 750GB SATA drive (see "My System").

The GUI does allow me to save the scripts, which means I can load them up and run them pretty simply. I don't think it makes a batch file, I've not tried yet.

What other implications come with the /move switch? I know there is /mov and /move the "e" being the sub-directories variable. I also use the retry switch and resumeable copy/move switch (I think they are /R and /Z switch...not at home so can't check).

[silentvoice]

I use two programs for backup; Karen's Hasher and Karen's Replicator. I use Hasher to create MD5 hashes of all the files in a specific folder (although you can use other algorithms) and then use Replicator to do the copying. You can select whether you wish to compare the files and/or delete them on the destination drive. I then edit the hash file (saved as [name].md5) with the updated paths (i.e. C:\Download\... to G:\Downloads...) and use Hasher to verify that all the files have been copied over successfully without any problems.

Both programs are free.

Hasher: http://www.karenware.com/powertools/pthasher.asp

Replicator: http://www.karenware.com/powertools/ptreplicator.asp

Note: When using Hasher, if the list of files you wish to hash are in the thousands, it will take a while for the list to be created before you have the option to hash all of them. Try a few hundred files first to see how it works.