Results 1 to 7 of 7

Thread: Hard Disk File Integrity Monitoring

  1. #1
    Senior Member
    Join Date
    Feb 2004
    Posts
    403
    Thanks
    58
    Thanked
    79 times in 68 posts
    • Firejack's system
      • Motherboard:
      • Asus PRIME X470-Pro
      • CPU:
      • AMD Ryzen 7 2700X
      • Memory:
      • TG Dark Pro "8pack Edition"
      • Storage:
      • Crucial 250GB SSD, Sandisk 128GB SSD, Samsung 1TB HDD
      • Graphics card(s):
      • Sapphire RX VEGA 56 8GB Pulse
      • PSU:
      • SeaSonic Focus Plus 650 Gold
      • Case:
      • Fractal Design Define S
      • Operating System:
      • Windows 10 Pro 64bit
      • Monitor(s):
      • Dell S2719DGF
      • Internet:
      • BT Infinity 2

    Hard Disk File Integrity Monitoring

    Hey
    I've got a hard disk containing all my family pics, files and videos which I backup to my NAS box.

    The week before last the Hard disk failed and corrupted a load of files. Which annoyingly corrupted some files in the backup on my NAS box when the backup job ran

    What I'm after is some way to prevent this. Is there any software that can monitor the file integrity of all my files and report any problems? Ideally after something I can schedule to run overnight.

    Cheers.

  2. #2
    Gentoo Ricer
    Join Date
    Jan 2005
    Location
    Galway
    Posts
    11,048
    Thanks
    1,016
    Thanked
    944 times in 704 posts
    • aidanjt's system
      • Motherboard:
      • Asus Strix Z370-G
      • CPU:
      • Intel i7-8700K
      • Memory:
      • 2x8GB Corsiar LPX 3000C15
      • Storage:
      • 500GB Samsung 960 EVO
      • Graphics card(s):
      • EVGA GTX 970 SC ACX 2.0
      • PSU:
      • EVGA G3 750W
      • Case:
      • Fractal Design Define C Mini
      • Operating System:
      • Windows 10 Pro
      • Monitor(s):
      • Asus MG279Q
      • Internet:
      • 240mbps Virgin Cable

    Re: Hard Disk File Integrity Monitoring

    What is your NAS box exactly?.. Is it an embedded device?.. Or generic PC hardware with a NAS OS, a general server OS like Linux, or Windows Home Server, etc...
    Quote Originally Posted by Agent View Post
    ...every time Creative bring out a new card range their advertising makes it sound like they have discovered a way to insert a thousand Chuck Norris super dwarfs in your ears...

  3. #3
    Senior Member
    Join Date
    Feb 2004
    Posts
    403
    Thanks
    58
    Thanked
    79 times in 68 posts
    • Firejack's system
      • Motherboard:
      • Asus PRIME X470-Pro
      • CPU:
      • AMD Ryzen 7 2700X
      • Memory:
      • TG Dark Pro "8pack Edition"
      • Storage:
      • Crucial 250GB SSD, Sandisk 128GB SSD, Samsung 1TB HDD
      • Graphics card(s):
      • Sapphire RX VEGA 56 8GB Pulse
      • PSU:
      • SeaSonic Focus Plus 650 Gold
      • Case:
      • Fractal Design Define S
      • Operating System:
      • Windows 10 Pro 64bit
      • Monitor(s):
      • Dell S2719DGF
      • Internet:
      • BT Infinity 2

    Re: Hard Disk File Integrity Monitoring

    Thanks for the reply.

    My NAS is running on an old computer. Athlon 1800+, 512MB memory, Nforce2 motherboard, 2xSamsung F1 1TB drives with Freenas 0.69 on it. Works really well

    The hard disk with all my files on is in my Vista 64 machine. Intel E2180, 4GB of Memory, Asus P5Q, 2x Samsung F1 1TB drives.

  4. #4
    Member
    Join Date
    Dec 2007
    Posts
    108
    Thanks
    6
    Thanked
    12 times in 8 posts
    • CX23882-19's system
      • Motherboard:
      • MSI K9AG Neo2-Digital
      • CPU:
      • AMD Athlon X2 BE-2350
      • Memory:
      • 2.0GB DDR2-800
      • Storage:
      • Western Digital WD6400AAKS
      • Graphics card(s):
      • ASUS EAH3650+ VNF100
      • PSU:
      • Antec Neo HE 430
      • Case:
      • Evercase ECE-4252
      • Operating System:
      • Windows Vista Ultimate 32-bit SP1
      • Monitor(s):
      • Dell 2407WFP-HC
      • Internet:
      • Vispa ADSL 2Mbps

    Re: Hard Disk File Integrity Monitoring

    This is a difficult one, because your backup software won't really know if the files it's backing up are corrupt or not. As far as it's concerned you have changed the files, just as you would update a Word document, so it will back up these bad files. You could use md5sums to quickly check for changes between the files in your backup and those currently on your system, but this is something that you would have to do manually.

    I would look at this problem from a different angle:

    I would recommend employing an incremental or differential backup to run daily. The first backup is always a full image and that is kept until you decide otherwise. Subsequent backups only store what has changed since the first backup was created (differential) or since the previous image (incremental).

    That way you have a full disk image of all of your data, and then if anything gets changed (or in your case corrupted), it will still back up the corrupted files, but you have your original files intact and can restore the good versions as you wish.

    I use Acronis True Image Home 2009 for this, and it can back up to a separate partition on your computer (or ideally a separate disk), an external disk, DVD-R or network folders. To back up my main Vista OS partition (80GB total and approx 30GB used) my current full backup requires 10GB space on my backup drive and each incremental varies between 500MB to 2.5GB depending on how much data changed during the day.

    The alternative is to produce a full backup and save it somewhere so that it won't be overwritten. This way you always have a good copy of the data.

  5. #5
    Senior Member
    Join Date
    Feb 2008
    Posts
    914
    Thanks
    4
    Thanked
    155 times in 143 posts
    • smargh's system
      • Motherboard:
      • Gigabyte GA-EP45-UD3P
      • CPU:
      • Xeon E5450 with 775-to-771 Mod
      • Memory:
      • 16GB Crucial
      • Storage:
      • Intel X25-M G2 80GB/Adaptec 3405 4x 2TB Ultrastar RAID1 / 1x 6TB Hitachi He6 / Dying 2TB Samsung
      • Graphics card(s):
      • GTX 750 Ti
      • PSU:
      • Seasonic X-560
      • Case:
      • Lian-Li PC-A71
      • Operating System:
      • Windows 7 Ultimate 64bit
      • Monitor(s):
      • BenQ G2400WD
      • Internet:
      • Really Crap ADSL2 <3Mbit

    Re: Hard Disk File Integrity Monitoring

    As a potential starting point for Google searches, how about Tripwire? It's a UNIX utility originally designed to monitor files which have been changed as a result of a security breach, but I guess it would also work okay for integrity checking. I've not personally used it in many years, though.

    http://sourceforge.net/projects/tripwire/

    On Windows, I use http://www.hdsentinel.com/ to check the SMART attributes on my HDDs every 10 minutes and email me any alerts.

  6. #6
    Senior Member
    Join Date
    Feb 2004
    Posts
    403
    Thanks
    58
    Thanked
    79 times in 68 posts
    • Firejack's system
      • Motherboard:
      • Asus PRIME X470-Pro
      • CPU:
      • AMD Ryzen 7 2700X
      • Memory:
      • TG Dark Pro "8pack Edition"
      • Storage:
      • Crucial 250GB SSD, Sandisk 128GB SSD, Samsung 1TB HDD
      • Graphics card(s):
      • Sapphire RX VEGA 56 8GB Pulse
      • PSU:
      • SeaSonic Focus Plus 650 Gold
      • Case:
      • Fractal Design Define S
      • Operating System:
      • Windows 10 Pro 64bit
      • Monitor(s):
      • Dell S2719DGF
      • Internet:
      • BT Infinity 2

    Re: Hard Disk File Integrity Monitoring

    Tripwire looks interesting. Doesn't appear to be a very active project though. Which makes me think it could have problems with recent operating systems. I'll look into it with more detail later.

    CX23882-19 -- I actually already tested using Acronis to handle the backup for me. During the test Acronis didn't seem to recognise the files were corrupted and backed up as normal with no errors. This was using Acronis TI 11.
    Going down the incremental route is something I could do. I'd need additional hard disk space to make this work. Simply to give me enough time to notice files are being corrupted before my last working backup is wiped to make room for new incremental backups.
    I'd also spent a long time having to add all the changed files since the last working backup.

    Really after a preventative technique. Beginning to think there is no solution out there though

  7. #7
    Gentoo Ricer
    Join Date
    Jan 2005
    Location
    Galway
    Posts
    11,048
    Thanks
    1,016
    Thanked
    944 times in 704 posts
    • aidanjt's system
      • Motherboard:
      • Asus Strix Z370-G
      • CPU:
      • Intel i7-8700K
      • Memory:
      • 2x8GB Corsiar LPX 3000C15
      • Storage:
      • 500GB Samsung 960 EVO
      • Graphics card(s):
      • EVGA GTX 970 SC ACX 2.0
      • PSU:
      • EVGA G3 750W
      • Case:
      • Fractal Design Define C Mini
      • Operating System:
      • Windows 10 Pro
      • Monitor(s):
      • Asus MG279Q
      • Internet:
      • 240mbps Virgin Cable

    Re: Hard Disk File Integrity Monitoring

    Using ZFS is another possible solution. I don't know the state of ZFS snapshots on FreeNAS though. Another possibility might be to use the NAS box as primary data storage.
    Quote Originally Posted by Agent View Post
    ...every time Creative bring out a new card range their advertising makes it sound like they have discovered a way to insert a thousand Chuck Norris super dwarfs in your ears...

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. LaTeX2html problem...
    By bsodmike in forum General Discussion
    Replies: 4
    Last Post: 15-12-2010, 03:51 PM
  2. External Hard disk buying advice
    By pp05 in forum PC Hardware and Components
    Replies: 19
    Last Post: 08-06-2008, 05:24 PM
  3. Someone Please help me with my hard disk, *this could take a while!*
    By Hunain in forum PC Hardware and Components
    Replies: 0
    Last Post: 04-01-2008, 09:29 AM
  4. Whats accessing my hard disk up?
    By spryszlak in forum PC Hardware and Components
    Replies: 2
    Last Post: 24-01-2006, 01:45 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •