Go Back   Home > Forums > >
Home Forums Rules Articles diyAudio Store Blogs Gallery Wiki Register Donations FAQ Calendar Search Today's Posts Mark Forums Read

PC Based Computer music servers, crossovers, and equalization

How to deal with music-file disk corruption?
How to deal with music-file disk corruption?
Please consider donating to help us continue to serve you.

Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
Reply
 
Thread Tools Search this Thread
Old 19th November 2018, 08:39 AM   #1
PatternChaser is offline PatternChaser  England
diyAudio Member
 
Join Date: Nov 2018
Location: NW England
Default How to deal with music-file disk corruption?

Backup. That's what we're recommended to do to manage disk file corruption. That's what I did. But disk corruption doesn't come with a warning, when it occurs. So when it happened to me, although I didn't know that it had, I backed-up and over-wrote the good copies with the corrupted ones. I imagine some of you might have done the same?


Before I post what I have done to combat disk corruption, please tell me your ideas? How do you keep your music files safe?
  Reply With Quote
Old 19th November 2018, 08:47 AM   #2
baeijsman is offline baeijsman  Denmark
diyAudio Member
 
Join Date: Mar 2015
Location: Copenhagen
How to deal with music-file disk corruption?
I have a small NAS with mirrored disks. I get an e-mail warning when SMART-errors occur.
  Reply With Quote
Old 19th November 2018, 09:18 AM   #3
PatternChaser is offline PatternChaser  England
diyAudio Member
 
Join Date: Nov 2018
Location: NW England
SMART-errors? What are they? Can the disk detect file corruption on its own?
  Reply With Quote
Old 19th November 2018, 09:25 AM   #4
davidsrsb is offline davidsrsb  Malaysia
diyAudio Member
 
Join Date: Dec 2005
Location: Kuala Lumpur
S.M.A.R.T. - Wikipedia.
In theory the disk can detect early signs of failure.
This is often true for platter drives, but SSDs have a bad habit of failing totally
  Reply With Quote
Old 19th November 2018, 10:20 AM   #5
PatternChaser is offline PatternChaser  England
diyAudio Member
 
Join Date: Nov 2018
Location: NW England
Sounds great, but I wonder if it protects against corruption of individual files? As far as I know, file corruption can happen at any time (although it doesn't happen too often, thankfully), and may have nothing to do with failure of the whole disk drive. It sounds like SMART deals mainly with whole-disk failure modes.



I don't use a NAS drive. My music files - 1.2 TByte of FLAC and MP3 files - are stored (and backed-up) on external USB drives. I suffered corruption on the master drive, and that was that. Once I'd over-written the good copies while backing up, there was no way back. I still don't know how many files were damaged, and which ones they were. I find them when foobar2000 complains the file is corrupt while playing.


So I took the best steps I could think of to protect my music files. Now, I use the Microsoft file checksum integrity verifier (downloaded free from here). Before I back up, I check the master files for corruption using the Microsoft program. Then, when I know that no further corruption has occurred, I back up.



Now I think my music files are as safe as I can make them. Any suggestions for improvements?


P.S. I can upload the files I wrote to do this, if anyone would like copies? They're simple Windows batch files, and you would need to edit them to meet your own needs. But you're welcome if you want them?
  Reply With Quote
Old 19th November 2018, 10:29 AM   #6
ximikas is offline ximikas
diyAudio Member
 
Join Date: Jan 2010
Hi, there is a file manager, named Total Commander, it has checksum verifier function. You can scan entire folder contents and it can create sfv file with individual file names and checksums, later you can check if checksum matches, if all files are present.
  Reply With Quote
Old 19th November 2018, 12:58 PM   #7
phofman is offline phofman  Czech Republic
diyAudio Member
 
Join Date: Apr 2005
Location: Pilsen
There are filesystems which check for bit corruption on block level. ZFS is readily available in the FreeNAS OS ZFS - FreeNAS - Open Source Storage Operating System . Some people start using BTRFS in linux.

IMO building a small NAS device running FreeNAS with ZFS would make your sleep deep. I personally have not encountered random bit corruption yet.
  Reply With Quote
Old 19th November 2018, 01:47 PM   #8
PatternChaser is offline PatternChaser  England
diyAudio Member
 
Join Date: Nov 2018
Location: NW England
I was excited when I first heard of ZFS. But it can't protect against random file corruption on disk. I don't think anything can. I will check Total Commander, though. That could be useful. It takes a looonnnggg time to checksum 120000 tracks.
  Reply With Quote
Old 19th November 2018, 03:23 PM   #9
phofman is offline phofman  Czech Republic
diyAudio Member
 
Join Date: Apr 2005
Location: Pilsen
Well, the ZFS protection against drive corruption assumes you are running some form of redundancy - raid. Which is an obvious setup on NAS.
  Reply With Quote
Old 19th November 2018, 03:59 PM   #10
dmills is offline dmills  United Kingdom
diyAudio Member
 
Join Date: Aug 2008
Location: High Wycombe
Music files are basically read only right? I mean we are not talking about the files used in a session, but simple stereo audio for playback?

If so, import the file, and record the file hash (More or less any hashing algorithm will do, even MD5), then when you write the backup you hash the backup file and compare the hashes, if they match then you know to a very high level of confidence that both files match what was originally written, if not then you need to restore from an earlier backup that did match.

RAID is about availability NOT backup, backup should be offline and should be rotated, so that a failure while writing the backup does not leave you with both your working disks and their backups corrupt, I still commend LTO tape as a backup medium.

The problem with a RAID array is that while it protects you from a disk failure, it does little against the far more common HUMAN screwup, you need backup as well.

Regards, Dan.
  Reply With Quote

Reply


How to deal with music-file disk corruption?Hide this!Advertise here!
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
What CD or Music file YOU USE to test bass??? murillollirum Music 48 26th November 2018 09:25 PM
SMSL M8 displays 44.1 kHz USB when playing a 24/98-kHz music file sinq Digital Line Level 2 21st July 2016 05:02 PM
What's the best way to store my video&music file that I have on my loptap ? which's h andrewan Everything Else 1 5th January 2010 09:07 AM
File Uploader - php script with file validation lineup The Lounge 5 11th February 2007 10:09 AM


New To Site? Need Help?

All times are GMT. The time now is 02:15 AM.


Search Engine Optimisation provided by DragonByte SEO (Pro) - vBulletin Mods & Addons Copyright © 2019 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 14.29%
vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2019 DragonByte Technologies Ltd.
Copyright ©1999-2019 diyAudio
Wiki