Go Back   Home > Forums > >
Home Forums Rules Articles diyAudio Store Blogs Gallery Wiki Register Donations FAQ Calendar Search Today's Posts Mark Forums Read

PC Based Computer music servers, crossovers, and equalization

How to deal with music-file disk corruption?
How to deal with music-file disk corruption?
Please consider donating to help us continue to serve you.

Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
Reply
 
Thread Tools Search this Thread
Old 20th November 2018, 10:56 PM   #21
Pano is offline Pano  United States
diyAudio Moderator
 
Pano's Avatar
 
Join Date: Oct 2004
Location: SW Florida
How to deal with music-file disk corruption?
Quote:
Originally Posted by Greg Erskine View Post
In my experience corruption is a rare occurrence usually a result of ignoring warnings. Failing disk drives and lack of disk space are usually know in advance.
That has not been my experience. In the case of my main music library there were no official warnings, just bad sectors that caused file corruption. The only sign of trouble was that the USB drive had been slow to connect a couple of times, so I set about to find out what was wrong. That's when the bad sectors and files were discovered. Fortunately I had not over-written the backup files.

I suppose I'm lucky, in 25 years of using hard drives this is the first that has failed me at home. And it's been rare at work.

What I would like to know (and I suppose the OP as well) is how to regularly - or better yet automatically - scan for file errors on the main library to avoid corrupting the backup.
__________________
Take the Speaker Voltage Test!
  Reply With Quote
Old 20th November 2018, 11:41 PM   #22
Greg Erskine is offline Greg Erskine  Australia
piCorePlayer
diyAudio Member
 
Join Date: Jan 2002
Location: Sydney/Australia
How to deal with music-file disk corruption?
I think we all know disk drives don't last forever. So we should be doing check disks regularly and checking the error logs, trying to anticipate when the number of bad blocks is significant, but we don't.

BTW: I always thought it was the OS that handles these issues not the application.

Here's my strategy. I use 3 USB drives. All with identical music files. One is on my ripping PC, one is on my LMS and the other is floating. I usually stick the other one of my pCP clients.

So I rip new CDs on my ripping PC. I sync from PC to my LMS. I use WinSCP which gives my a list of files that it is going to copy. This lists only the new files. I then sync from my ripping PC to USB drive on pCP. Now I have 3 identical copies.

I disconnect USB drives from ripping PC and pCP. I should store them with a neighbor, but I don't.

A few years goes by, LMS starts acting up. Slow to boot, USB drive in read only mode, some tracks won't play. What do I do? Ignore it of course until LMS completely stops working. I then buy a new drive and sync from one of the other two.

It seems simple and logical to me but I know no one will agree.

regards
Greg
__________________
Greg Erskine
  Reply With Quote
Old 21st November 2018, 12:06 AM   #23
Tromperie is offline Tromperie  Australia
diyAudio Member
 
Join Date: May 2015
Location: Melbourne
I have a small NAS based on the HP Gen 8 microserver with 8G of ECC unbuffered RAM and 4 x 2TB NAS drives in mirror.
Every 28 days, FreeNAS is scheduled to do a scrub o check for corrupted files, but I haven't had one in over five years. It sits under the desk and quietly hums away while being accessed by Daphile a few rooms away.
__________________
Source: LMS 7.9.2 on FreeNAS, PiCorePlayer * Storage: FreeNAS 11 * DIY PGA2310 * Amps: 8 x LM3886TF
Xover: 24dB @ 300 and 3kHz * Speakers: CNC cut 54 laminations * SB 10", SB 5", VIFA 1" * Remote: Squeeze Control
  Reply With Quote
Old 21st November 2018, 12:14 AM   #24
MAAC0 is offline MAAC0  Portugal
diyAudio Member
 
Join Date: May 2010
How does Your bank protect it's customer data, account balances etc from data corruption ? I believe they use raid systems with backups and even on different datacenters around the world. … Fire, storms and hearth-quakes happen.
The problem is... in order to check our multi-terabyte music collections, our computer systems are too slow. A simple copy of 1TB of music (150000 files) to my QNAP Nas on a 1Gb network needs a whole night unless we invest in a IBM Z mainframe with 64 processors and 8TB of RAIM (not a typo) memory...
  Reply With Quote
Old 21st November 2018, 12:20 AM   #25
Tromperie is offline Tromperie  Australia
diyAudio Member
 
Join Date: May 2015
Location: Melbourne
Quote:
A simple copy of 1TB of music (150000 files) to my QNAP Nas on a 1Gb network needs a whole night
Those proprietary NAS boxes tend to have slow transfer rates. My FreeNAS box is much faster.
__________________
Source: LMS 7.9.2 on FreeNAS, PiCorePlayer * Storage: FreeNAS 11 * DIY PGA2310 * Amps: 8 x LM3886TF
Xover: 24dB @ 300 and 3kHz * Speakers: CNC cut 54 laminations * SB 10", SB 5", VIFA 1" * Remote: Squeeze Control
  Reply With Quote
Old 21st November 2018, 12:34 AM   #26
Greg Erskine is offline Greg Erskine  Australia
piCorePlayer
diyAudio Member
 
Join Date: Jan 2002
Location: Sydney/Australia
How to deal with music-file disk corruption?
Quote:
Originally Posted by Tromperie View Post
I have a small NAS based on the HP Gen 8 microserver with 8G of ECC unbuffered RAM and 4 x 2TB NAS drives in mirror.
Every 28 days, FreeNAS is scheduled to do a scrub o check for corrupted files, but I haven't had one in over five years. It sits under the desk and quietly hums away while being accessed by Daphile a few rooms away.
What do you do for backups?
__________________
Greg Erskine
  Reply With Quote
Old 21st November 2018, 12:43 AM   #27
grimberg is offline grimberg  United States
diyAudio Member
 
grimberg's Avatar
 
Join Date: Oct 2002
Location: California, USA
Quote:
Originally Posted by MAAC0 View Post
I believe they use raid systems with backups and even on different datacenters around the world. Fire, storms and hearth-quakes happen.
Hopefully not, or I would not bank with them. Most likely they use a SAN.
  Reply With Quote
Old 21st November 2018, 01:08 AM   #28
Greg Erskine is offline Greg Erskine  Australia
piCorePlayer
diyAudio Member
 
Join Date: Jan 2002
Location: Sydney/Australia
How to deal with music-file disk corruption?
The bank datacenters I've seen have the SAN on the first row then all the servers in the rows behind.

I think they do this because the SANs look so nice... really big, shinny black box with blue lights.

I am amazed that hackers can get into these systems because the effort required to do even legitimate work is enormous.
__________________
Greg Erskine
  Reply With Quote
Old 21st November 2018, 01:26 AM   #29
grimberg is offline grimberg  United States
diyAudio Member
 
grimberg's Avatar
 
Join Date: Oct 2002
Location: California, USA
Quote:
Originally Posted by Greg Erskine View Post
Every OS I have used has some form of disk checking.
Yes, but the OS utility only checks for information consistency in the used portion of the disk. It does not do anything about the disk's unused area. Manufacturer provided utilities scan the entire surface of the disk, whether or not it is in use, and reports any error it encounters.
  Reply With Quote
Old 21st November 2018, 01:37 AM   #30
grimberg is offline grimberg  United States
diyAudio Member
 
grimberg's Avatar
 
Join Date: Oct 2002
Location: California, USA
Quote:
Originally Posted by Pano View Post
What I would like to know (and I suppose the OP as well) is how to regularly - or better yet automatically - scan for file errors on the main library to avoid corrupting the backup.
I do it once a month. On the 27th in the evening I start the Full Scan feature of the Data Lifeguard Diagnostic utility, which takes a few hours to complete. In the morning of the 28th, if no error was reported, I generate and start the script to run flac.exe -t on each and every music file. Late in the afternoon I have the result.
  Reply With Quote

Reply


How to deal with music-file disk corruption?Hide this!Advertise here!
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
What CD or Music file YOU USE to test bass??? murillollirum Music 48 26th November 2018 09:25 PM
SMSL M8 displays 44.1 kHz USB when playing a 24/98-kHz music file sinq Digital Line Level 2 21st July 2016 05:02 PM
What's the best way to store my video&music file that I have on my loptap ? which's h andrewan Everything Else 1 5th January 2010 09:07 AM
File Uploader - php script with file validation lineup The Lounge 5 11th February 2007 10:09 AM


New To Site? Need Help?

All times are GMT. The time now is 12:36 PM.


Search Engine Optimisation provided by DragonByte SEO (Pro) - vBulletin Mods & Addons Copyright © 2019 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 14.29%
vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2019 DragonByte Technologies Ltd.
Copyright ©1999-2019 diyAudio
Wiki