From my experience, I can confirm when a drive temperature gets higher than 60º C, it begins to have problems. Laptop hard drives or Hard drives in USB enclosures are prone to this. Cooling is critical. All my laptops use a cooling base in summer, unless You can provide a stable ambient with AC. If You have a drive which burns Your fingers after a few seconds touching, backup ASAP. It Will Fail after a short time. I had a drive which failed a few days after purchase, just by touching the PCB chips with the fingers while working.
Last edited:
Renegade software and the idiot human operator...
A late addition to this mouldering thread
In my experience, far more common is the mysterious alteration or disappearance of files. Of course, some may be blamed on human error. But software may be to blame for some. No matter how good your disk system is, if a program says to modify or delete a file ... will be done.
In my case, I've noted odd missing or altered files. For example, most recently, some Van Halen tracks were missing. "Dance the Night Away" had been renamed to "Ice Cream Man" (or some such). Metadata AND name of file!
Is Windows 10 doing evil selective file deletions?
Is JRiver Media Center screwing up?
I use various tagging programs: Mp3tag and Mp3gain are my favorites.
Some program bug somewhere? I don't have any proof of what is causing the problems, only the suspicion the exist. I'm not posting in industry web sites since I have no documentation, and we know that Microsoft would never do anything like spy on its users
A late addition to this mouldering thread
In my experience, far more common is the mysterious alteration or disappearance of files. Of course, some may be blamed on human error. But software may be to blame for some. No matter how good your disk system is, if a program says to modify or delete a file ... will be done.
In my case, I've noted odd missing or altered files. For example, most recently, some Van Halen tracks were missing. "Dance the Night Away" had been renamed to "Ice Cream Man" (or some such). Metadata AND name of file!
Is Windows 10 doing evil selective file deletions?
Is JRiver Media Center screwing up?
I use various tagging programs: Mp3tag and Mp3gain are my favorites.
Some program bug somewhere? I don't have any proof of what is causing the problems, only the suspicion the exist. I'm not posting in industry web sites since I have no documentation, and we know that Microsoft would never do anything like spy on its users
You always have to be careful about applications renaming and moving files and always prefer to just have one application do that. So if JRiver can modify tags, i wouldn't use the others or disable the ability for JRiver to update the tags.
Disk rot and dumb-user tricks will poison your backups and even cloud backups, so it is important to either keep the original media and back up downloads. If you don't keep the original media, then backup the initial rip.
I've also have had issues with cloud backups not restoring files. During a hard drive crash i did a dumb user trick that started to backup the drive to the drive i was restoring from. I just fat-fingered the "okay" dialog, and realized i just wiped the drive i was restoring from. Luckily i had a cloud backup to restore from and also i was doing file checksums and after the restore i noticed a files that failed the checksum. In fact those files were zero-byte sized files from the restore, i contacted the vendor (backblaze) and they said if i rebooted, updated or restarted the backup applications those files mid-flight would not get backed-up. I then asked if there was a way to confirm files were properly backed up and they promptly said "no". From my dumb user trick, i was about 3 files short that i just had to re-rip or restore from another drive, not a huge deal since I had multiple copies.
in the end, backup and replicate as much as possible now that disk space is cheap.
Disk rot and dumb-user tricks will poison your backups and even cloud backups, so it is important to either keep the original media and back up downloads. If you don't keep the original media, then backup the initial rip.
I've also have had issues with cloud backups not restoring files. During a hard drive crash i did a dumb user trick that started to backup the drive to the drive i was restoring from. I just fat-fingered the "okay" dialog, and realized i just wiped the drive i was restoring from. Luckily i had a cloud backup to restore from and also i was doing file checksums and after the restore i noticed a files that failed the checksum. In fact those files were zero-byte sized files from the restore, i contacted the vendor (backblaze) and they said if i rebooted, updated or restarted the backup applications those files mid-flight would not get backed-up. I then asked if there was a way to confirm files were properly backed up and they promptly said "no". From my dumb user trick, i was about 3 files short that i just had to re-rip or restore from another drive, not a huge deal since I had multiple copies.
in the end, backup and replicate as much as possible now that disk space is cheap.
i always have backups, i have the NAS (in RAID6) that is my master, and i have backups to 2 different external drives from 2 different computers in my home. One at my mediaplayer-computer does do a daily update of the music folder used by my mediaplayer, the other one daily on an older computer that is used more as server for other (IT) purposes, but only change files that are 14days old on the master. If the master goes bad and i see it too late, i still have my older backup on that that runs 2 weeks behind. This is how many big companies (where i work as IT consultant) also do their backups today.
Or using a NAS with filesystem supporting snapshots, such as ZFS on the above mentioned FreeNAS Schedule periodic snapshots of a ZFS volume with FreeNAS
I try to keep on top of my hard drives and don't keep them until they fail.
Probably a couple of years is long enough before sending them off to the bay.
As time goes by drives get better and faster anyway so its good to keep up with tech.
I also have a 128GB flash drive for backing up now and then.
Anything that would be a fatal loss is kept on DVD and flash drive in the car in case of fire etc.
Probably a couple of years is long enough before sending them off to the bay.
As time goes by drives get better and faster anyway so its good to keep up with tech.
I also have a 128GB flash drive for backing up now and then.
Anything that would be a fatal loss is kept on DVD and flash drive in the car in case of fire etc.
Snapshot is great for dumb user tricks and my documents, code, etc all make use of that, but snapshot is normally short lived. Keep in mind snapshot only stores the delta, not a true copy of the file and any failure can be a full loss. I also use git for documents and of course code and push changes to a remote location. Neither are great for large files that mostly sit there and occasionally read.
FreeNAS and or any NAS is not a backup, and should consider yourself lucky you haven’t had any issues. My Synology NAS has been good too, but it is just a step away from a virus/hack, which has happened forcing me to disable all non-critical services a precaution.
FreeNAS and or any NAS is not a backup, and should consider yourself lucky you haven’t had any issues. My Synology NAS has been good too, but it is just a step away from a virus/hack, which has happened forcing me to disable all non-critical services a precaution.
ZFS/btrfs snapshot is a pointer at the filesystem block tree top at the moment the snapshot was created. It is a complete state of the filesystem, no delta. As safe as the whole filesystem. If a virus/user removes/damages a file later, no problem, that change is not part of the original snapshot. If the hardware fails or something erases the whole filesystem, that is what backup is for. Especially simple and efficient to do in ZFS due to support for streaming only changes from last backup.
It is not a complete copy of all the blocks, only modified blocks as it tracks changes based on the snapshot duration. The more snapshots created the larger the delta between the snapshots and current view for unique blocks. If there were no modified blocks between snapshots, the additional snapshot doesn't consume any additional storage. Hence, you only get a delta of what has changed, it is not a method to backup your files, it is a method to enable backups of your our files with little to no resources.
Last edited:
Of course it is not a method of backup. But no filesystem access (i.e. virus, user action) can damage the snapshot content, unlike e.g. the .git directory or some other incremental copies on top of the filesystem (rsnapshot etc.). Therefore it is safe to say ZFS/btrfs snapshots protect against virus- or user-induced data loss. They are used e.g. for safepoints between upgrades in linux.
You mentioned bad experience with Synology - only their latest models offer btrfs. Old dumb ext4 has no such features. While FreeNAS (ZFS) offers making snapshots periodically, automatically deleting older snapshots to release disk space. Had a virus erased/encrypted files, you would just revert to a snapshot made one hour ago. Simple and effective.
You mentioned bad experience with Synology - only their latest models offer btrfs. Old dumb ext4 has no such features. While FreeNAS (ZFS) offers making snapshots periodically, automatically deleting older snapshots to release disk space. Had a virus erased/encrypted files, you would just revert to a snapshot made one hour ago. Simple and effective.
Advantage of an old back-up on hand...
While I've "rarely" lost critical data, I don't make frequent back-ups of the media drive. But backup(s) exist. They may be a year old, but so what? I have on a IDE drive that hasn't been re-used in a long time, owing to the old interface cable died When I have access to do so, I am going to copy this old version of my important music (in a directory KEEP) and compare it to the up-to-date live KEEP. In theory this directory only grows. In any case, I will use some file comparison utility to find out what's added and, more importantly, what mysteriously vanished!
While I've "rarely" lost critical data, I don't make frequent back-ups of the media drive. But backup(s) exist. They may be a year old, but so what? I have on a IDE drive that hasn't been re-used in a long time, owing to the old interface cable died
When I have access to do so, I am going to copy this old version of my important music (in a directory KEEP) and compare it to the up-to-date live KEEP. In theory this directory only grows. In any case, I will use some file comparison utility to find out what's added and, more importantly, what mysteriously vanished!
Odder and curiouser...
My oldest (known) backup is from about August 2016, 5181 files, 88 GB.
I found a 2nd copy of the directory KEEP, from about a year later. Curiously, it is 4597 files and only 27 GB! So something odd happened between 2016 and 2017. It will be interesting to see what the file comparison comes up with. This will take a while.
My oldest (known) backup is from about August 2016, 5181 files, 88 GB.
I found a 2nd copy of the directory KEEP, from about a year later. Curiously, it is 4597 files and only 27 GB! So something odd happened between 2016 and 2017. It will be interesting to see what the file comparison comes up with. This will take a while.
Tracking down the file changes wasn’t very hard.
Using the “archive” directory (2016 or 2017) compared to current (Nov 2019), changes weren’t quite as bad as I’d worried.
The vast shrinkage from 2016 to 2017 was explained by a large collection (1000 hours+) of a radio show’s directory having moved elsewhere.
I made simple directory listings of 2017 and today and then compared using Word for added or deleted file names. As should be, there a few dozen added files in two years. Nearly all of these were song’s I’d added in past few years or correcting a filename.
Some errors were found:
“Ice Cream Man” by Van Halen is the song “Dance the Night Away” for some reason.
“Dirty Woman” is apparently a misnamed file; it is the live version of “Dirty Women” by Black Sabbath.
“D.O.A.” by Bloodrock was missing from both. This may have been a clash with the same named song by Van Halen, which was present. “When you have two identically named files, what do I do?” asks a computer. “Delete one of the files!” Of course the human makes similar mistakes J
To be sure, some song names are used by different artists, whether a cover or different entirely but for title. Mostly, these were handled ok. But I suspect tagging utils may have buggered some. As a guide, consider that my collection has songs named “Gypsy” from Suzanne Vega, as well as three distinct ones by Black Sabbath, as well as former singers Ozzie and Dio!
In a similar vein, consider Yes’s album “90125”; many hits here and therefore in my KEEP directory. Problem: Trevor Rabin, co-singer on that album, released his own album with typical dry British humor, dubbed “90124” with the same tracks, just no Jon Anderson vocals J Same song names, just another confuser for the tagging software.
In conclusion, very little was actually wrong with my files. The names at least. I have not checked tags but I have no reason to think they have been naughty. My fears of rogue software or bad hardware are mostly if not completely baseless.
And best of all, I got to use a fossil PC at the office to copy off my old files, and I dumped, er, donated the old IDE drive to our junk drawer J
Using the “archive” directory (2016 or 2017) compared to current (Nov 2019), changes weren’t quite as bad as I’d worried.
The vast shrinkage from 2016 to 2017 was explained by a large collection (1000 hours+) of a radio show’s directory having moved elsewhere.
I made simple directory listings of 2017 and today and then compared using Word for added or deleted file names. As should be, there a few dozen added files in two years. Nearly all of these were song’s I’d added in past few years or correcting a filename.
Some errors were found:
“Ice Cream Man” by Van Halen is the song “Dance the Night Away” for some reason.
“Dirty Woman” is apparently a misnamed file; it is the live version of “Dirty Women” by Black Sabbath.
“D.O.A.” by Bloodrock was missing from both. This may have been a clash with the same named song by Van Halen, which was present. “When you have two identically named files, what do I do?” asks a computer. “Delete one of the files!” Of course the human makes similar mistakes J
To be sure, some song names are used by different artists, whether a cover or different entirely but for title. Mostly, these were handled ok. But I suspect tagging utils may have buggered some. As a guide, consider that my collection has songs named “Gypsy” from Suzanne Vega, as well as three distinct ones by Black Sabbath, as well as former singers Ozzie and Dio!
In a similar vein, consider Yes’s album “90125”; many hits here and therefore in my KEEP directory. Problem: Trevor Rabin, co-singer on that album, released his own album with typical dry British humor, dubbed “90124” with the same tracks, just no Jon Anderson vocals J Same song names, just another confuser for the tagging software.
In conclusion, very little was actually wrong with my files. The names at least. I have not checked tags but I have no reason to think they have been naughty. My fears of rogue software or bad hardware are mostly if not completely baseless.
And best of all, I got to use a fossil PC at the office to copy off my old files, and I dumped, er, donated the old IDE drive to our junk drawer J
I've read this thread with interest, and one recurring feature is baffling me.... why the urge to regularly back up files that don't change? I keep multiple copies of my music library - 1 mirror set at home, another at the office, and a rarely-synchronised single drive backup at each location - but I never perform full backups, I just update each copy with any changes using linux rsync and a portable drive that fits in my pocket. This I make easier by keeping my music in folders, 1 per year (I now have 11) and I only make changes to the current year's folder. So any new music goes in folder 2019, and if I want to retag an album in the 2012 folder I copy it first to 2019, update it then delete from 2012. And if I forget, it's not the end of the world is it?
My main storage for music is a 4TB drive in my desktop computer. It is backed up to a NAS using UnRAID. All other servers in the house only have access to the NAS. Every few months I collect a 6TB USB drive from my parents house .. backup the entire collection .. and take it back to their house. Off site storage
Because storage mediums can fail and not many backup programs test to see if a file is changed... they just copy everything.
Quite some time ago I decided to address this problem and came up with "Mirror" ... A smarter backup routine for windows with options for backing up only changed files, removing orphans from backups and rebuilding exact file trees in the backup. (Copy attached ... promise, no viruses)
In use it can cut backup time down from many hours to a couple of minutes.
Nice! Myself I use rsync, but then I'm an old-time unixer.
How does 'mirror' decide that a file has changed? If it cuts down backup times so dramatically, presumably not by checksumming. I set rsync to go by filesize only, but there's obviously some risk still of undetected degradation. I just calculate that any such cancer is less likely to spread if I'm not copying files wholesale.