Hello all
Thinking of something DIY hack against drive failure and power outage attached to the main pc. This pc has a dongle with 8 Bluetooth connection ability and a really good Wi-Fi dongle. The 65" screen attached to this is the main screen for entertainment, DAW, pc monitor
Pair of USB flash drives in raid 1 on a powered hub plugged into a solar topped powerbank? Will this work as a cheap raid 1 array accessible by all the devices on the network?
Thanks and regards
Randy
Thinking of something DIY hack against drive failure and power outage attached to the main pc. This pc has a dongle with 8 Bluetooth connection ability and a really good Wi-Fi dongle. The 65" screen attached to this is the main screen for entertainment, DAW, pc monitor
Pair of USB flash drives in raid 1 on a powered hub plugged into a solar topped powerbank? Will this work as a cheap raid 1 array accessible by all the devices on the network?
Thanks and regards
Randy
Solved, a pair of these coming in the mail with USB 3.0 external cases. I'll run them in Raid 1, should be decent redundancy. At some point, I would like to get them away from windows altogether into some sort of NAS
https://www.mwave.com.au/product/cr...d-nand-sata-iii-ssd-with-95mm-adapter-ac11047
https://www.mwave.com.au/product/cr...d-nand-sata-iii-ssd-with-95mm-adapter-ac11047
If you wish to keep it simple and USB-only, this could be an option:
https://www.qnap.com/da-dk/product/tr-002
Personally I have all my audio/video on a NAS, that is accessible to everyone on my LAN.
https://www.qnap.com/da-dk/product/tr-002
Personally I have all my audio/video on a NAS, that is accessible to everyone on my LAN.
Hey man, thanks for pointing that one out. Looks perfect for the job. I Searched around for shopping and came across this
https://www.mwave.com.au/product/si...-usb-31-gen-2-external-raid-enclosure-ac29366
Less than half the price, do you think this would be a better value option?
https://www.mwave.com.au/product/si...-usb-31-gen-2-external-raid-enclosure-ac29366
Less than half the price, do you think this would be a better value option?
Another way to look at it. If you aren’t trying to keep track of writes, just protect from device failure, you could do periodic data syncs and keep one drive for disaster recovery. A little more of a manual process though.
The most resilient backups are offline. Which means that you have to backup regularly, but you are protected in the case of full drive failure. Consumer level RAID 1 is for the most part useless unless you get read speed aggregation, which most store-bought enclosures do not have.
For my entire collection (music, movies, documents, and miscellaneous files), I have it stored on three USB external drives totaling 32TB , served 24x7 by two Raspberry Pi4s, with a complete offline backup server that has the exact same data capacity. Backups are fully manual in my case, literally having to flip a power switch on the backup server, logging in remotely, and initiating the backup process. I use SyncBack Free version, which allows me to autmoate most of the process. Actually it can automate everything, but I choose to keep some decisions on approval basis.
This has literally saved my skin as a lot of my PCs have very old hard drives, and I lost about 8 drives to bad sectors in the last 2 years alone (all very old drives, so no regrets there).
For my entire collection (music, movies, documents, and miscellaneous files), I have it stored on three USB external drives totaling 32TB , served 24x7 by two Raspberry Pi4s, with a complete offline backup server that has the exact same data capacity. Backups are fully manual in my case, literally having to flip a power switch on the backup server, logging in remotely, and initiating the backup process. I use SyncBack Free version, which allows me to autmoate most of the process. Actually it can automate everything, but I choose to keep some decisions on approval basis.
This has literally saved my skin as a lot of my PCs have very old hard drives, and I lost about 8 drives to bad sectors in the last 2 years alone (all very old drives, so no regrets there).
Go for RAID5.
You can do full blown NAS... I got two of these: Western Digital PR4100 stuffed with 10TB Red drives.
https://www.westerndigital.com/prod...my-cloud-pro-series-pr4100#WDBNFA0000NBK-NESN
I also have three of these ( I have a fourth one unused in the closet ) with WD 6TB Red drives. Connected via USB3 and shared on the network as samba drives.
https://www.newegg.com/mediasonic-hfr2-su3s2/p/N82E16816322004
All in total, not counting the parity drives, that makes for 114TB.... I also have two 6TB greens, one backing the other, manually... and the two Tablos each have 4TB USB3 external drives.
You can do full blown NAS... I got two of these: Western Digital PR4100 stuffed with 10TB Red drives.
https://www.westerndigital.com/prod...my-cloud-pro-series-pr4100#WDBNFA0000NBK-NESN
I also have three of these ( I have a fourth one unused in the closet ) with WD 6TB Red drives. Connected via USB3 and shared on the network as samba drives.
https://www.newegg.com/mediasonic-hfr2-su3s2/p/N82E16816322004
All in total, not counting the parity drives, that makes for 114TB.... I also have two 6TB greens, one backing the other, manually... and the two Tablos each have 4TB USB3 external drives.
I had my music (16gb) on a USB flash drive and music video clips (150gb) on hard disk. All of a sudden, I had my USB flash start, bringing up the drive needs formatting error. This has happened to a few drives now. The HD also died at one point. Since then, I placed my audio collection on cloud so that is safe. Same for work files
I have rebuilt the video clips' library back up to about 120gb now, but it's really slowing the rest of the PC down. I am also starting to record audio and video.
Time to sort out an integrated storage system, as cloud is not really practical for me for video due to being in a remote regional area and being limited to mobile data plans
I looked into what causes reliability issues with flash drives. It's the power outage thing, which is fairly frequent here. The crucial SSD touts a feature against data loss due to sudden power outage and I thought that it would make a good start to a great storage system
Is it not true that RAID 1 is a set and forget, self running system that clones all the 'writes' to the drives? I have used it like this over 25 years ago for hard disk inside my biz PC to secure against drive loss, and it worked for that. Raid 1 is wasteful when it comes to capacity, but the crucial mx500 500gb was cheaper than USB flash of the same size and much faster, so look at it like. Raid 1 redundancy of total usable size 500gb for $100 AUD plus the $100 raid USB 3.0 case. Am I mis-seeing the combination of lazy but solid redundancy and value?
I have rebuilt the video clips' library back up to about 120gb now, but it's really slowing the rest of the PC down. I am also starting to record audio and video.
Time to sort out an integrated storage system, as cloud is not really practical for me for video due to being in a remote regional area and being limited to mobile data plans
I looked into what causes reliability issues with flash drives. It's the power outage thing, which is fairly frequent here. The crucial SSD touts a feature against data loss due to sudden power outage and I thought that it would make a good start to a great storage system
Is it not true that RAID 1 is a set and forget, self running system that clones all the 'writes' to the drives? I have used it like this over 25 years ago for hard disk inside my biz PC to secure against drive loss, and it worked for that. Raid 1 is wasteful when it comes to capacity, but the crucial mx500 500gb was cheaper than USB flash of the same size and much faster, so look at it like. Raid 1 redundancy of total usable size 500gb for $100 AUD plus the $100 raid USB 3.0 case. Am I mis-seeing the combination of lazy but solid redundancy and value?
I run RAID5 because it is the most efficient and safe for a home, and small office, environment.
The WD NASs have dual power supplies and use large enough cache that you don't lose data during a write.
The Sandisk RAID5 enclosures, and the laptops/micro form factors that host them are behind UPS's.
I ran fairly deep redundancy, not only do I have RAID5 but I have one NAS back the other, and the Scandisk are triple redundant. Consequently I have effectively less than 60GB of storage but if I have a hardware failure, all I have to do is remount the network drives in the client machines.
Our personal machines do not have any data in them... everything is in the network. Local configuration data, ie: C drive, is backed up on a secondary local drive. If I lose a "C:" drive it is trivial to swap...
I did this because back in '99 I had a data disaster where I lost the main drive and the back up drive in a back up machine. Three years of pictures... Thankfully, my documents and emails were elsewhere!
The NAS are also very handy as they also host Plex servers.... so I got lots of movies and TV shows in them.
The WD NASs have dual power supplies and use large enough cache that you don't lose data during a write.
The Sandisk RAID5 enclosures, and the laptops/micro form factors that host them are behind UPS's.
I ran fairly deep redundancy, not only do I have RAID5 but I have one NAS back the other, and the Scandisk are triple redundant. Consequently I have effectively less than 60GB of storage but if I have a hardware failure, all I have to do is remount the network drives in the client machines.
Our personal machines do not have any data in them... everything is in the network. Local configuration data, ie: C drive, is backed up on a secondary local drive. If I lose a "C:" drive it is trivial to swap...
I did this because back in '99 I had a data disaster where I lost the main drive and the back up drive in a back up machine. Three years of pictures... Thankfully, my documents and emails were elsewhere!
The NAS are also very handy as they also host Plex servers.... so I got lots of movies and TV shows in them.
When you go for RAID 1, best to use different brands for both drives. Reduces the risk of both failing at approx the same moment. They will see exactly the same load and running hours and in those days of planned obsolescence you can expect everything. I have seen identical SSD drives fail within a few hours of each other more than once. In that respect the backup based system makes more sense.
Also keep in mind that flash drives don't like lots of writing cycles.
Also keep in mind that flash drives don't like lots of writing cycles.
SSDs and HDDs are very different.
Failures on an SSD are cell wear related due to Read/Write access cycles... if you buy two SSDs from the same batch and then use them identically then you are likely to see very similar failure modes.
Failures on an HDD are more randomized.
DO NOT use SSD for any kind of RAID that will see lots of writes. Big server clouds, like Netflix use SSD because they are mostly Read only. Other large server clouds are well maintained, keep track of the SMART statistics and proactively replace their drives ( and/or refresh them ).
Failures on an SSD are cell wear related due to Read/Write access cycles... if you buy two SSDs from the same batch and then use them identically then you are likely to see very similar failure modes.
Failures on an HDD are more randomized.
DO NOT use SSD for any kind of RAID that will see lots of writes. Big server clouds, like Netflix use SSD because they are mostly Read only. Other large server clouds are well maintained, keep track of the SMART statistics and proactively replace their drives ( and/or refresh them ).
I highly recommend TrueNAS: https://www.truenas.com/
TrueNAS CORE is free to use.
I've been running FreeNAS/TrueNAS since 2019 and it's rock solid. I use a 2010 vintage Mac Pro ("cheese grater") with four SATA slots for it. One slot has the SSD for the TrueNAS OS, one slot has a 2 TB enterprise grade HDD for Time Machine, and two slots have 10 GB enterprise grade HDDs for data in a configuration similar to RAID1. I paid about $150 for the Mac Pro at the local eWaste recycling place.
I have TrueNAS configured to run automatic backups to cloud storage on Backblaze B2. It also talks to my UPS and will turn itself off gracefully if my house is without power for more than four minutes.
Those interested in the failure rates of HDDs and SSDs should take a look at Backblaze's drive stats: https://www.backblaze.com/b2/hard-drive-test-data.html Their most recent report includes SSDs.
Tom
TrueNAS CORE is free to use.
I've been running FreeNAS/TrueNAS since 2019 and it's rock solid. I use a 2010 vintage Mac Pro ("cheese grater") with four SATA slots for it. One slot has the SSD for the TrueNAS OS, one slot has a 2 TB enterprise grade HDD for Time Machine, and two slots have 10 GB enterprise grade HDDs for data in a configuration similar to RAID1. I paid about $150 for the Mac Pro at the local eWaste recycling place.
I have TrueNAS configured to run automatic backups to cloud storage on Backblaze B2. It also talks to my UPS and will turn itself off gracefully if my house is without power for more than four minutes.
Those interested in the failure rates of HDDs and SSDs should take a look at Backblaze's drive stats: https://www.backblaze.com/b2/hard-drive-test-data.html Their most recent report includes SSDs.
Tom
HGST has been owned by WDC for eight years now.
Those are SATA drives.... enterprise drives are NVME/PCIe. We used to track those since enterprise SSDs don't use SATA anymore, that's a dead market for enterprise. SATA drives have not been "enterprise" since '18, they might be used for small office, advanced home usage. Otherwise, it's just a marketing name.
Anything under 10TB in the 3.5" HDD world is considered archaic.
The action for home SSD is in the NVME M.2 format. Even my laptops and MFF machines use them.
Before I left that company, I had some 32TB drives on my desk....
Those are SATA drives.... enterprise drives are NVME/PCIe. We used to track those since enterprise SSDs don't use SATA anymore, that's a dead market for enterprise. SATA drives have not been "enterprise" since '18, they might be used for small office, advanced home usage. Otherwise, it's just a marketing name.
Anything under 10TB in the 3.5" HDD world is considered archaic.
The action for home SSD is in the NVME M.2 format. Even my laptops and MFF machines use them.
Before I left that company, I had some 32TB drives on my desk....
I have a small UPS in front of my NAS to deal with that. It shuts down the NAS gracefully before the battery runs out.
It also runs my internet router, so in case of a local power-outage I can still access the internet on my laptop.
I like the UPS option. It would be good to get all my studio and pc gear except for power amps onto a solar charged battery bank. I usually have all of them connected and on 24/7. It would hurt to brick the digital mixer and such
It really depends on how your NAS is set up and designed. More expensive NAS devices have built in protection on power loss.. cheaper "RAID enclosure" only don't.
My WDC NASs have built in cache and some capacitance to them, so they don't lose data on a write if the power goes down.
OTOH, my homebrewed NASs with laptops and RAID enclosures are behind UPSs... not so much the laptop which is safe but I need to make sure a power loss on the RAID enclosure won't lose data during a write.
That said, all of my network gear (switches, wireless access points), and the non-laptop PCs are behind UPSs. They don't cost much and I can keep the home Intranetwork running for an hour or more. I also keep the router and cable modem behind an UPS in the case that the cable company is up (with connection to the Internet) while our neighborhood is down. This is because the telco vault in the area has its own separate power.
Last edited:
Some have gotten NVME/PCIe running on the old cheese graters. I'm sure you'll get faster I/O that way. I'm perfectly content with my setup though. It more than meets my needs.
Tom
Speed should not be an issue with network attached storage... I think the worst would be with Raspberry 4 using USB3... something I keep playing with as well.
Ideally you keep the "data" on the network and keep the "applications" and OS in the local machine. Do a hardware backup of the local machine.
Configuration data for applications should also be stored in the network.
There is a hit on start up and on accessing some of the data, but not a great thing. You can always keep some temporary cache storage in your local machine but remember that it is the most at risk.
In a perfect World, you "local" machine should also be superfluous and backed up by others. For example, for me access to my email is the killer app. Second is access to the Boorkmarks and stored username/passwords in my browsers... So, I got three different machines that are configured identically... so even if I lose one of them, I got two more. My Plex servers are duplicated, and I got extra drives just in case one of the RAID5 drives fails. Heck, I can have an entire NAS fail and I still have another one.
It's not a matter of storage size, but a matter of reliability and robustness.
As I wrote earlier... after losing three years of pictures: pictures of our house walls as they were being built and pictures of our kids and family trips... I become very obsessive in not losing anything again.
Ideally you keep the "data" on the network and keep the "applications" and OS in the local machine. Do a hardware backup of the local machine.
Configuration data for applications should also be stored in the network.
There is a hit on start up and on accessing some of the data, but not a great thing. You can always keep some temporary cache storage in your local machine but remember that it is the most at risk.
In a perfect World, you "local" machine should also be superfluous and backed up by others. For example, for me access to my email is the killer app. Second is access to the Boorkmarks and stored username/passwords in my browsers... So, I got three different machines that are configured identically... so even if I lose one of them, I got two more. My Plex servers are duplicated, and I got extra drives just in case one of the RAID5 drives fails. Heck, I can have an entire NAS fail and I still have another one.
It's not a matter of storage size, but a matter of reliability and robustness.
As I wrote earlier... after losing three years of pictures: pictures of our house walls as they were being built and pictures of our kids and family trips... I become very obsessive in not losing anything again.
Raid of any kind is not a backup strategy. It's for continuity.
Better off backing up to external drive(s). Keep it simple.
Thing is if the drive controller or motherboard crap out or have OS crash, there might be difficulties recovering the raid config.
Better off backing up to external drive(s). Keep it simple.
Thing is if the drive controller or motherboard crap out or have OS crash, there might be difficulties recovering the raid config.
It depends how you read "back up". You can use RAID strategies for backing up your data. Simply by adding more RAID servers in your intranet fabric.
Corporate users will also leverage their storage servers across geographically separate data centers... thus achieving back up as well. In the home, yes, the safest way to "back up" your data is to store in element safe storage.
Read how I described my configuration. If an OS crashes, it takes me five minutes to swap local hard drive. If the user machine crashes, I got two more ready to rock. If one NAS crashes, I got another one. If both NAS crash, I got three more RAID hosts... All on line.
Quadruple redundancy. With backup of drivers, hardware, host machines, local drives.. etc...
Everything is external...
I also store the data in large 10TB drivers, monthly or so, and store that in a fire/water proof safe.
I have been looking at stuff like this for a few years now but I haven't pulled the trigger yet:
https://www.amazon.com/ioSafe-218-DISKLESS-Diskless-Fireproof-Waterproof/dp/B079VR38V1
Unfortunately, I don't know how to "backup" my LP collection... and, no, don't tell me to use an AD/DAC... I'm doing that, but then I go and update the turntable/phono preamp/cartridge.... and I'm back to square one.
Corporate users will also leverage their storage servers across geographically separate data centers... thus achieving back up as well. In the home, yes, the safest way to "back up" your data is to store in element safe storage.
Read how I described my configuration. If an OS crashes, it takes me five minutes to swap local hard drive. If the user machine crashes, I got two more ready to rock. If one NAS crashes, I got another one. If both NAS crash, I got three more RAID hosts... All on line.
Quadruple redundancy. With backup of drivers, hardware, host machines, local drives.. etc...
Everything is external...
I also store the data in large 10TB drivers, monthly or so, and store that in a fire/water proof safe.
I have been looking at stuff like this for a few years now but I haven't pulled the trigger yet:
https://www.amazon.com/ioSafe-218-DISKLESS-Diskless-Fireproof-Waterproof/dp/B079VR38V1
Unfortunately, I don't know how to "backup" my LP collection... and, no, don't tell me to use an AD/DAC... I'm doing that, but then I go and update the turntable/phono preamp/cartridge.... and I'm back to square one.
Last edited: