I've been running a 8TB homeserver for the cost of an optiplex on eBay (80) and 2 8tb external drives on Amazon (300). I cronjob an rsync every night and that's that. Cheap, works great, and (surprisingly) I understand how it works. I also have cloud storage for the subset of things I don't want to lose.
A thing I'm working on right now is for the rsync disk to be on a switch I control with an Arduino/pi so it's only powered during the backup -- I reckon the drive will last longer that way. Homeservers are fun :)
If you're ok losing 24hours of data and not a performance junky, the backup plan becomes trivial and the raiding isn't really necessary.
Daily rsync alone isn't enough. Any data which gets corrupted on the main drive will, within 24 hours, also be corrupted on the 'backup'.
If you're confident enough that your computer won't get stolen, burn, or zapped by a power surge, then you can keep the same hardware setup you have, but:
1. Use borg backup (or restic) instead of rsync (so that you can restore to any day in the past, not only to yesterday)
2. Disconnect the 'backup' drive when it's not being used (so that it's protected from malware or fat fingers).
Even better would be:
3. Have a separate computer host the borg backups, and have it run 'borg serve' in append-only mode (so that, no matter what you do the main computer, you cannot destroy or corrupt past backups).
Even better:
4. Have this separate computer in a different physical location (so that you're protected from fire, theft and the like).
Not the parent, but I use a similar setup. I avoid this be keeping my 2nd drive formatted as btrfs, primary is xfs. After I rsync to the btrfs drive, I take a snapshot.
I only rsync every week, my data doesn't change often and if it does, I can trigger a manual rsync using the same script in my /etc/cron.weekly/
edit: I do use borg to backup to my cloud storage server, since it's encrypted.
I didn't think corrupted data would change the size of the file or the mtime? If it doesn't then rsync wouldn't overwrite the second drive as it would see it as an existing, good copy. Unless you had rsync using checksum comparisons of course.
You're probably right for corruption caused by hardware failure or filesystem bugs. I was thinking about corruption in the wider sense, e.g. caused by application bugs.
borg looks awesome. It comes up all the time when I research efficient data storage replication (buzzhash) so now I'm really going to try it out, thanks for the tips! :)
The heads fly on a fluid cushion when spun up, they must physically land when the thing spins down. The head landing/takeoff operations are sources of physical wear.
Assuming a brushless motor and quality sealed ball bearings, it could be argued that the drive wears less kept spun up all the time.
My understanding is that spinning them down is primarily a power-saving measure.
When I last read up on this, which admittedly was over a decade ago, head parking was just a matter of locating the heads over an unused region of the platters. The head still goes through a landing and takeoff cycle, so the head still sees physical contact. It's just avoiding doing it over the data portion of the platters.
zfs or bust. If you are not using zfs for your long term storage and backups, you might want to consider what exactly you are doing and why you aren’t paying someone else to make sure your 1s stay as 1s and 0s as 0s.
To be fair, ZFS isn't the only game in town. Depending on what you're doing, gluster, ceph, minio, restic/borg/etc. may prove sufficient. I like ZFS, don't get me wrong, but if the goal is bitrot protection there are a number of different tools to solve it at different places in the stack.
For a NAS, I don’t think backup tools with checksumming would be sufficient. I also am not a fan of them as the only tool to prevent bitrot. If borg detects that a backup is corrupt, my option is what? To ditch that backup? zfs can not only detect it, but also recover from it. That’s not something that most tools can do.
To be fair, I don’t know about the capabilities of gluster, ceph, and minio.
Has it gotten less crash crash eat your data stash? Last I checked, distros were putting it into their install options while it was still happy to crash the FS with major data loss. I haven’t checked on whether it was worth it since.
It's been the default OS for opensuse for at least afew years. I've been using it on root filesystems and data volumes with no problems. I was backing up to a 1TB btrfs volume but it filled up with snapshots and cleaning them didn't cause any problems. I've since moved to a 4TB drive and I rsync another drive to it weekly and take a snapshot. I've had no issues.
If you're ok losing 24hours of data and not a performance junky, the backup plan becomes trivial and the raiding isn't really necessary.