I just bought a Quantum Superloader3 last Xmas. Each LTO-8 tape (it can take 16 in 2 magazines, I use 15 + 1 cleaning tape) will hold 12 TB without compression, 30TB with, and 7 of them can back up the 100TB used on the 128TB of disk that is the house RAID array.
It takes about 2 days to make a full backup, and I can fit incrementals for the next 5 days on the batch-of-7. Then I switch to the second magazine, and do the same thing. I actually have 3 magazines, one of which I swap in and out every week, and during the before-times, I'd take that off-site to work.
I have ~30 years of data, from back when I was in college and writing CD-ROMs for backup, all on the one system. Admittedly, the major space-taking thing is the Plex library, but I wouldn't want to lose that either. It takes about 5 minutes to walk into the garage (where the server-rack is), swap magazines and I'm done - the rest is automatic.
I have vague ideas for writing a more-efficient tar designed for this specific type of setup (big disk with attached tape). The best way to do it I think is to have multiple threads reading and bzip2-compressing data, piping blobs through to a singleton tape-writer thread. Every now and then (50GB, 500GB, 1TB ?) close the device and reopen the non-rewindable device to get a record-marker on the tape, and then store the tape/record-marker/byte-offset etc. into a SQLite database on the disk. That way I'd get:
- High levels of compression without making the tape head wait for the data, which ruins the tape head. Multiple threads pooling highly-compressed data into a "record"
- fast lookup of what is where, I'm thinking a SQL LIKE query syntax for search, against the disk-based DB. No more waiting for the record to page in from the end of the tape.
- fast find on-tape, since you'd know to just do the equivalent of 'mt fsf N' before you actually have to start reading data
Right now, tar is good enough. One of these days when I get time, I'll write 'bar' (Backup And Restore :)
Too loud for an office. Which is a shame because in these pandemic times, I repurposed the shed at the bottom of the garden as an office (insulated, add power and a/c) and I wanted it there.
It's not noisy when writing to tape, but the mechanism is noisy when a tape is being loaded, the magazine is being shuffled to get the right tape, etc. It's the mechanical parts rather than the tape drive itself that's too loud, especially with Webex conferencing being a part of the day now.
So I have it set up in the garage, in the server-rack. I was worried about temperatures in the Summer, so I bought a 100W solar panel, an attic fan, and linked them up, positioning the fan above the rack. That fan shifts so much air that the in-rack fans (with temperature monitoring) didn't get above 85 all summer, which is pretty amazing for the Bay Area. The tape deck seems to be fine in that sort of temperature, and yes I do do the occasional 'tar tvf' to check the data is readable :)
Yes. I'd forgotten about that. There is a reasonably-above-ambient-noise fan that is running constantly. Another reason it was banished to the server-rack in the garage.
I got mine at Backupworks.com[1] - they seem to have a perpetual sale on for pricier items like this. They gave me a discount on a batch of tapes bought at the same time and threw in some barcodes and a cleaner tape as well.
It takes about 2 days to make a full backup, and I can fit incrementals for the next 5 days on the batch-of-7. Then I switch to the second magazine, and do the same thing. I actually have 3 magazines, one of which I swap in and out every week, and during the before-times, I'd take that off-site to work.
I have ~30 years of data, from back when I was in college and writing CD-ROMs for backup, all on the one system. Admittedly, the major space-taking thing is the Plex library, but I wouldn't want to lose that either. It takes about 5 minutes to walk into the garage (where the server-rack is), swap magazines and I'm done - the rest is automatic.
I have vague ideas for writing a more-efficient tar designed for this specific type of setup (big disk with attached tape). The best way to do it I think is to have multiple threads reading and bzip2-compressing data, piping blobs through to a singleton tape-writer thread. Every now and then (50GB, 500GB, 1TB ?) close the device and reopen the non-rewindable device to get a record-marker on the tape, and then store the tape/record-marker/byte-offset etc. into a SQLite database on the disk. That way I'd get:
- High levels of compression without making the tape head wait for the data, which ruins the tape head. Multiple threads pooling highly-compressed data into a "record"
- fast lookup of what is where, I'm thinking a SQL LIKE query syntax for search, against the disk-based DB. No more waiting for the record to page in from the end of the tape.
- fast find on-tape, since you'd know to just do the equivalent of 'mt fsf N' before you actually have to start reading data
Right now, tar is good enough. One of these days when I get time, I'll write 'bar' (Backup And Restore :)