What do people use for a shelf-stable backup
-
That's effectively what I have now. However I seem to kill a drive every couple of years, so I am keen for something that can be stored for many years (preferably decades).
-
Yes my issue is that I seem to be replacing a drive somewhere every couple of years. I am keen for something that can be stored in a cupboard for years, preferably a decent chance at lasting decades.
-
So my offsites are an incremental backup, but at some point the oldest version is gone. I am keen for a completely separate, long term snapshot of what I had that could be thrown in a cupboard, and any random family member clearing my house out as I get moved into a rest home at 108 can go through the photos and find a good one to put on my headstone.
I am also keen for protection against doing something dumb and losing everything (like losing my hard drive and finding out for some reason I can't access my backups because I lost the encryption key because I put it in bitwarden and they shut down years ago and I never moved the key over because I forgot it was stored there).
-
So, 50 years isn't a reasonable goal unless you have a pretty big budget for this. Essentially no media is likely to survive that long and be readable unless they're stored in a vault, under perfect climate controlled conditions. And even if the media is fine, finding an ancient drive to read a format that no longer exists is not a guaranteed proposition.
You frankly should be expecting to have to replace everything every couple of years, and maybe more often if your routine tests of the media show it's started rotting.
Long term archival storage really isn't just a dump it to some media and lock it up and never look at ever again.
Alternately, you could just make someone else pay for all of this, and shove all of this to something like Glacier and make the media Amazon's problem. (Assuming Amazon is around that long and that nothing catches fire.)
-
Hmm damn. I don't really think cloud is the right answer for what I'm trying to do.
I disagree that formats like JPEG won't be readable in 50 years. I feel like there would be big demand for being able to read the format even if it's been superceded, on account of all the JPEGs that still living people have.
Maybe I get a big drive. Each year I copy over files from the last year. Every X years I swap the hard drive for a new one, copy all data.
How can I tell if individual files get corrupted? Like the hard drive failed in that section, then I copy the corrupted file to the new drive, and I'd never know. Can I test in bulk? 50k+ photos and videos so far.
-
The format is the tape in the drive, or the disk or whatever.
Tape existed 50 years ago: nothing modern and in production can read those tapes.
The problem is, given a big enough time window, the literal drives to read it will simply no longer exist, and you won't be able to access even non-rotted media because of that.
As for data integrity, there's a lot of options: you can make a md5 sum of each file, and then do it again and see if anything is different.
The only caveat here is you have to make sure whatever you're using to make the checksums gets stored somewhere that's not JUST on the drive because if the drive DOES corrupt itself, and your only record of the "good" hashes is on the drive, well, you can't necessarily trust those hashes either.
-
ZFS and BTRFS both provide that functionality. Have a look into the features.
-
-
Ah - I gotcha. That's some terrible luck with drives.
-
If you buy your LTO drive new, then yes they rip you a new one, for sure!
Buy it used...but it still will cost you a few hundred. Like I said, if money is not a concern.
If losing the encryption key is a concern, then USB is still your best bet. Make two, keep them simple and unencrypted, stick em in two different safes, update them regularly. And print the documentation with pictures! -
-
-
-
-
Ah good thinking. I am thinking a spare drive that I update once a year with new content and replace every few years with a new drive is a good idea.
-
I mean, there are a lot of drives. Two laptops with a drive each. A desktop/server with three drives, and a spare laptop used for Kodi is the current setup. I'm not counting but I think it's three drives, one laptop, and one mobo since I started self-hosting perhaps 5 or 6 years ago.
The drives themselves, one was still under warranty, one was probably 3 or 4 years old, and the last was probably 6 or 8 and was in an old laptop and well used.
I think some of the drives have had a hard live while I messed around self-hosting, especially during my phase of trying out photo solutions.
-
-
I have had terrible experience with USBs failing, including losing a bunch of photos beyond recovery (some 15 years ago, but it still hurts). Plus it's quite aot of data.
I'm thinking a hard drive + USB SATA cable might be my best option. Add the new content each year. Work out some way of verifying it's not corrupted. Replace drive every 5 years or something, it can be swapped in when I get a failure and the new one can be the new backup.
-
Yeah based on the suggestions so far this seems like the best option. Just need to make sure I have a way to verify no files are corrupted (and if they are, which ones), and remember to swap it out for a new one every 5 years or so or each time I need a new drive.
-
No, the "live" filesystems will repair themselves when they detect problems. They keep revisions of your data, and run checksums constantly. When they find a file has inadvertently changed without access, it will restore said files. Think of it like Mac "Time Machine", but it's just the filesystem . You can restore stuff from points in time when needed.
Just read up on it.
