What do people use for a shelf-stable backup
-
For local backups it depends on what you want to have:
- The cheapest option is a usb or thumb drive. But you have to regularly plug it in and copy your backup on it.
- The lazy option is to buy a NAS and configure a backup job that regularly creates a backup. Versioned, incremental, differentials and full backups are possible as is WORM to add a bit of extra security. You can configure a NAS to only turn on specified times, do a backup and then turn off again. This will increase protection against encrypting malware. WORM also helps in this case.
Or just let it run 24/7, create backups every hour and install extra services on it like AI powered image analysis to identify people and objects and let it automatically tag your photos. Cool stuff! Check out QNAP and Synology or build a NAS yourself.
A NAS can also be configured to present its content in a LAN by itself. Any computer will automatically connect to it if the access isn't secured by user/password or certificate.
I recommend buying a NAS.
-
This will do nothing at all. Drives don't die by rust. They usually die because the motor somehow can't get the discs to spin. Very often dry lube is the reason. That can occur if you leave the drive off too long.
-
So, I have a server that has a backup drive, automated backups, and replication to laptops as well as cloud storage in Backblaze B2. What I'm looking for is something completely separated from the automation that is a backup for if I screw up the automation, as well as a backup that a layman can access (i.e. no encryption, media that is usable by anyone). I have had some very bad experiences with flash drives but I am thinking a HDD with SATA->USB cable attached (I already have the cable).
From the other conversations in this thread mentioning many options, the hard drive option seems the best for my use case, but I've also been convinced of the benefit of printing out some physical photos as well, so my current plan is to get a big container, put a couple of mirrored hard drives in there (to validate against each other as protection against bit-rot), and print 100 photos each year to add to the container to have an extra layer of redundancy.
-
Any file systems Windows can read out-of-the-box are no good file systems. What Windows read? FAT and NTFS. Former is so basic it has no mechanisms to detect errors and bitrot and the later one is a mess.
You should stick to ext4, btrfs and zfs.If you want to make if fool-proof then add a sticker with 'bring me to a computer shop to access my content'.
-
I have considered that exact message. It does seem making it easily plug and play may be out of the question if I want the error correction capabilities.
-
Btrfs and zfs are self-healing.
You can make a script to check for errors and autocorrection yourself but that needs at least a second hdd. On both drives are the same data and a file or database with the checksums of the data. The script then compares the actual checksums of the two copies and the db checksum. If they match -> perfect. If they don't match the file where there are two matching checksum is the good one and replaces the faulty one or corrects the db entry, whichever is defect. That's it. It doesn't have to be more complicated.
-
Printing the photos won't help much. After 20 or so years they are all discolored. You can't prevent that.
I think SSDs might be the best storage medium for you. Consumer-grade ssds have a 1 year data retention when powered off. That means at least once per year you have to turn it on and copy the data around one time to refresh the cells. This way it'll probably last several 100 years.
You can't exactly make it fool-proof. Outside people will never know what you did to create your backup and what to do to access it. Who knows if the drives file system or file types are still readable after 20 years? Who knows if SATA and USB connectors are still around after that time?
For example it is very likely that SATA will disappear within the next 10-15 years as hdds are becoming more and more an enterprise thing and consumers are switching to M.2 ssds. -
Yip I think this is the setup I will want (probably both - zfs + a custom script for validation, just to be sure). Two mirrored drives. I do need to read up some about zfs mirroring to understand it a bit more but I think I have a path to follow now.
-
Then I'd go with FAT on a USB, which should be plenty portable into the future. You'll want to replace it every 5-10 years, and check on it every other year or so.
That's about as easy to use as I can think of. Decades down the road, physical media like DVDs and tapes may be difficult to find readers for, but USB is versatile enough that someone is bound to have access. Micro SD cards may also be a good option, as long as you keep a couple USB readers around.
-
I have a terrible track record with USB sticks, including completely losing a stack of photos because of a USB stick.
I'm now thinking the benefits of a nice error-correcting file system probably outweigh the benefits of using a widely supported one. So I might use a pair of mirrored hard drives with SATA->USB cable, then include instructions along the lines of "plug into my linux laptop to access, or take to a computer repair show if you can't work it out".
-
The printed photos are only there as an extra layer of redundancy in case everything else fails. It's ok if they get discoloured a bit, it never put me off going through my grandparents' suitcases of photos. Ideally the digital files survive, if not then at least there is something rather than nothing.
Is SSD really necessary? Everything I search up says SSDs have worse retention than HDD in cold storage. A couple TB of HDD is pretty cheap these days, and seems like a better cold storage option.
You can’t exactly make it fool-proof. Outside people will never know what you did to create your backup and what to do to access it. Who knows if the drives file system or file types are still readable after 20 years? Who knows if SATA and USB connectors are still around after that time?
Yes, so now I'm thinking a rotation cycle. About every 5 years replace the drives with new ones, copy over all data. If newer technology exists then I can move to that newer technology. This way I'm keeping it up to date as long as I can.
For example it is very likely that SATA will disappear within the next 10-15 years as hdds are becoming more and more an enterprise thing and consumers are switching to M.2 ssds.
Does this matter if I have a SATA->USB cable stored with it? Other than if USB A standards change or get abandoned for USB C, but that should be covered by the review every 5 years.
-
Is SSD really necessary? Everything I search up says SSDs have worse retention than HDD in cold storage. A couple TB of HDD is pretty cheap these days, and seems like a better cold storage option.
SSDs are by design less susceptible by design. No moving parts and able to work in much harsher conditions than hdds will ever be able to. The standard set by JEDEC requires every consumer ssd to have a 1 year data retention while powered off at 30 °C (I think). That's the minimum it has to archieve but usually they are better than that. Do not buy the cheapest thumb drives because they contain the all the crap that wasn't good enough to make ssds from it.
Btw you need to fire hdds up regularly too or the motor gets stuck. I think every 3-6 months was the recommendation.Yes, so now I’m thinking a rotation cycle. About every 5 years replace the drives with new ones, copy over all data.
Don't make it flat every 5 years. Let a software monitor the SMART values of the drives and send notifications if the values indicate an increased change of a dying disc/ssd.
Does this matter if I have a SATA->USB cable stored with it?
Those are the first that fail, followed by the usb controller chip in the tray. Keep it as simple as possible. Removable trays are probably the best way but I'm not sure how much wear they can take.Do not buy 2.5" drives. This class will die out soon
. There were no new hdds introduced in years and ssds are often replaced by M.2 ones because of the faster connection. -
Completely understandable. I say this as someone with way more photos in digital only media than I should.
-
I've decided I should have a small number of physical prints, as extra redundancy. I'm thinking I'll print 100 each year to store with the hard drive backup.
-
I decided instead to use ZFS. Better protection than just letting something sit there. Your backups are only as good as your restores. So, if you are not testing your restores, those backups may be useless anyway.
ZFS with snapshots, replicated to another ZFS box. The replicated data also stores the snapshots and they are read-only. I have snapshots running every hour.
I have full confidence that my data is safe and recoverable.
-
I have automated backups including to cloud, but I want a separated manual system that cannot get erased if I mess something up (accidentally sync a delete, lose encryption key, forget to pay cloud bill). I have 3 2 1 but it's all automated and backups are eventually replaced, if it's not a critical failure I won't necessarily know I've lost something.
Basically, I specifically want cold storage, and not cloud. I will only add, not delete from it. And I don't want it encrypted.
Based on other conversations I'm planning on using duel disks mirrored, zfs, annual updates and disk checks with disks rotated out every 5 years (unless failing/failed). Handling the need for layman retrival of data by including instructions with the hard drives.
-
You are the first person who has recommended SSD for cold storage. Everyone else (including what I've googled) says HDD for cold storage, just spin up every year or two and they will be fine. Can you point me at further reading?
Don't worry, I'll SMART check the drives each year as I update as required.
As for types of drives dying out soon, I can reassess the situation every 5 years when I do drive replacement. I would be confident 2.5" drives will still be readable in 5 years.
-
Reminds me of project Silica. Media historically was more durable (stone/ ink and cloth paper, etc) but had a low data density. As density increased, so did fragility
-
If you need something which can withstand some bitrot on single drive, just use par2. As long is filesystem is readable, you can recover files even if bit of data get corrupted
-
That is not gonna work - read up on byte rot.
