Question about what to put on RAID and what to put on NVME

[email protected]

Alternatively, if your databases are on a filesystem that supports snapshots (LVM, btrfs or ZFS for instance), you can make a snapshot of the filesystem, mount the snapshot and backup thame database from it. This will ensure the backup is consistent with itself (the backed up directory was not written to between the beginning and the end of the backup)

[email protected]

RAID gives you greater uptime. That is all. You should also have backups. So how much uptime do you need?

[email protected]

Both your RAID and NVMe data should be getting backed up daily to 2 different destinations, if it's irreplaceable.

But to answer your question, just place the DB and cache files for Photoprism on the NVMe, and the photos themselves on the RAID.

[email protected]

Hah. I see your looking into ZFS caching. Highly recommend. I'm running Ubuntu 24.04 Root on ZFS RAID10. Twelve each data drives and one nvme cache drive. Gotta say it's performing exceptionally. ZFS is a bit tricky, it requires an HBA not a RAID card. You may to to flash the raid card to get it working like I did. After that, I have put together a GitHub for the install on ZFS RAID 10, but you should easily be able to change it to RAIDz2.

https://github.com/Reddimes/ubuntu-zfsraid10/

[email protected]

Can you suggest a method for two-destination daily backups that don’t involve a 3rd party service? At the moment, I’m doing every six months or so on two sets of cold storage, one offsite.

[email protected]

Picked up a LSI SAS 9305-16I. I was always planning to do software raid, so I think it’ll do the trick for zfs.

[email protected]

That sounds reasonable, although I'd be worried about not having an extra local daily backup in case files get corrupted or accidentally deleted.

[email protected]

I’ve heard that too. Hmm.

Up until recently, the server mostly hosted a photo library and media library that didn’t tend to change very often. So a hdd in a fireproof save updated once a year was enough for me.

I guess I’ll have to come up with a better solution. What would you recommend for automatic backups? I’m trying to avoid 3rd party services.

[email protected]

Hell yeah, it will. I need one of those bad boys.

[email protected]

I've switched to restic for my backups and have been very happy with it. Very fast, encrypted and snapshot history.

[email protected]

Building RAID on top of SSDs is an answer.

[email protected]

Spinning disks can perform well if you do it right

[email protected]

It also can improve performance when done with enough disks

[email protected]

My new motherboard actually has a RAID controller for the M.2 slots. I know people frown on hardware raid, but given it's the boot drive, it might just be easiest to count on it for daily operation and backup to the software RAID/something else every night.

[email protected]

Lucked out on eBay and got it for $50.

[email protected]

So are you thinking like a raspberry pi with an 18TB hard drive accepting nightly backups through restic?

[email protected]

Doesn't this just pass the issue to when the snapshot is made? If the snapshot is created mid-database update, won't you have the same problem?

[email protected]

Wouldn't this require the service to go down for a few minutes every night?

[email protected]

Make sure, if you use hardware RAID, you know what happens if your controller dies.

Is the data in a format you can access it easily? Do you need a specific raid controller to be able to read it in the future? How are you going to get a new controller if you need it?

That's a big reason why people nudge you to software raid: if you're using md and doing a mirror, then that'll work on any damn drive controller on earth that linux can talk to, and you don't need to worry about how you're getting your data back if a controller dies on you.

[email protected]

I meant software RAID of course. Hardware RAIDs just cause headacehes, but fake RAIDs that are built into motherboards are a real nightmare.

agnos.is Forums

Question about what to put on RAID and what to put on NVME