Filesystem and virtualization decisions for homeserver build
-
FYI, zfs is pretty fucking fragile, it breaks a lot, especially if you like to keep your kernel up to date. The kernel abi is just unstable and it takes months to catch up.
Which is part of why I don't trust zfs on root.
Worst case you can sometimes recover with zfs-fuse.
Right, thanks for the heads up!
On the desktops I have simply installed zfs as root via the Ubuntu 24.04 installer. Then, as the option was not available in the server variant I started to think maybe that is not something that should be done
-
I work for a medium size enterprise as a backup architect. All of our backups are crash consistent and we’ve never had an issue.
Windows has an easy way of dealing with this in the form of VSS. As long as the application supports it, VSS can prepare the system and application for a backup, putting it in an application-consistent state before the snapshot is taken. Unfortunately, there is no equivalent for Linux. The best you can do is pre-freeze and post-thaw scripts to put the application/OS in a backup-ready state. Really though, I wouldn’t worry too much about it. Unless you are running an in-memory database, you really don’t need to worry about application consistency. If you are running an in-memory database, take database level backups (can also be done with pre-freeze/post-thaw scripts) and back up the backups.
Just remember to test whatever solution you end up going with, and make reminders to frequently re-test your backups. You never know what might change in a year’s time, so re-testing periodically is a good way to make sure everything is still functioning properly and make sure your data is still protected. And testing needs to be more than just making sure the VM powers on. Make sure the application can start up and function properly before calling it a successful test.
Always a good reminder to test the backups, no I would not sleep properly if I didn't test them
Aiming to keep it simple, too many moving parts in the VM snapshots / hard to figure out best practices and notice mistakes without work experience in the area, so I'll just backup the data separately and call it a day. But thanks for the input! I don't think any of my services have in-memory db's.
-
Boy. You asked about Proxmox. Nobody said anything.
How does Proxmox make it easier? Have you used it? All sorts of ways. Like, its a full virtual infrastructure management system instead of just an OS. Proxmox loves ZFS. It does many of the things you've mentioned here.
Proxmox does have its own backup system that can work with an NFS target or with their smart dedupe storage and replication server product. https://www.proxmox.com/en/products/proxmox-backup-server/overview
You've got some pretty advanced ideas and perhaps have already moved beyond the Proxmox question. But if you are curious and haven't used it, spin up a server and give it a whirl.
I guess I'll give it a spin. There seems to be a big community around it. I initially thought I might migrate later so keeping the host OS layer as thin as possible. Ubuntu was mainly an easy start as I was familiar with it from before and the spirit in this initiative is DIY over framework - but if there's a widely used solution for exactly this.. Yeah.
-
Right, thanks for the heads up!
On the desktops I have simply installed zfs as root via the Ubuntu 24.04 installer. Then, as the option was not available in the server variant I started to think maybe that is not something that should be doneIt's good, but be aware you want to stick to LTS kernels or at least don't upgrade casually.
Arch is the worst for this, ubuntu and debian are better but still get hit.
https://forums.opensuse.org/t/zfs-on-tumbleweed-how-to-keep-a-working-kernel-version/151323
https://github.com/openzfs/zfs/issues/15759
https://www.reddit.com/r/archlinux/comments/137pucy/zfs_not_compatible_with_kernel_63/
Hit this recently on an arch build, switched to kernel-lts and it worked, but basically once every year or so the abi breaks and zfs is dead for 3-6 months on github.com/torvalds/linux@master. Just FYI.
-
Boy. You asked about Proxmox. Nobody said anything.
How does Proxmox make it easier? Have you used it? All sorts of ways. Like, its a full virtual infrastructure management system instead of just an OS. Proxmox loves ZFS. It does many of the things you've mentioned here.
Proxmox does have its own backup system that can work with an NFS target or with their smart dedupe storage and replication server product. https://www.proxmox.com/en/products/proxmox-backup-server/overview
You've got some pretty advanced ideas and perhaps have already moved beyond the Proxmox question. But if you are curious and haven't used it, spin up a server and give it a whirl.
Proxmox is Debian, so much of your ideas could translate directly across. That said, I try to mod the PVE server as liitle as possible.
Proxmox makes it so easy to spin up yet another VM or LCX to handle services with its core offerings. Also google "proxmox helper scripts" to find tteck's additional stash of ready-made LCX.
-
It's good, but be aware you want to stick to LTS kernels or at least don't upgrade casually.
Arch is the worst for this, ubuntu and debian are better but still get hit.
https://forums.opensuse.org/t/zfs-on-tumbleweed-how-to-keep-a-working-kernel-version/151323
https://github.com/openzfs/zfs/issues/15759
https://www.reddit.com/r/archlinux/comments/137pucy/zfs_not_compatible_with_kernel_63/
Hit this recently on an arch build, switched to kernel-lts and it worked, but basically once every year or so the abi breaks and zfs is dead for 3-6 months on github.com/torvalds/linux@master. Just FYI.
Really good to know. Planned to keep using very mainstream LTS versions anyway, but this solidifies the decision. Maybe on a laptop I'll install something more experimental but that's then throwaway style.
-
System shared this topic on
