Proxmox vs. Debian: Running media server on older hardware

[Guest]

I’m surprised no one’s mentioned Incus, it’s a hypervisor like Proxmox but it’s designed to install onto Debian no prob. Does VMs and containers just like Proxmox, and snapshots too. The web UI is essential, you add a repo for it.

Proxmox isn’t reliable if you’re not paying them, the free people are the test people - and a bit back there was a bad update they pushed that broke shit. If I’d have updated before they pulled it, I’d have been hosed.

Basically you want a device that you don’t have to worry about updates, because updates are good for security. And Proxmox ain’t that.

On top of their custom kernel and stuff, it’s just less eyes than, say, the kernel Debian ships. Proxmox isn’t worth the lock-in and brittleness for just making VMs.

So to summarize, Debian and Incus installed. BTRFS if you’re happy with 1 drive or 2 RAID 1 drives. BTRFS gets scrubbing and bitrot detection (protection with RAID 1). ZFS for more drives. Toss on Cockpit too.

If you want less hands-on, do to OpenMediaVault. No room for Proxmox in my view, esp. for no clustering.

Also the iGPU on the 6600K likely is good enough for whatever transcoding you’d do (esp. if it’s rare and 1080p, it’ll do 4k no prob and multiple streams at once). The Nvidia card is just wasting power.

[catloaf@lemm.ee]

Some of us don't build applications, we use them as built by other companies. If we're really unlucky they refuse to support running on a VM.

[jlh@lemmy.jlh.name]

Not QEMU in particular, poor phrasing on my part. I just mean setting up new environments that run applications on VMs.

[marauding_gibberish142@lemmy.dbzer0.com]

I prefer some of my applications to be on VMs. For example, my observability stack (ELK + Grafana) which I like to keep separate from other environments. I suppose the argument could be made that I should spin up a separate k8s cluster if I want to do that but it's faster to deploy directly on VMs, and there's also less moving parts (I run two 50 node K8S clusters so I'm not averse to containers, just saying). Easier and relatively secure tool for the right job. Sure, I could mess with cgroups and play with kernel parameters and all of that jazz to secure k8s more but why bother when I can make my life easier by trusting Red Hat? Also I'm not yet running a k8s version that supports SELinux and I tend to keep it enabled.

[jlh@lemmy.jlh.name]

Sure!

I haven't used quadlets yet, but I did set up a few systemd services for containers back in the day before quadlets came out. I also used to use docker compose back in 2017/2018.

Docker compose and Kubernetes are very similar as a homelab admin. Docker compose syntax is a little less verbose, and it has some shortcuts for storage and networking. But that also means it's less flexible if you are doing more complex things. Docker compose doesn't start containers on boot by default I think(?) which is pretty bad for application hosting. Docker-compose has no way of automatically deploying from git like ArgoCD does.

Kubernetes also has a lot of self-healing automation, like health checks that can either disable the load balancer and/or restart the container if an app is failing, automatic killing of containers when resources are low, preventing the scheduling of new containers when resources are low, gradual roll-out of containers so that the old version of a container doesn't get killed until the new version is up and healthy (helpful in case the new config is broken), mounting secrets as files in a container, and automatic retry on failed containers.

There's also a lot of ubiquitous automation tools in the Kubernetes space, like cert-manager for setting up certificates (both ACME and local CA), Ingress for setting up reverse proxy, CNPG for setting up postgres clusters with automated backups, and first-class instrumentation/integration with prometheus and loki (both were designed for kubernetes first).

The main downsides with Kubernetes in a homelab is that there is about a 1-2GiB RAM overhead for small clusters, and most documentation and examples are written for docker-compose, so you have to convert apps into a Deployment (you get used to writing deployments for new apps though). I would say installing things like Ingress or CNPG is probably easier than installing similar reverse-proxy automations on Docker-compose, though.

[jlh@lemmy.jlh.name]

Yeah, that's fair. I have set up Openshift Virtualization for customers using 3rd party appliances. I've even worked on some projects where a 3rd party appliance is part of the original spec for the cluster, so installing Openshift Virtualization to run VMs is part of the day 1 installation of the Kubernetes cluster.

[jlh@lemmy.jlh.name]

The production-scale Grafana LGTM stack only runs on Kubernetes fwiw. Docker and VMs are not supported. I'm a bit surprised that Kubernetes wouldn't have enough availability to be able to co-locate your general workloads and your observability stack, but that's totally fair to segment those workloads.

I've heard the argument that "kubernetes has more moving parts" a lot, and I think that is a misunderstanding. At a base level, all computers have infinite moving parts. QEMU has a lot of moving parts, containerd has a lot of moving parts. The reason why people use kubernetes is that all of those moving parts are automated and abstracted away to reduce the daily cognitive load for us operations folk. As an example I don't run manual updates for minor versions in my homelab. I have a cronjob that runs renovate, which goes and updates my Deployments, and ArgoCD automatically deploys the changes. Technically that's a lot of moving parts to use, but it saves me a lot of manual work and thinking, and turns my whole homelab into a sort of automated cloud service that I can go a month without thinking about.

I'm not sure if container break-out attacks are a reasonable concern for homelabs. See the relatively minor concern in the announcement I made as an Unraid employee last year when Leaky Vessels happened. Keep in mind that containerd uses cgroups under the hood.

Yeah, apparmor/selinux isn't very popular in the k8s space. I think it's easy enough to use them, plenty of documentation out there; but Openshift/okd is the only distribution that runs it out of the box.

[jlh@lemmy.jlh.name]

I have 33 database servers in my homelab across 11 postgres clusters, all with automated barman backups to S3.

This stuff is all automated these days.

[tofu@lemmy.nocturnal.garden]

Yes, but that's a supported way to install Proxmox.

https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm

[mhzawadi@lemmy.horwood.cloud]

but you shouldn't be building out new applications and new environments on VMs or LXC

That's a bold statement, VMs might be just fine for some.

Use what ever is best for you, if thats containers great. If that's a VM, sure. Just make sure you keep it secure.

[ancoraunamoka@lemmy.dbzer0.com]

Not at all

[ancoraunamoka@lemmy.dbzer0.com]

Thanks so much for mentioning this, trying it out now

[tbi@lemmy.world]

I use OpenMediaVault to run something similar. It’s a headless Debian distribution with web based config. Takes a bit of work but I like it.

[kr0n@piefed.social]

I don't know about your first need ("MergerFS") but if you find useful, I have an old Intel NUC 6i3SYH (i3-6100U) with 16Gb RAM and I was running with Windows 10 for Plex+Arr and also HomeAssistant in VirtualBox. I was running into issues until I switched to Proxmox. Now I'm running Proxmox to run Docker with a bunch of containers (plex+arr and others) and also a virtual machine which has HomeAssistant and everything was smooth. I have to say that there is a learning curve, but it's very stable.

[lemmchen@feddit.org]

Only when using zfs, which op is not.

[counciloffriends@slrpnk.net]

My server runs Debian VMs in Proxmox on an i7-2600 which has a lower benchmark than the 6600k. I also used the Perfect Media Server guide, and have 2 x 8T data drives pooled with MergerFS and 1 for snapraid parity, these are passed through to the main VM from Proxmox with 'qm set'. One thing I often forget after deleting/restoring this VM is to run qm set again after restore, ensuring it has the flag to not back up those drives or else backups will fail and I have to go uncheck the backup option to fix it. If I need to spin up another VM for tinkering it's also easy enough to mount the NFS share as a volume with docker compose.

Proxmox rarely shows CPU usage go above 50%, and this handles the whole *arr stack plus usenet and torrents in a single VM and compose file. I don't even have GPU passthrough setup because the motherboard on this older rig didn't support something like IOMMU. Never had issues with Plex or Jellyfin transcoding for Chromecast.

[lemmchen@feddit.org]

Proxmox is pretty much focused on ZFS, LXC containers and VMs. You want mergerFS and Docker. I say avoid Proxmox and go for Debian or another distro.

[merthyr1831@lemmy.ml]

MergerFS and SnapRAID could be good for you. It's not immediate parity like with ZFS RAID (You run a regular cronjob to calculate RAID parity) but it supports mismatched drive sizes, expansion of the pool at any time, and some other features that should be good for a media server where live parity isn't critical.

Proxmox and TrueNAS are nice because they help manage ZFS and other remote management within a nice UI but really you can just use Debian with SSH and do the same stuff. DietPi has a few nice utilities on top of Debian (DDNS manager and CLI fstab utilities, for example)but not super necessary.

Personally I use TrueNAS but I also used DietPi/Debian for years and both have benefits and it really matters what your workflow is.

Docker or LXC containers won't hurt your performance btw. There's supposedly some tiny overhead but both are designed to use the basic Linux system as much as possible: they're way faster than on WSL. For hardware acceleration it'll be deferred to the GPU for most things and there's lots of documentation to set it up. The best thing about docker is that every application is kept separate to eachother - updates can be done incrementally and rollbacks are possible too!

[mr_jaaay@lemmy.ml]

Seconding this, I'm currently running Proxmox on 3 small NUC-type PCs (two Dell Optiplexes and a Topton from AliExpress). The Topton has a slower Celeron, the two Dells have a i5-6500 and i3-8100t and are both very snappy running a few different containers and VMs (including HomeAssistant).

[marauding_gibberish142@lemmy.dbzer0.com]

Ah thanks, I'll go through it!