Proxmox vs. Debian: Running media server on older hardware
-
yeah, and qemu and lxc are very much legacy at this point. Stick with docker/podman/kubernetes for containers.
What are you going to run containers on? You need VMs to power everything.
-
I'm not saying it's bad software, but the times of manually configuring VMs and LXC containers with a GUI or Ansible are gone.
All new build-outs are gitops and containerd-based containers now.
For the legacy VM appliances, Proxmox works well, but there's also Openshift virtualization aka kubevirt if you want take advantage of the Kubernetes ecosystem.
If you need bare-metal, then usually that gets provisioned with something like packer/nixos-generators or cloud-init.
You are going to what, install Kubernetes on every node?
It is far easier and more flexible to use VMs and maybe some VM templates and Ansible.
-
Proxmox is Debian.
I do always suggest installing Debian first, and then installing Proxmox on top. This allows you to properly set up your disks, and networking as needed, as the Proxmox installer is a bit limited: https://pve.proxmox.com/wiki/Install_Proxmox_VE_on_Debian_12_Bookworm
Once you have it up and running, have a look at the CT Templates. There's a whole set of pre-configured templates from TurnkeyLinux (again, debian+) that make it trivial to set up all kinds of services in lightweight LXC Containers.I do always suggest installing Debian first, and then installing Proxmox on top.
Correct me if I'm wrong, but isn't Proxmox it's own OS unto itself? What would be the advantage of installing Proxmox 'on top of' Debian when it's Debian already as you pointed out?
-
I'm still running a 6th-generation Intel CPU (i5-6600k) on my media server, with 64GB of RAM and a Quadro P1000 for the rare 1080p transcoding needs. Windows 10 is still my OS from when it was a gaming PC and I want to switch to Linux. I'm a casual user on my personal machine, as well as with OpenWRT on my network hardware.
Here are the few features I need:
- MergeFS with a RAID option for drive redundancy. I use multiple 12GB drives right now and have my media types separated between each. I'd like to have one pool that I can be flexible with space between each share.
- Docker for *arr/media downloaders/RSS feed reader/various FOSS tools and gizmos.
- I'd like to start working with Home Assistant. Installing with WSL hasn't worked for me, so switching to Linux seems like the best option for this.
Guides like Perfect Media Server say that Proxmox is better than a traditional distro like Debian/Ubuntu, but I'm concerned about performance on my 6600k. Will LXCs and/or a VM for Docker push my CPU to its limits? Or should I do standard Debian or even OpenMediaVault?
I'm comfortable learning Proxmox and its intricacies, especially if I can move my Windows 10 install into a VM as a failsafe while building a storage pool with new drives.
OP, I'm running Proxmox on and old Dell T320 /32gb RAM. I am not having any real issues doing so. I run Docker and a handful of Docker containers. I'm really not into the arr stack, but I wouldn't think you'd have much issue.
-
I'm not saying it's bad software, but the times of manually configuring VMs and LXC containers with a GUI or Ansible are gone.
All new build-outs are gitops and containerd-based containers now.
For the legacy VM appliances, Proxmox works well, but there's also Openshift virtualization aka kubevirt if you want take advantage of the Kubernetes ecosystem.
If you need bare-metal, then usually that gets provisioned with something like packer/nixos-generators or cloud-init.
Why would you install a GUI on a VM designated to run a Docker instance?
You should take a serious look at what actual companies run. It's typically nested VMs running k8s or similar. I run three nodes, with several VMs (each running Docker, or other services that require a VM) that I can migrate between nodes depending on my needs.
For example: One of my nodes needed a fan replaced. I migrated the VM and LXC containers it hosted to another node, then pulled it from the cluster to do the job. The service saw minimal downtime, kids/wife didn't complain at all, and I could test it to make sure it was functioning properly before reinstalling it into the cluster and migrating things back.
-
I do always suggest installing Debian first, and then installing Proxmox on top.
Correct me if I'm wrong, but isn't Proxmox it's own OS unto itself? What would be the advantage of installing Proxmox 'on top of' Debian when it's Debian already as you pointed out?
You have some options that aren't in the installer e.g. full disk encryption
-
You have some options that aren't in the installer e.g. full disk encryption
hmmmm. Wouldn't you have to remove the Debian kernal and use the Proxmox kernal? Sorry, not trying to be obtuse, I just have never installed Proxmox 'on top' of Debian. I always opted for the clean install.
-
You are going to what, install Kubernetes on every node?
It is far easier and more flexible to use VMs and maybe some VM templates and Ansible.
Yes.
It is not easier to use Ansible. My customers are trying to get rid of Ansible.
-
Why would you install a GUI on a VM designated to run a Docker instance?
You should take a serious look at what actual companies run. It's typically nested VMs running k8s or similar. I run three nodes, with several VMs (each running Docker, or other services that require a VM) that I can migrate between nodes depending on my needs.
For example: One of my nodes needed a fan replaced. I migrated the VM and LXC containers it hosted to another node, then pulled it from the cluster to do the job. The service saw minimal downtime, kids/wife didn't complain at all, and I could test it to make sure it was functioning properly before reinstalling it into the cluster and migrating things back.
I'm a DevOps/ Platform Engineering consultant, so I've worked with about a dozen different customers on all different sorts of environments.
I have seen some of my customers use nested VMs, but that was because they were still using VMware or similar for all of their compute. My coworkers say they're working on shutting down their VMware environments now.
Otherwise, most of my customers are running Kubernetes directly on bare metal or directly on cloud instances. Typically the distributions they're using are Openshift, AKS, or EKS.
My homelab is all bare metal. If a node goes down, all the containers get restarted on a different node.
My homelab is fully gitops, you can see all of my kubernetes manifests and nixos configs here:
-
Yes, but no. There is still a lot of places using old fashioned VMs, my company is still building VMs from an AWS ami and running ansible to install all the stuff we need. Some places will move to containers and that's great, but containers won't solve every problem
Yes, it's fine to still have VMs, but you shouldn't be building out new applications and new environments on VMs or LXC.
The only VMs I've seen in production at my customers recently are application test environments for applications that require kernel access. Those test environments are managed by software running in containers, and often even use something like Openshift Virtualization so that the entire VM runs inside a container.
-
Kubernetes is also designed for clustered workloads, so if you are mostly hosting on one or two machines, YAGNI applies.
I recommend people start w/ docker compose due to documentation, but I personally am switching to podman quadlets w/ rootless containers.
Yeah, definitely true.
I'm a big fan of single-node kubernetes though, tbh. Kubernetes is an automation platform first and foremost, so it's super helpful to use Kubernetes in a homelab even if you only have one node.
-
What are you going to run containers on? You need VMs to power everything.
I dont have any VMs running in my homelab.
Most of my customers run their Kubernetes nodes either on bare metal, or on a cloud provisioned VM from AWS/GCP/Azure etc
-
Yeah, definitely true.
I'm a big fan of single-node kubernetes though, tbh. Kubernetes is an automation platform first and foremost, so it's super helpful to use Kubernetes in a homelab even if you only have one node.
What's so nice about it? Have you tried quadlets or docker compose? Could you give a quick comparison to show what you one like about it?
-
I'm still running a 6th-generation Intel CPU (i5-6600k) on my media server, with 64GB of RAM and a Quadro P1000 for the rare 1080p transcoding needs. Windows 10 is still my OS from when it was a gaming PC and I want to switch to Linux. I'm a casual user on my personal machine, as well as with OpenWRT on my network hardware.
Here are the few features I need:
- MergeFS with a RAID option for drive redundancy. I use multiple 12GB drives right now and have my media types separated between each. I'd like to have one pool that I can be flexible with space between each share.
- Docker for *arr/media downloaders/RSS feed reader/various FOSS tools and gizmos.
- I'd like to start working with Home Assistant. Installing with WSL hasn't worked for me, so switching to Linux seems like the best option for this.
Guides like Perfect Media Server say that Proxmox is better than a traditional distro like Debian/Ubuntu, but I'm concerned about performance on my 6600k. Will LXCs and/or a VM for Docker push my CPU to its limits? Or should I do standard Debian or even OpenMediaVault?
I'm comfortable learning Proxmox and its intricacies, especially if I can move my Windows 10 install into a VM as a failsafe while building a storage pool with new drives.
Not calling you out specifically OP, but can someone tell me why this is a thing on the internet?
multiple 12GB drives
GB??? I assume TB automatically when people say this but it still is a speedbreaker when I'm thinking about the post.
-
yeah, and qemu and lxc are very much legacy at this point. Stick with docker/podman/kubernetes for containers.
QEMU is legacy? Pray tell me how you're running VMs on architectures other than x86 on modern computers without QEMU
-
I'm not saying it's bad software, but the times of manually configuring VMs and LXC containers with a GUI or Ansible are gone.
All new build-outs are gitops and containerd-based containers now.
For the legacy VM appliances, Proxmox works well, but there's also Openshift virtualization aka kubevirt if you want take advantage of the Kubernetes ecosystem.
If you need bare-metal, then usually that gets provisioned with something like packer/nixos-generators or cloud-init.
Sometimes, VMs are simply the better solution.
I run a semi-production DB cluster at work. We have 17 VMs running and it's resilient (a different team handles VMWare and hardware)
-
I'm still running a 6th-generation Intel CPU (i5-6600k) on my media server, with 64GB of RAM and a Quadro P1000 for the rare 1080p transcoding needs. Windows 10 is still my OS from when it was a gaming PC and I want to switch to Linux. I'm a casual user on my personal machine, as well as with OpenWRT on my network hardware.
Here are the few features I need:
- MergeFS with a RAID option for drive redundancy. I use multiple 12GB drives right now and have my media types separated between each. I'd like to have one pool that I can be flexible with space between each share.
- Docker for *arr/media downloaders/RSS feed reader/various FOSS tools and gizmos.
- I'd like to start working with Home Assistant. Installing with WSL hasn't worked for me, so switching to Linux seems like the best option for this.
Guides like Perfect Media Server say that Proxmox is better than a traditional distro like Debian/Ubuntu, but I'm concerned about performance on my 6600k. Will LXCs and/or a VM for Docker push my CPU to its limits? Or should I do standard Debian or even OpenMediaVault?
I'm comfortable learning Proxmox and its intricacies, especially if I can move my Windows 10 install into a VM as a failsafe while building a storage pool with new drives.
I’m surprised no one’s mentioned Incus, it’s a hypervisor like Proxmox but it’s designed to install onto Debian no prob. Does VMs and containers just like Proxmox, and snapshots too. The web UI is essential, you add a repo for it.
Proxmox isn’t reliable if you’re not paying them, the free people are the test people - and a bit back there was a bad update they pushed that broke shit. If I’d have updated before they pulled it, I’d have been hosed.
Basically you want a device that you don’t have to worry about updates, because updates are good for security. And Proxmox ain’t that.
On top of their custom kernel and stuff, it’s just less eyes than, say, the kernel Debian ships. Proxmox isn’t worth the lock-in and brittleness for just making VMs.
So to summarize, Debian and Incus installed. BTRFS if you’re happy with 1 drive or 2 RAID 1 drives. BTRFS gets scrubbing and bitrot detection (protection with RAID 1). ZFS for more drives. Toss on Cockpit too.
If you want less hands-on, do to OpenMediaVault. No room for Proxmox in my view, esp. for no clustering.
Also the iGPU on the 6600K likely is good enough for whatever transcoding you’d do (esp. if it’s rare and 1080p, it’ll do 4k no prob and multiple streams at once). The Nvidia card is just wasting power.
-
Yes, it's fine to still have VMs, but you shouldn't be building out new applications and new environments on VMs or LXC.
The only VMs I've seen in production at my customers recently are application test environments for applications that require kernel access. Those test environments are managed by software running in containers, and often even use something like Openshift Virtualization so that the entire VM runs inside a container.
Some of us don't build applications, we use them as built by other companies. If we're really unlucky they refuse to support running on a VM.
-
QEMU is legacy? Pray tell me how you're running VMs on architectures other than x86 on modern computers without QEMU
Not QEMU in particular, poor phrasing on my part. I just mean setting up new environments that run applications on VMs.
-
Not QEMU in particular, poor phrasing on my part. I just mean setting up new environments that run applications on VMs.
I prefer some of my applications to be on VMs. For example, my observability stack (ELK + Grafana) which I like to keep separate from other environments. I suppose the argument could be made that I should spin up a separate k8s cluster if I want to do that but it's faster to deploy directly on VMs, and there's also less moving parts (I run two 50 node K8S clusters so I'm not averse to containers, just saying). Easier and relatively secure tool for the right job. Sure, I could mess with cgroups and play with kernel parameters and all of that jazz to secure k8s more but why bother when I can make my life easier by trusting Red Hat? Also I'm not yet running a k8s version that supports SELinux and I tend to keep it enabled.
