Docker in LXC vs VM
-
I can't say much to docker in LXC as I'm not using it, I vaguely remember some limitation I've read of but if it works fine for you those don't seem to apply.
A VM has more overhead than an LXC, but with several LXCs maybe a single VM wins on overhead.
I currently have most Docker containers in one VM and am thinking about splitting it, the main reason is that 2 deployments have way larger volumes than the rest. This leads to the snapshots of the VM being very large as well and if I would need to restore from snapshots for a "small" application, it would take super long because of the large ones.
A single VM may be a bit easier on maintenance than several LXCs.
If you don't have a specific reason to switch, I would not.
-
I have been run Docker container in both LXC and VM for a long time without issues or meaningful performance penalties. So I run important single docker containers on top of LXC and everything else in Dockge / Portainer VMs.
-
What's the purpose of running container in a container? Why not install docker on your host machine?
-
If you do that, Docker is stuck on that host. If it’s in an LXC it can move to another host. Plus, backing up and snapshotting are easier IMO.
-
You can also create a single LXC for Docker and run multiple Docker containers on it. The VM argument is for security as it’s harder to escalate to the host from a VM than from an LXC.
-
Snapshotting in docker is as easy as
docker commit. After that you can back it up withdocker save. Then move to another host, but not without downtime.However normally you need to backup/move only volumes attached to containers. If that's not the way how you like to organize your services, you likely don't need docker.
-
I personally like lxc's over vms for my home lab and i run a dedicated lxc for docker and one running a single node k8s.
-
You could create a fresh container, install docker, and create a new template image from it. This way the overhead of installing disapears. The overhead in resource usage for each docker installation would remain the same as before.
As mentioned in another reply, you could run several container in one lxc. For example with docker compose or podman. Since I have no experience with podman but with docker compose, docker compose is pretty simple.
But all in all, I prefer to install everything "bare metal" in lxc containers. The main reason is, I don't want to mess around with the extra layer of configurating ports etc.
-
Is your server a dedicated server, or a VPS? Because if it's a VPS, you're probably already running in a VM.
Adding a VM might provide more security, especially if you aren't an expert in LXC security configuration. It will add overhead. Running Docker inside Docker provides nothing but more overhead and unnecessary complexity to your setup.
Also, because it isn't clear to me from your post: LXC and Docker are two ways of doing the same thing, using the same Kernel capabilities. Docker was, in fact, written in top of LXC. The only real difference is the container format. Saying "running Docker on LXC" is like saying "running Docker on Docker," or "running Docker on Podman," or "running LXC on Docker". All you're doing is nesting container implementations. As opposed to VMs, which do not just use Linux namespace capabilities, and which emulate an entirely different computer.
LXC, Podman, and Docker use the underlying OS kernel and resources. VMs create new, virtual hardware (necessarily sharing the same hardware architecture, but nothing else from the host) and run their own kernels.
Saying "Docker VM" is therefore confusing. Containers - LXC, Podman, or Docker - don't create VMs. They partition and segregate off resources from the host, but they do not provide a virtual machine. You can not run OpenBSD in a Docker container on Linux; you can run OpenBSD in a VM on Linux.
-
Dockers 'take-over-system' style of network management will interfere with proxmox networking.
-
Regardless of VM or LXC, I would only install docker once. There's generally no need to create multiple docker VMs/LXCs on the same host. Unless you have a specific reason; like isolating outside traffic by creating a docker setup for only public services.
Backups are the same with VM or LXC on Proxmox.
The main advantages of LXC that I can think of:
- Slightly less resource overhead, but not much (debian minimal or alpine VM is pretty lightweight already).
- Ability to pass-through directories from the host.
- Ability to pass-through hardware acceleration from a GPU, without passing through the entire GPU.
- Ability to change CPU cores or RAM while it's running.
-
Docker doesn't need to portable because containers are...
I don't even understand this logic.
-
Run Docker at the host level. Every level down from there is not only a knock to performance across the spectrum, it just makes a mess of networking. Anyone in here saying "it's easy to backup in a VM" has completely missed the point of containers, and apparently does not understand how to work with them.
You shouldn't ever need to backup containers, and if you're expecting data loss if one goes away, yerdewinitwrawng.
-
I use individual lxc for each docker compose so I don't have to revert 8 services at once if I need to restore.
I would also argue that an alpine lxc runs in 22mb ram by itself ... Significantly smaller footprint on disk and in memory. But most importantly, lxc can actually share memory space effectively, one doesn't need to reserve blocks of ram.
-
Lxc and docker are not equivalent. They are system and software containers respectively.
-
If you use Live Migrate, realize that it doesn't work on an LXC, only VMs. Your containers will be restarted with the LXC on the new node.
-
I used to use LXC, and switched to VM since internet said it was better.
I kinda miss the LXC setup. Day to day I don't notice any difference, but increasing storage space in VM was a small pain compared to LXC. In VM I increased disk size through proxmox, but then I had to increase the partition inside VM.
In LXC you can just increase disk size and it immediately is available to the containers
-
This thread has raised so many questions I'd like answered:
- Why are people backing up containers?
- Why are people running docker-in-docker?
- I saw someone mention snapshotting containers...what's the purpose of this?
- Why are people backing up docker installs?
Seriously thought I was going crazy reading some of these, and now I'm convinced the majority of people posting suggestions in here do not understand how to use containers at all.
Flat file configs, volumes, layers, versioning...it's like people don't know what these are are how to use them, and that is incredibly disconcerting.
-
Because a lot of people don’t learn docker, they install docker because some software they want to use is distributed that way.
-
Dont listen to them! The main issue with containers vs vm is security as you lxc runs in the hosts, while a vm runs on the host.
Use what you are familiar with and remember that lxc are containers and docker are containers, but the use of them are vastly different.
