Help with Home Server Architecture and Hardware Selection?

[email protected]

I know most of the less expensive used hardware is going to be server-shaped/rackmount. Don't go for it unless you have a garage or shed that you can stuff them in. They put out jet-engine levels of noise and require god tier soundproofing in order to quiet them. The ones that are advertised as quiet are quiet as compared to other server hardware.

You can grab an epyc motherboard that is ATX and will do all you want, and can then move it to a rackmount later if you end up going that way.

The NVIDIA launch has been a bit of a paper one. I don't expect the prices of anything else to adjust down, rather the 5090 may just end up adjusting itself up. This may change over time, but the next couple of months aren't likely to have major deals worth holding out for.

[email protected]

Given the price of P40’s on eBay vs the price you can get 3090’s for, fuck the P40’s, in rocking quad 3090’s and they kick ass.

Also, Pascale is the OLDEST hardware supported…….. for how long?

[email protected]

Thank you! Honestly I am probably going way overboard myself (I think I've tried to convince myself that it might make sense given the likelihood of tariffs around the corner, but honestly might still end up downscaling to more like 10-20TB of storage and radically reduce my LLM expectations). And thanks also for the Crafty Controller rec, I hadn't heard of them and will definitely check them out!

[email protected]

Am I overcomplicating this?

I fear that you may be overthinking things a bit. For a home server I wouldn't worry about things like min/maxing memory to storage sizes. If you're new to this then sizing can be tricky.

For a point of reference - I'm running a MD RAID5 with 4TiB x 4 disks (12TiB usable) on an old Dell PowerEdge T110 with 8GiB of RAM. It's a file server (NFS) that does little else (just a bind9 server and dhcpd). I've had disks fail in the RAID but I've never had a 2 disk failure in 10+ years. I always keep my fileserver separate so that I can keep it simple and stable since everything else depends on it. I also do my backups to and from it so it's a central place for all storage.

That's just a file-server. I have 3 proxmox servers of widely variable stats from acquired machines.. An old System76 laptop with 64GiB RAM (and NVidia 1070 GTX that is used by Jellyfin), a Lenovo Thinkserver with 16GiB RAM, and an old Dell Z740 with 120GiB RAM (long story).

None of these servers are speed demons by any current standards, but they support a variety of VMs comfortably (home assistant, jellyfin, web sever, DNS, DHCP, a 3 node microk8s cluster running searxng, subsonic, a docker registry etc.)

RAM has always mattered more to me for servers. The laptop is the most recent and has 8 cores, the Lenovo only has 4.

Could things be faster? Sure. Do they perform "well enough for me?" Absolutely. I'm not as worried about low-power as you seem to be but my point is that you can get away with pretty modest hardware for MOST of the types of things you're looking to do.

The AI one is the thing to worry about - that could be your entire budget. VRAM is king for LLMs and gets pricey quick. My personal laptop's NVidia 3070 with 8GiB VRAM runs models that fit in that low amount of memory just fine. But I'm restricted to models that fit...

[email protected]

check out serverpartdeals

[email protected]

This is a great point and one I sort of struggled with tbh; I think you're right that if I built it out as a gaming PC I would probably use Windows (not to say I am not very excited about the work Steam is doing for Linux gaming, it's just hard to beat the native OS). I was leaning toward a Linux build for the server form though just to try to embrace a bit more FOSS (and because I am still a little shocked that Microsoft could propose the Recall feature with a straight face). Maybe I could try a gaming setup that uses some flavor of Linux as a base, though then I am not sure I take advantage of the ability to use the AI stuff easier. Will definitely think more on it though, thanks for raising this!

[email protected]

It’s easy to feel like you know nothing when A) there’s seemingly infinite depth to a skill and B) there are so many options.

You’re letting perfection stop you from starting my dude. Dive in!

[email protected]

Oh damn that’s YOUR photo lmao

[email protected]

Thanks for this! The jet engine sound level and higher power draw were both what made me a little wary of used enterprise stuff (plus jumping from never having a home server straight to rack mounted felt like flying a little too close to the sun). And thanks also for the epyc rec; based on other comments it sounds like maybe pairing that with dual 3090s is the most cost effective option (especially because I fear you're right on prices not being adjusted downward; not sure if the big hit Nvidia took this morning because of DeepSeek might change things but I suppose that ultimately unless underlying demand drops, why would they drop their prices?) Thanks again for taking the time to respond!

[email protected]

@cm0002 @aberrate_junior_beatnik That looks like a 15A receptacle (https://www.icrfq.net/15-amp-vs-20-amp-outlet/). If it was installed on a 20A circuit (with a 20A breaker and wiring sized for 20A), then the receptacle was the weak point. Electricians often do this with multiple 15A receptacles wired together for Reasons (https://diy.stackexchange.com/questions/12763/why-is-it-safe-to-use-15-a-receptacles-on-a-20-a-circuit) that I disagree with for exactly what your picture shows. That said, overloading it is not SUPER likely to cause a fire - just destroy the outlet and appliance plugs.

[email protected]

Thanks, will do!

[email protected]

That's the solution I take. I use Proxmox for a Windows VM which runs Ollama. That VM can then be used for gaming in the off chance a LLM isn't loaded. It usually is. I use only one 3090 due to the power load of my two servers on top of my [many] HDDs. The extra load of 2 isn't something I want to worry about.

I point to that machine through LiteLLM* which is then accessed through nginx which allows only Local IPs. Those two are in a different VM that hosts most of my docker containers.

*I found using Ollama and Open WebUI causes the model to get unloaded since they send slightly different calls. LiteLLM reduces that variance.

[email protected]

Thanks so much for flagging that, the above 4g decoding wasn't even on my radar. And I think you and another commenter have sold me on trying for an EPYC mobo and dual 3090 combination. If you don't mind my asking, did you get your 3090's new or used? I feel like used is the way to go from a cost perspective, but obviously only if it wasn't used 24/7 in a mining rig for years on end (and I am not confident in my ability to make a good call on that as of yet. I guess I'd try to get current benchmarks and just try to visually inspect from photos?) But thanks again!

[email protected]

Yea, I keep the outlet around as a reminder lol

[email protected]

Thank you! I think I am just at the "Valley of Despair" portion of the Dunning-Kruger effect lol, but the good news is that it's hopefully mostly up from here (and as you say, a good finished product is infinitely better than a perfect idea).

[email protected]

They're the best site around for high quality/capacity drives that don't cost an arm and a leg. Another great resource for tools n' stuff is awesomeselfhosted

Website: https://awesome-selfhosted.net/

Github: https://github.com/awesome-selfhosted/awesome-selfhosted

[email protected]

Makes sense, this was also years ago so small details are being forgotten, could have also been a 15 or possibly 20. It was one circuit split between 2 rooms, which was the norm apparently for the time it was built in the early 80s (and not a damn thing was ever upgraded, including the outlets)

It was also a small extinguisher handleable fire, but it was enough to be scary AF LMAO

[email protected]

Don't worry about how a video card was used. Unless it was handled by howtobasic, they're gonna break long after they're obsolete. You might worry about a bad firmware setup, but you avoid that by looking at the seller rating, not the video card.

there's an argument to be made that a mining gpu is actually the better card to buy since they never went hot>cold>hot>cold (thus stressing the solder joints) like a regular user would do. But it's just that; an argument. I have yet to find a well researched article on the effects of long-term gaming as compared to long term mining, but I can tell you that the breaking point for either is long after you would have kept the card in use, even second or third hand.

[email protected]

So, I'm a rabid selfhoster because I've spent too many years watching rugpull tactics from every company out there. I'm just going to list what I've ended up with, and it's not perfect, but it is pretty damn robust. I'm running pretty much everything you talk about except much in the way of AI stuff at this point. I wouldn't call it particularly energy efficient since the equipment isn't very new. But take a read and see if it provokes any thoughts on your wishlist.

My Machine 1 is a Proxmox node with ZFS storage backing and machine 2 is mirror image but is a second Proxmox node for HA. Everything, even my OPNsense router runs on Proxmox. My docker/k8s hosts are LXCs or VMs running on the nodes, and the nodes replicate nearly everything between them as a first level, fast recovery backup/high availability failover. I can then live migrate guests around very quickly if I want to upgrade and reboot or otherwise maintain a node. I can also snapshot guests before updates or maintainance that I'm scared will break stuff. Or if I'm experimenting and like to rollback when I fuck up.

Both nodes are backed up via Proxmox Backup Server for any guests I consider prod, and I take backups every hour and keep probably 200 backups at various intervals and amounts. These dedup in PBS so the space utilization for all these extra backups is quite low. I also backup via PBS to removable USB drives on a longer schedule, and swap those out offsite weekly. Because I bind mount everything in my docker compose stacks, recovering a particular folder at a point in time via folder restore lets me recover a stack quite granularly. Also, since it's done as a ZFS snapshot backup, it's internally consistent and I've never had a db-file mismatch issue that didn't just journal out cleanly.

I also zfs-send critical datasets via syncoid to zfs.rent daily from each proxmox node.

Overall, this is highly flexible and very, very bulletproof over the last 5 or 6 years. I bought some decade old 1-U dell servers with enough drive bays and dual xeons, so I have plenty of threads and ram and upgraded to IT-mode 12G SAS RAID cards , but it isn't a powerhouse server or anything, I might be $1000 into each of them. I have considered adding and passing through an external GPU. The PBS server is a little piece of trash i3 with a 8TB sata drive and a GB NIC in it.

[email protected]

I’m rocking 4 used ones from 4 different people.

So far, all good

You can’t buy 3090’s new anymore anyways.

4090’s are twice as much for 15% better perf, and the 5090’s will be ridiculous prices.

2x3090 is more than enough for basic inference, I have more for training and fine tuning.

You want epyc/threadrupper etc.

You want max pcie lanes.

agnos.is Forums

Help with Home Server Architecture and Hardware Selection?