Help with Home Server Architecture and Hardware Selection?

[libretech@reddthat.com]

Tl;dr

I have no idea what I’m doing, and the desire for a NAS and local LLM has spun me down a rabbit hole. Pls send help.

Failed Attempt at a Tl;dr

Sorry for the long post! Brand new to home servers, but am thinking about building out the setup below (Machine 1 to be on 24/7, Machine 2 to be spun up only when needed for energy efficiency); target budget cap ~ USD 4,000; would appreciate any tips, suggestions, pitfalls, flags for where I’m being a total idiot and have missed something basic:

Machine 1: TrueNAS Scale with Jellyfin, Syncthing/Nextcloud + Immich, Collabora Office, SearXNG if possible, and potentially the *arr apps

On the drive front, I’m considering 6x Seagate Ironwolf 8TB in RAIDz2 for 32TB usable space (waaay more than I think I’ll need, but I know it’s a PITA to upgrade a vdev so trying to future-proof), and I am thinking also want to add in an L2ARC cache (which I think should be something like 500GB-1TB m.2 NVMe SSD); I’d read somewhere that back of the envelope RAM requirements were 1GB RAM to 1TB storage (though the TrueNAS Scale hardware guide definitely does not say this, but with the L2ARC cache and all of the other things I’m trying to run I probably get to the same number), so I’d be looking for around 48GB (though I am under the impression that using an odd number of DIMMs isn’t great for performance, so that might bump up to 64GB across 4x16GB?); I’m ambivalent on DDR4 vs. 5 (and unless there’s a good reason not to, would be inclined to just use DDR4 for cost), but am leaning ECC, even though it may not be strictly necessary

Machine 2: Proxmox with LXC for Llama 3.3, Stable Diffusion, Whisper, OpenWebUI; I’d also like to be able to host a heavily modded Minecraft server (something like All The Mods 9 for 4 to 5 players) likely using Pterodactyl

I am struggling with what to do about GPUs here; I’d love to be able to run the 70b Llama 3.3, it seems like that will require something like 40-50GB VRAM to run comfortably at a minimum, but I’m not sure the best way to get there; I’ve seen some folks suggest 2x3090s is the right balance of value and performance, but plenty of other folks seem to advocate for sticking with the newer 4000 architecture (especially with the 5000 series around the corner and the expectation prices might finally come down); on the other end of the spectrum, I’ve also seen people advocate for going back to P40s

Am I overcomplicating this? Making any dumb rookie mistakes? Does 2 machines seems right for my use cases vs. 1 (or more than 2?)? Any glaring issues with the hardware I mentioned or suggestions for a better setup? Ways to better prioritize energy efficiency (even at the risk of more cost up front)? I was targeting something like USD 4,000 as a soft price cap across both machines, but does that seem reasonable? How much of a headache is all of this going to be to manage? Is there a light at the end of the tunnel?

Very grateful for any advice or tips you all have!

---

Hi all,

So sorry again for the long post. Just including a little bit of extra context here in case it’s useful about what I am trying to do (I feel like this is the annoying part of an online recipe where you get a life story instead of the actual ingredient list; I at least tried to put that first in this post.) Essentially I am a total noob, but have spent the past several months lurking on forums, old Reddit and Lemmy threads, and have watched many hours of YouTube videos just to wrap my head around some of the basics of home networking, and I still feel like I know basically nothing. But I felt like I finally got to the point where I felt that I could try to articulate what I am trying to do with enough specificity to not be completely wasting all of your time (I’m very cognizant of Help Vampires and definitely do not want to be one!)

Basically my motivation is to move away from non-privacy respecting services and bring as much in-house as possible, but (as is frequently the case), my ambition has far outpaced my skill. So I am hopeful that I can tap into all of your collective knowledge to make sure I can avoid any catastrophic mistakes I am likely to blithely walk myself into.

Here are the basic things I am trying to accomplish with this setup:

• A NAS with a built in media server and associated apps
• Phone backups (including photos) 
• Collaborative document editing
• A local ChatGPT 4 replacement 
• Locally hosted metasearch
• A place to run a modded Minecraft server for myself and a few friends

The list in the tl;dr represent my best guesses for the write software and (partial) hardware to get all of these done. Based on some of my reading, it seemed that a number of folks recommend running TrueNAS baremetal as opposed to in ProxMox for when there is an inevitable stability issue, and that got me thinking more about how it might be valuable to split out these functions across two machines, one to hand heavier workloads when needed but to be turned off when not (e.g. game server, all local AI), and a second machine to function as a NAS with all the associated apps that would hopefully be more power efficient and run 24/7.

There are two things that I think would be very helpful to me at this point:

1. High level feedback on whether this strategy sounds right given what I am trying to accomplish. I feel like I am breaking the fundamental Keep It Simple Stupid rule and will likely come to regret it.
2. Any specific feedback on the right hardware for this setup.
3. Any thoughts about how to best select hardware to maximize energy efficiency/minimize ongoing costs while still accomplishing these goals.

Also, above I mentioned that I am targeted around USD 4,000, but I am willing to be flexible on that if spending more up front will help keep ongoing costs down, or if spending a bit more will lead to markedly better performance.

Ultimately, I feel like I just need to get my hands on something and start screwing things up to learn, but I’d love to avoid any major costly screw ups before I just start ordering parts, thus writing up this post as a reality check before I do just that.

Thanks so much if you read this far down the post, and for all of you who share any thoughts you might have. I don’t really have folks IRL I can talk to about these sorts of things, so I am extremely grateful to be able to reach out to this community. -------

[illness@infosec.pub]

Reading the title and looking at the thumbnail, I was thinking, "sure I'll do a good deed and help out a noob." Then I read your post and I realized you know what you're doing better than me.

HomerInBushes.gif

[libretech@reddthat.com]

Thank you for this! Honestly maybe it's just been all of the Youtubers I watch but I constantly feel like I have no idea about how to make things work (and also, to be fair, basically everything I wrote is just me reading what other people who seem to know what they're talking about think and then trying to fit all the pieces together. I sort of feel like a money at a typewriter in that way.) Really appreciate you commenting though! It's given me a little more confidence

[cm0002@lemmy.world]

Stick with DDR4 ECC for a server environment, if you want to not be limited to 70b models, id dump more money in trying to snag more GPUs, otherwise you'd probably be fine with the 3000 series as long as you meet vRAM requirements

Have you considered secondary variables? Where are you going to run this? If you're running it in your house this is going to be noisy and power hungry. What room are you running it in? What's the amperage of the lines going to the outlets there? Is your house older? It's probably a 20 amp on a shared circuit and really easy to overload and cause a fire

This is what happens when you overload a homes circuit lines

[aberrate_junior_beatnik@midwest.social]

How did the breaker not trip on that? It had one job

[cm0002@lemmy.world]

The way the electrician explained it to me at the time was that I didn't technically exceed 20 AMPs but I was running close to it for sustained long periods of time heating up the wire in the wall and outlet slowly melting it over time until it finally buckled causing a small fire and then tripping the breaker

[blisterexe@lemmy.zip]

you seem pretty on track, and being broke i haven't looked at the expensive stuff you're considering, so i can't give you any value tips.

However, i would like to point out that if you're just going to be hosting minecraft game servers crafty controller is a much easier to setup&use tool than pterodactyl

[libretech@reddthat.com]

Thanks so much! Appreciate the DDR4 and DRAM thoughts, and great point on secondaries. I have actually been debating the right place to put this as well. My ONT is in the basement (which is I feel like is probably the best place to put this from a noise perspective), though my sad cable company router is in a spare bedroom that I was considering as well (this option would require a little less rewiring, though honestly I'm probably going to have to either figure out how to run my own ethernet or hire out for it regardless of where I put it). No worries if not, but do you have a sense of what noise I might expect from the TrueNAS machine I am thinking of running 24/7 vs. the Proxmox that I won't be using all the time? I think I could live with occasional noise spikes, but having something loud 24/7 in a bedroom would probably be cruel. And huge thank you for the warning on power draw: I have not been considering amperage at all and will need to look into that to figure out what I can sustain without burning the house down. Are there any other secondary variables you'd recommend I should consider? Appreciate all of your thoughts!

[calamityjanitor@lemmy.world]

Would you consider making the LLM/GPU monster server as a gaming desktop? Depends on how you plan to use it, you could have a beast gaming PC than can do LLM/stable diffusion stuff when not gaming. You can install loads of AI stuff on windows, arguably easier.

[dageek247@fedia.io]

I know most of the less expensive used hardware is going to be server-shaped/rackmount. Don't go for it unless you have a garage or shed that you can stuff them in. They put out jet-engine levels of noise and require god tier soundproofing in order to quiet them. The ones that are advertised as quiet are quiet as compared to other server hardware.

You can grab an epyc motherboard that is ATX and will do all you want, and can then move it to a rackmount later if you end up going that way.

The NVIDIA launch has been a bit of a paper one. I don't expect the prices of anything else to adjust down, rather the 5090 may just end up adjusting itself up. This may change over time, but the next couple of months aren't likely to have major deals worth holding out for.

[arbiterxero@lemmy.world]

Given the price of P40’s on eBay vs the price you can get 3090’s for, fuck the P40’s, in rocking quad 3090’s and they kick ass.

Also, Pascale is the OLDEST hardware supported…….. for how long?

[libretech@reddthat.com]

Thank you! Honestly I am probably going way overboard myself (I think I've tried to convince myself that it might make sense given the likelihood of tariffs around the corner, but honestly might still end up downscaling to more like 10-20TB of storage and radically reduce my LLM expectations). And thanks also for the Crafty Controller rec, I hadn't heard of them and will definitely check them out!

[atzanteol@sh.itjust.works]

Am I overcomplicating this?

I fear that you may be overthinking things a bit. For a home server I wouldn't worry about things like min/maxing memory to storage sizes. If you're new to this then sizing can be tricky.

For a point of reference - I'm running a MD RAID5 with 4TiB x 4 disks (12TiB usable) on an old Dell PowerEdge T110 with 8GiB of RAM. It's a file server (NFS) that does little else (just a bind9 server and dhcpd). I've had disks fail in the RAID but I've never had a 2 disk failure in 10+ years. I always keep my fileserver separate so that I can keep it simple and stable since everything else depends on it. I also do my backups to and from it so it's a central place for all storage.

That's just a file-server. I have 3 proxmox servers of widely variable stats from acquired machines.. An old System76 laptop with 64GiB RAM (and NVidia 1070 GTX that is used by Jellyfin), a Lenovo Thinkserver with 16GiB RAM, and an old Dell Z740 with 120GiB RAM (long story).

None of these servers are speed demons by any current standards, but they support a variety of VMs comfortably (home assistant, jellyfin, web sever, DNS, DHCP, a 3 node microk8s cluster running searxng, subsonic, a docker registry etc.)

RAM has always mattered more to me for servers. The laptop is the most recent and has 8 cores, the Lenovo only has 4.

Could things be faster? Sure. Do they perform "well enough for me?" Absolutely. I'm not as worried about low-power as you seem to be but my point is that you can get away with pretty modest hardware for MOST of the types of things you're looking to do.

The AI one is the thing to worry about - that could be your entire budget. VRAM is king for LLMs and gets pricey quick. My personal laptop's NVidia 3070 with 8GiB VRAM runs models that fit in that low amount of memory just fine. But I'm restricted to models that fit...

[tsesejuer@lemmy.world]

check out serverpartdeals

[libretech@reddthat.com]

This is a great point and one I sort of struggled with tbh; I think you're right that if I built it out as a gaming PC I would probably use Windows (not to say I am not very excited about the work Steam is doing for Linux gaming, it's just hard to beat the native OS). I was leaning toward a Linux build for the server form though just to try to embrace a bit more FOSS (and because I am still a little shocked that Microsoft could propose the Recall feature with a straight face). Maybe I could try a gaming setup that uses some flavor of Linux as a base, though then I am not sure I take advantage of the ability to use the AI stuff easier. Will definitely think more on it though, thanks for raising this!

[landedgentry@lemmy.zip]

It’s easy to feel like you know nothing when A) there’s seemingly infinite depth to a skill and B) there are so many options.

You’re letting perfection stop you from starting my dude. Dive in!

[landedgentry@lemmy.zip]

Oh damn that’s YOUR photo lmao

[libretech@reddthat.com]

Thanks for this! The jet engine sound level and higher power draw were both what made me a little wary of used enterprise stuff (plus jumping from never having a home server straight to rack mounted felt like flying a little too close to the sun). And thanks also for the epyc rec; based on other comments it sounds like maybe pairing that with dual 3090s is the most cost effective option (especially because I fear you're right on prices not being adjusted downward; not sure if the big hit Nvidia took this morning because of DeepSeek might change things but I suppose that ultimately unless underlying demand drops, why would they drop their prices?) Thanks again for taking the time to respond!

[andres4ny@social.ridetrans.it]

@cm0002 @aberrate_junior_beatnik That looks like a 15A receptacle (https://www.icrfq.net/15-amp-vs-20-amp-outlet/). If it was installed on a 20A circuit (with a 20A breaker and wiring sized for 20A), then the receptacle was the weak point. Electricians often do this with multiple 15A receptacles wired together for Reasons (https://diy.stackexchange.com/questions/12763/why-is-it-safe-to-use-15-a-receptacles-on-a-20-a-circuit) that I disagree with for exactly what your picture shows. That said, overloading it is not SUPER likely to cause a fire - just destroy the outlet and appliance plugs.

[libretech@reddthat.com]

Thanks, will do!