New Intel Processor and 192 GB/256 GB RAM
-
All current popular AI is meant to run on GPU. Why are you going to spend more money to run it on hardware for which it isn't intended?
-
Twice, because usually it's two sticks.
In any case, RAM failure is rare enough that quadrupling its chances is not gonna make any meaningful difference. Even if it does, RAM is the easiest thing to replace in a PC. Don't even need to go offline while waiting for a new stick. Someone who's got the cash to build that thing in the first place won't be too upset by the cost of another 32gb stick either, I don't think.
-
This isn’t really true — a lot of the newer MoE models run just fine on a CPU coupled with gobs of RAM. Yes, they won’t be quite as fast as a GPU, but getting 128GB+ of VRAM is out of reach of most people.
You can even run Deepseek R1 671b (Q8) on a Xeon or Epyc with 768GB+ of RAM, at 4-8 tokens/sec depending on configuration. A system supporting this would be at least an order of magnitude cheaper than a GPU setup to run the same thing.
-
If you need that much ram, I would look at getting a AMD Threadripper. If you need DDR5, the 7960X is their “budget” option, or if you are fine with DDR4, there is lots of used 3000 and 5000 series for cheap.
-
Well, anecdotal evidence of course but with the exception where a PSU blew up (and damaged a whole lot of things) I only ever have had RAM stick problems since like -95. Three times. Over some 30-40 PCs.
-
So if I had more memory channels it would be better to have say ollama use the cpu versus the gpu?
-
Because there's no advantage to having this much RAM in an economy build. If you're looking to max out your mainboard RAM then you're looking for a thread ripper anyways, not some economy i9...
-
He goes over the different ways to run a selfhost AI without a GPU, like you want to do, including maxing RAM and using PCI-e M.2 add-on boards.
-
The i9-10900 has 4 channels (Quadro-Channel DDR4-2933 (PC4-23466, 93.9GB/s).
would this be better in this way than an i9-14xxx (Dual-Channel DDR5-5600 (PC5-44800, 89.6GB/s))?does the numbers (93 GB/s and 89GB/s) mean the speed for a RAM-stick or the speed all together? maybe an old i9-10xxx with 4channel-ram was better than a new dual-channel.
-
Thank you very much! This leads to this article: https://forum.level1techs.com/t/deepseek-deep-dive-r1-at-home/225826/2
Maybe the 9959x is what I am looking for. -
Well, the numbers I find on google are: a Nvidia 4090 can transfer 1008 GB/s. And a i9 does something like 90 GB/s. So you'd expect the CPU to be roughly 11 times slower than that GPU at fetching numbers from memory.
-
Seems to mean all together. (5600MT/s / 1000) x 2 sticks x 64bit / 8bits/Byte = 89.6 GB/s
or 2933/1000 x 4 x 64bit / 8 = 93.9 GB/s
so they calculated with double the DDR bus width, or 4times the bus width.
-
OP wants to store all of their porn collection in RAM
