Can you run DeepSeek R1 on a AMD 7900 XTX 24GB GPU?

[mixedunivers@discuss.tchncs.de]

i also have a 6700xt but i don't get ollama running on it. it only defaults to the cpu ryzen 5600
I plan to tackle this problem on a free weekend and now i have a new Reason for solving it.

[gay4dudes@sh.itjust.works]

Well, I dont know what you are running, but on Debian or Fedora it automatically installed Drivers and picked the GPU. I had a Problem like this ones, where it had wrong Drivers (but it was in an NVIDIA GPU).

[fisch@discuss.tchncs.de]

I don't know how big the original model is but I have an RX 6700 XT and I can easily run the Llama 3 8B distill of Deepseek R1 with 32k context. I just haven't figured out how to get good results yet, it always does the <thinking><thinking/> thing.

[fisch@discuss.tchncs.de]

I have the same GPU but I always run 7B/8B variants as exl2. Do you use GGUF to use your system RAM?

[ieatpwns@lemmy.world]

Hey not to side track ops post or your own but I’m new to the home llm space and I was wondering once you have the model set up is there a gui? And how do you input tasks for it to do?

[gay4dudes@sh.itjust.works]

You can use the Terminal or something like AnythingLLM.
It has a GUI and you can import pictures and Websites.

[eager_eagle@lemmy.world]

check this out

https://apxml.com/posts/gpu-requirements-deepseek-r1

[eddybot@discuss.tchncs.de]

on some Linux distros like Arch Linux you might need to install a ollama-rocm package too

[domi@lemmy.secnd.me]

I run the 32b one on my 7900 XTX in Alpaca https://jeffser.com/alpaca/

There is no way to fit the full model in any single AMD or Nvidia GPU in existence.

[wuphysics87@lemmy.ml]

I run it on a 6700xt