OpenAI hits back at DeepSeek with o3-mini reasoning model

[womble@lemmy.world]

Sure but i can run the decensored quants of those distils on my pc, I dont need to even open the article to know that openai isnt going to allow me to do that and so isnt really relevant.

[padit@feddit.org]

Well, honestly: I have this kind of computational power at my university, and we are in dire need of a locally hosted LLM for a project, so at least for me as a researcher, its really really cool to have that.

[brucethemoose@lemmy.world]

Dense models that would fit in 100-ish GB like mistral large would be really slow on that box, and there isn't a SOTA MoE for that size yet.

So, unless you need tons of batching/parallel requests, its... kinda neither here nor there?

As someone else said, the calculus changes with cheaper Strix Halo boxes (assuming those mini PCs are under $3K).

[zecg@lemmy.world]

I use 14b and it's certainly great for my modest highschool physics and python (to help the kids) needs, but for party games and such it's a drag its pop culture stops at mid 2023

[brucethemoose@lemmy.world]

Thing is, there are a lot of free APIs for 30B-70B class models.

Self hosting is great of course, and if 14B does the job then it does the job.

[teanut@lemmy.world]

Lucky you! I need to check my university's current GPU power but sadly my thesis won't be needing that kind of horsepower, so I won't be able to give it a try unless I pay AWS or someone else for it on my own dime.

[cupcakezealot@lemmy.blahaj.zone]

Someone please write a virus that deletes all knowledge from LLMs.

[blind3rdeye@lemm.ee]

Deleting data from them might not be feasible, but there are other tactics.

[kingthrillgore@lemmy.ml]

How this impacts me, someone who doesn't use AI:

[floquant@lemmy.dbzer0.com]

Yeah ok we get it, they just release the latest checkpoint of their continuously trained model whenever convenient and make big headlines out of it.