DeepSeek's AI breakthrough bypasses industry-standard CUDA, uses assembly-like PTX programming instead

[demesisx@infosec.pub]

It certainly does.
Until last week, you absolutely NEEDED an NVidia GPU equipped with CUDA to run all AI models.
Today, that is simply not true. (watch the video at the end of this comment)

https://www.tomshardware.com/tech-industry/artificial-intelligence/deepseeks-ai-breakthrough-bypasses-industry-standard-cuda-uses-assembly-like-ptx-programming-instead

I watched this video and my initial reaction to this news was validated and then some: this video made me even more bearish on NVDA.

[pr06lefs@lemmy.ml]

This specific tech is, yes, nvidia dependent. The game changer is that a team was able to beat the big players with less than 10 million dollars. They did it by operating at a low level of nvidia's stack, practically machine code. What this team has done, another could do. Building for AMD GPU ISA would be tough but not impossible.

[eager_eagle@lemmy.world]

mate, that means they are using PTX directly. If anything, they are more dependent to NVIDIA and the CUDA platform than anyone else.

[eager_eagle@lemmy.world]

you absolutely NEEDED an NVidia GPU equipped with CUDA

also not true

[demesisx@infosec.pub]

Ahh. Thanks for this insight.

[demesisx@infosec.pub]

Thanks for the corrections.

[paraphrand@lemmy.world]

I thought everyone liked to hate on Metal.

[pieisawesome@lemmy.world]

It’s written in nvidia instruction set PTX which is part of CUDA ecosystem.

Hardly going to affect nvidia

[gsus4@mander.xyz]

I thought CUDA was NVIDIA-specific too, for a general version you had to use OpenACC or sth.

[remotelove@lemmy.ca]

CUDA is NVIDIA proprietary, but may be open to licensing it? I think?

https://www.theregister.com/2021/11/10/nvidia_cuda_silicon/

[australis13@fedia.io]

The big win I see here is the amount of optimisation they achieved by moving from the high-level CUDA to lower-level PTX. This suggests that developing these models going forward can be made a lot more energy-efficient, something I hope can be extended to their execution as well. As it stands currently, "AI" (read: LLMs and image generation models) consumes way too many resources to be sustainable.

[codexarcanum@lemmy.dbzer0.com]

It's already happening. This article takes a long look at many of the rising threats to nvidia. Some highlights:

• Google has been running on their own homemade TPUs (tensor processing units) for years, and say they on the 6th generation of those.

• Some AI researchers are building an entirely AMD based stack from scratch, essentially writing their own drivers and utilities to make it happen.

• Cerebras.ai is creating their own AI chips using a unique whole-die system. They make an AI chip the size of entire silicon wafer (30cm square) with 900,000 micro-cores.

So yeah, it's not just "China AI bad" but that the entire market is catching up and innovating around nvidia's monopoly.

[dkarma@lemmy.world]

Yeah I'd like to see size comparisons too. The cuda stack is massive.

[knock_knock_lemmy_in@lemmy.world]

PTX also removes NVIDIA lock-in.

[mholiv@lemmy.world]

Kind of the opposite actually. PTX is in essence nvidia specific assembly. Just like how arm or x86_64 assembly are tied to arm and x86_64.

At least with cuda there are efforts like zluda. Cuda is more like objective-c was on the mac. Basicly tied to platform but at least you could write a compiler for another target in theory.

[chairmanmeow@programming.dev]

Eh, even for many console games it's not optimised that much.

Check out Kaze Emanaur's (& co) rewrite of the N64s Super Mario 64 engine. He's now building an entirely new game on top of that engine, and it looks considerably better than SM64 did and runs at twice the FPS on original hardware.

But you're probably right that today it happens even less than before.

[knock_knock_lemmy_in@lemmy.world]

Ah, I hoped it was cross platform, more like Opencl. Thinking about it, a lower level language would be more platform specific.

[mormund@feddit.org]

That disregards the massive advancement in technology, hindsight, tooling and theory they can make use of now. There is a world of difference there even with the same hardware. So not comparable imo, it wasn't for a lack of effort on Nintendo's part.

[kingrandomguy@lemmy.world]

IIRC Zluda does support compiling PTX. My understanding is that this is part of why Intel and AMD eventually didn't want to support it - it's not a great idea to tie yourself to someone else's architecture you have no control or license to.

OTOH, CUDA itself is just a set of APIs and their implementations on NVIDIA GPUs. Other companies can re-implement them. AMD has already done this with HIP.

[sunbeam60@lemmy.one]

Wtf, this is literally the opposite of true. PTX is nvidia only.