Can anybody explain why CUDA and Rocm are necessary and why OpenCL isn't the solution?
-
I've read multiple times that CUDA dominates, mostly because NVIDIA dominates. Rocm is the AMD equivalent, but OpenCL also exists. From my understanding, these are technologies used to program graphics cards - always thought that shaders were used for that.
There is a huge gap in my knowledge and understanding about this, so I'd appreciate somebody laying this out for me. I could ask an LLM and be misguided, but I'd rather not
Now, I don't write code. So I can't really tell you if this is the truth or not — but:
I hear that OpenCL is much more difficult and less accessible to write than CUDA code. CUDA is easier to write, and thus gets picked up and used by more developers.
Someone mentions CUDA "sometimes" having better performance, but I don't think it's only sometimes. I think that due to the existence of the tensor cores (which are really good at neural nets and matrix multiplication), CUDA has vastly better performance when taking advantage of those hardware features.
Tensor cores are not Nvidia specific, but they are the "most ahead". They have the most in their GPU's, and probably most importantly: CUDA only supports Nvidia, and therefore by extension, their tensor cores.
There are alternative projects, like how leela chess zero mentions tensorflow for google's Tensor Processing Units, but those aren't anywhere near as popular due to performance and software support.
-
AFIK it’s only NVIDIA that allows containers shared access to a GPU on the host.
This cannot be right. I'm pretty sure that it is possible to run OpenCL applications in containers that are sharing a GPU.
I should test this if I have time. My plan was to use a distrobox container since that shares the GPU by default and run something like lc0 to see if opencl acceleration works.
Now where is my remindme bot?
You really piqued my interest. I use docker/podman.
W/ an AMD graphics card, eglinfo on the host shows the card is AMD Radeon and driver is matching that.
In the container, without --gpus=all, it shows the card is unknown and the driver is "swrast" (so just CPU fallback).
To make --gpus=all work, it gives the error
docker: Error response from daemon: could not select device driver "" with capabilities: [[gpu]
I was doing a bad job searching before. I found that AMD can share the GPU, it just works a little differently in terms of how to launch the container.
https://rocm.docs.amd.com/projects/install-on-linux/en/latest/install/amdgpu-install.html#amdgpu-install-dkmsBut sadly my AMD GPU is too old/junk to have current driver support.
Anyways, appreciate the reply! Now I can mod my code to run on cheaper cloud instances.
(Note I'm an OpenGL/3D app developer, but probably OpenCL works about the same architecturally)
-
I've read multiple times that CUDA dominates, mostly because NVIDIA dominates. Rocm is the AMD equivalent, but OpenCL also exists. From my understanding, these are technologies used to program graphics cards - always thought that shaders were used for that.
There is a huge gap in my knowledge and understanding about this, so I'd appreciate somebody laying this out for me. I could ask an LLM and be misguided, but I'd rather not
ROCm is an implementation/superset of OpenCL.
ROCm ships its installable client driver (ICD) loader and an OpenCL implementation bundled together. As of January 2022, ROCm 4.5.2 ships OpenCL 2.2
Shaders are computational post-processing - think pixel position based adjustments to rendering.
OpenCL and CUDA are computation frameworks where you can use the GPU for other processing than rendering. You can use it for more general computing.
-
AFIK it’s only NVIDIA that allows containers shared access to a GPU on the host.
With the majority of code being deployed in containers, you end up locked into the NVIDIA ecosystem even if you use OpenCL. So I guess people just use CUDA since they are limited by the container requirement anyways.
That’s from my experience using OpenGL headless. If I’m wrong please correct me; I’d prefer being GPU agnostic.
Check implementations before saying shit like that.
Nvidia has historical bad open source driver support, which makes it hard for people to implement vGPU usage.
They actually actively blocked us from using their cards remotely, until COVID hit. Then they gave out the code to do it. They are still limiting customer level cards usage on virtualization cases. They had to give out a toolkit for us to be able to use their cards on docker. Other cards can be accessed just by sharing dev driver files to the volume. -
Check implementations before saying shit like that.
Nvidia has historical bad open source driver support, which makes it hard for people to implement vGPU usage.
They actually actively blocked us from using their cards remotely, until COVID hit. Then they gave out the code to do it. They are still limiting customer level cards usage on virtualization cases. They had to give out a toolkit for us to be able to use their cards on docker. Other cards can be accessed just by sharing dev driver files to the volume.Check Wolf implementation for context. It's a mess with Nvidia.
https://games-on-whales.github.io/wolf/stable/user/quickstart.html
-
Check implementations before saying shit like that.
Nvidia has historical bad open source driver support, which makes it hard for people to implement vGPU usage.
They actually actively blocked us from using their cards remotely, until COVID hit. Then they gave out the code to do it. They are still limiting customer level cards usage on virtualization cases. They had to give out a toolkit for us to be able to use their cards on docker. Other cards can be accessed just by sharing dev driver files to the volume.Can you share sample code I can try or documentation I can follow of using an AMD GPU in that way (shared, virtualized, using only open source drivers)?
-
Can you share sample code I can try or documentation I can follow of using an AMD GPU in that way (shared, virtualized, using only open source drivers)?
Check Wolf (in my other comment), it's the best example of GPU virtualization usage.
Otherwise you can check other docker images using GPU for computing, like jellyfin for instance, or nextcloud recognize, nextcloud memories and its transcoding instance,...
-
I've read multiple times that CUDA dominates, mostly because NVIDIA dominates. Rocm is the AMD equivalent, but OpenCL also exists. From my understanding, these are technologies used to program graphics cards - always thought that shaders were used for that.
There is a huge gap in my knowledge and understanding about this, so I'd appreciate somebody laying this out for me. I could ask an LLM and be misguided, but I'd rather not
Because Nvidia wrote an absolute fuck-ton of their Cuda stack to make it far more performant for many common tasks out of the box.
Early opencl implementations absolutely sucked.
-
I've read multiple times that CUDA dominates, mostly because NVIDIA dominates. Rocm is the AMD equivalent, but OpenCL also exists. From my understanding, these are technologies used to program graphics cards - always thought that shaders were used for that.
There is a huge gap in my knowledge and understanding about this, so I'd appreciate somebody laying this out for me. I could ask an LLM and be misguided, but I'd rather not
Nvidia has the money and influence to make CUDA a standard. Popular means better...
-
Nvidia has the money and influence to make CUDA a standard. Popular means better...
Calling nVidia popular seems wrong. There's not much choice in this space and that choice is pushed further in their favor by anticompetitive bullshit.
-
I've read multiple times that CUDA dominates, mostly because NVIDIA dominates. Rocm is the AMD equivalent, but OpenCL also exists. From my understanding, these are technologies used to program graphics cards - always thought that shaders were used for that.
There is a huge gap in my knowledge and understanding about this, so I'd appreciate somebody laying this out for me. I could ask an LLM and be misguided, but I'd rather not
I'm pretty sure OpenCL was just a play by Apple to standardize heterogeneous compute across different hardware companies and prevent CUDA from dominating
But then they deprecated it in favor of Metal which is just an Apple-specific thing. Probably because they were going to their own hardware anyway
So the main company pushing OpenCL is no longer pushing it, pretty sure it's dying out at this point
-
I've read multiple times that CUDA dominates, mostly because NVIDIA dominates. Rocm is the AMD equivalent, but OpenCL also exists. From my understanding, these are technologies used to program graphics cards - always thought that shaders were used for that.
There is a huge gap in my knowledge and understanding about this, so I'd appreciate somebody laying this out for me. I could ask an LLM and be misguided, but I'd rather not
Because CUDA and ROCm/HIP are far easier to program.
The Khronos competitor to CUDA/ROCm is SYCL not OpenCL.
SYCL vs these other options is a fun theoretical problem, but only Intel seems to be pushing SYCL at all. OpenCL got stuck in OCL1.2 (the 2.0 release was dead. 3.0+ OpenCL ignores OCL2.0 but it's too late, OpenCL is seen as a dead end tech these days).
The biggest issue is that OpenCL is a different language, while CUDA/HIP/SYCL are 'just' C++ extensions. This means that if you ever shared data between CPU and GPU in OpenCL (or DirectX or Vulkan for that matter), you have to carefully write and rewrite structs{} to line up between the two languages.
Meanwhile, CUDA/HIP support passing structs, classes and more between CPU and GPU (subject to conditions of course. GPUs can't do function pointers or vtables for example, but cpu-only classes can have vtables)
-
System shared this topic on
