Jim Keller, the legendary processor architect who worked on x86, Arm, MISC and RISC-V processors, this weekend criticized Nvidia's CUDA architecture and software stack and compared it to x86, calling it a quagmire. He noted that even Nvidia itself has several specialized software packages that rely on open source frameworks for performance reasons.
CUDA is a swamp, not a moat. x86 was also a swamp. […] CUDA is not pretty. It was built by piling one thing at a time.
— wrote Keller.
https://twitter.com/jimkxa/status/1758943525662769498
Indeed, like x86, CUDA is gradually adding functionality while maintaining backward compatibility in software and hardware. This makes the Nvidia platform backwards compatible, but it affects performance and complicates application development. Meanwhile, many open source software development frameworks can be used more efficiently than CUDA, reports Tom’s Hardware.
Basically no one writes CUDA. If you're writing CUDA, then it's probably not fast. […] There is a good reason why Triton, Tensor RT, Neon and Mojo exist.
Even Nvidia itself has tools that don't rely solely on CUDA. For example, Triton Inference Server — is an open source tool from Nvidia that makes it easy to deploy AI models at scale, supporting frameworks such as TensorFlow, PyTorch and ONNX. Triton also provides features such as model versioning, multiple model maintenance, and parallel model execution to optimize GPU and CPU resource utilization.
Nvidia's TensorRT is a high-performance deep learning optimizer and runtime library that accelerates deep learning on Nvidia GPUs. TensorRT takes trained models from various frameworks such as TensorFlow and PyTorch and optimizes them for deployment, reducing latency and increasing throughput for real-time applications such as image classification, object detection and natural language processing.
Course How to see the book. You will learn how to work with teams, form a team, plan budgets and find finances. Understand copyright, legal aspects and design concepts. Get to know more
And while architectures like Arm, CUDA and x86 can be considered a quagmire due to their relatively slow development, mandatory backward compatibility and bulkiness, these platforms are also not as fragmented as, for example, GPGPU, which not too bad.
It's unknown what Jim Keller thinks of AMD's ROCm and Intel's OneAPI, but it's clear that while he's spent many years of his life developing the x86 architecture, he's not thrilled about its future prospects. His statements also indicate that while he has worked for some of the largest chipmakers in the world, including Apple, Intel, AMD, Broadcom (and now Tenstorrent), we won't be seeing him on Nvidia's staff anytime soon.< /p>
The competition for ITS authors continues. Write an article about the development of games, gaming and gaming devices and win a professional gaming wheel Logitech G923 Racing Wheel, or one of the low-profile gaming keyboards Logitech G815 LIGHTSYNC RGB Mechanical Gaming Keyboard!