How Nvidia’s CUDA Monopoly in Machine Learning Is Breaking
I wish this were so, but I'm very, very sceptical. Everything I use is built to CUDA. Nothing actually works on non-NVidia hardware. "Actually works" is different from "theoretically works." A lot of what I use theoretically supports ROCm, but in practice, when you try to use it, you have enough bugs big and small, and things crash or don't work right.That means everything I build is locked to NVidia as well, and everything people build on top of my stuff will be locked to NVidia too. Trying to maintain compatibility is like making Wine run Windows apps natively.The lock-in is now very significant. Virtually no one else has shipped anything really stable and usable, and we're now a very long time into people coding and testing only on NVidia.I hear that higher-level APIs are being built, nominally intended to be cross-platform, but I suspect (1) those will go the same way (2) we'll want backwards-compatibility. I have code which uses old ML models, CuPy, or otherwise. That stuff is locked to NVidia. I'm buying NVidia. This means I'm only making patches to bugs impacting NVidia. Network effects.So long as most people buy NVidia, NVidia will have more engineering resources too. It's very hard for a smaller player to catch up or pass. See Intel for many decades.To break from NVidia would require either a gross failure in execution by NVidia or something fundamentally new. To break the monopoly, I could see new products which might deliver:- Very low latency- Support for conventional DDR in large quantity (so I can run GPT-3 models at home)- Much more general functionality (e.g. an open architecture, and integration with conventional Python to parallelize normal code)... or other significant new features. However, I don't see a lot of stuff like that being worked on, let alone coming out.