Jim Keller calls CUDA NVidia's swamp (not their moat) and I think that's so so so accurate.
Nvidia's fief has a lot of folks in it, who've built sloppy stuff all over the place because they can. Most of that won't get replaced.
But the modern tooling like PyTorch and its underlying JAX doesn't need or care about this legacy. The future can build far more intimately & with deliberation to folks who haven't such large swamps around their ecosystems. We can tune systems more efficiently & carefully when our vendors don't keep us so so far away from the core hardware. Nvidia has to keep maintaining this huge footprint of software, keep it going themselves, supporting a vast legacy, while everyone else gets to innovate forward, with much better access to the hardware.
> Nvidia has to keep maintaining this huge footprint of software, keep it going themselves, supporting a vast legacy, while everyone else gets to innovate forward, with much better access to the hardware.
When I hit an input box on my phone, a keyboard pops up on the screen, with the first row saying QWERTYUIOP, just like the Sholes and Glidden typewriter of 1878. I'm waiting for innovation on this first, then I'll look for innovation beyond CUDA.
Wat. The keyboard you pop up this way will autocomplete your words, autocorrect your typos, and allow you to input whole words by swiping, twirling gestures. It's very much not your legacy typewriter keyboard, even though the order of the letters is the same. Maintaining the order of a couple dozen keys is not expensive or hard.
I thought the opposite: if CUDA were easy to replace, wouldn't it have been replaced already?
PyTorch supports both ROCm and Ibex, but it's support is spotty and incomplete to the point where it becomes a pain to use despite AMD spending billions on improving its software layer.
NVIDIA is reaping a decade of design decisions on CUDA compatibility made less sloppily than its competitors.
Nvidia's fief has a lot of folks in it, who've built sloppy stuff all over the place because they can. Most of that won't get replaced.
But the modern tooling like PyTorch and its underlying JAX doesn't need or care about this legacy. The future can build far more intimately & with deliberation to folks who haven't such large swamps around their ecosystems. We can tune systems more efficiently & carefully when our vendors don't keep us so so far away from the core hardware. Nvidia has to keep maintaining this huge footprint of software, keep it going themselves, supporting a vast legacy, while everyone else gets to innovate forward, with much better access to the hardware.