Andrej Karpathy (who was one of the presenters today) actually made similar statements last year, about how the part of the software stack that is not neural networks is gradually shrinking: https://youtu.be/y57wwucbXR8?t=352