> We're seeing the opposite. Instead, all the competing models are basically converging on the same benchmark performance numbers...
That question is really important. Comments on that? Maybe LLMs are asymptotically approaching some fundamental limit for that technology.
What goes on inside an LLM hasn't changed much in years. More data and more compute is thrown at that little kernel that makes it all go. Additional gimmicks are added around the core algorithm. But there's not much progress down at the bottom.
There's no reaching into the net and extracting a reliable "don't know".
Is there an upper limit to what that architecture can do?
AGI may not be reachable by this route.
Maybe someone will find a way around that limit. The field has way too much money, too much visibility, and too many people. The technology works, although it has limits.
Money can't (necessarily) buy a breakthrough. It could take 10 or 100 or 1000 years, we just can't know. This kind of technical risk isn't usually so prominent. It's usually more diffuse with fallbacks, offramps, less palatable but workable alternatives.. This time it seems like the technical risk is looming very, very large.
That question is really important. Comments on that? Maybe LLMs are asymptotically approaching some fundamental limit for that technology.
What goes on inside an LLM hasn't changed much in years. More data and more compute is thrown at that little kernel that makes it all go. Additional gimmicks are added around the core algorithm. But there's not much progress down at the bottom. There's no reaching into the net and extracting a reliable "don't know". Is there an upper limit to what that architecture can do? AGI may not be reachable by this route.
Maybe someone will find a way around that limit. The field has way too much money, too much visibility, and too many people. The technology works, although it has limits.