I don't get that feeling. There are so many papers and so many avenues of research. It's extremely difficult for me to predict what's going to "pop" like the diffusion paper, the transformer paper, AlphaZero, Chat GPT-3, etc. But even these research or product advances that seem like step functions are built on a lot of research and trial-and-error.
Can all 3 of these you listed be combined somehow? Hopefully, but I have no idea.
Don’t give too much importance to individual papers. At best, you’re mostly disregarding all the work that lead to it. At worst, you’re placing a lot of faith on an idea analyzed through rose-tinted glasses and presented with a lot of intentional omissions.
I mean the Zero Data reasoning directly cites previous work on the same principle. That in particular seems like a significant step forwards though - one of the main critiques of current methods is "humans don't learn by ingesting terabytes of common crawl, they learn from experience".
Of course some works are indeed genuinely good ideas. But sometimes things just don’t work out quite as well. Many nature papers in my field are dead ends applications-wise.
I’m not saying any of those works specifically are, just that research should be approached with a healthy dose of skepticism.
In some sense, they did. The world may not be full of human-like robots autonomously roaming the wastelands^Wurban landscape - but it's chock-full of actuators, sensors and batteries. There are sensors and actuators in your coffee maker. There are plenty of them in your car too, whether they control the wheels, or the angle of the mirrors, or the height of the windows, or the state of your door knob. Etc. And all of those robotic parts were mostly made by... larger robots in factories.
Both Intellect-2 and zero data reasoning work on LLMs ("Zero data reasoning" is quite a misleading name of a method. It's not very ground-breaking.) If you wanna see a major leap in LLMS, you should check out what InceptionLabs did recently to speed up inference by 16x using a diffusion model. (https://www.inceptionlabs.ai/)
Our algorithms for time-series reinforcement learning are abysmal compared to inference models. Despite the explosion of the AI field, robotics and self-driving are stuck without much progress.
I think this method has potential, but someone else needs to boil it down a bit and change the terminology because, despite the effort, this is not an easily digested article.
We're also nowhere close to getting these models to behave properly. The larger the model we make, the more likely it is to find loopholes in our reward functions. This holds us back from useful AI in a lot of domains.
I'll take a moment to comment on this, after reading the responses which challenge your conclusion.
This criticism is entirely justified for a narrow read of your point, that the specific and relatively-widely-disseminated papers/projects, themselves, represent specific progress towards e.g. take-off or AGI or SI.
But it's also unjustified to the extent that these particular papers are proxies for broader research directions—indeed, many of the other comments provide reading lists for related and prior work.
I.e. it's not that this or that particular paper is the hop. It's that the bunny is oriented in the right direction and many microhops are occurring. What one chooses to label a hop amid the aggregate twitches and movement is a question for pedants.
But when you try to run their code or use the product, it's either missing or doesn't perform as well as marketed in the paper. Personal recommendation for building mental resistance against AI hype is to:
- read the paper and the concrete claims, results and limitations
- download and run the code whenever possible
- test for out of distribution inputs and/or practical examples outside of the training set
- Continuous thought machines: temporally encoding neural networks (more like how biological brains work)
- Zero data reasoning: (coding) AI that learns from doing, instead of by being trained on giant data sets
- Intellect-2: a globally distributed RL architecture
I am not an expert in the field but this feels like we just bunny hopped a little closer to the singularity...