Unless there is a significant increase in the effective context window in LLMs. Pursuing the goal of having agents working on complex goals is not going to work well. All the tricks and hacks trying to work around this problem is not going to fundamentally change that.
LLM agents will lose track of what they are trying to do after couple of trials. That's something that would differentiate human PhD is that while not fast or always creative, they have better attention memory span.
https://github.com/MiniMax-AI/MiniMax-01 is an open model that claims a 4 million context. Note however that longer context makes evaluation expensive as you are paying for every token. Still, it is true that OpenAI seriously needs a better solution for it.
I think it's time to partition the context into L1, L2, and L3 contexts. L1 is the current context with a quadratic memory requirement. L2 is based on fancy mechanisms such as what is used by Gemini and MiniMax-01, having a sub-quadratic to linear memory requirement. L3 is based on document and chunk embeddings having a linear to logarithmic memory requirement. LLMs don't use this approach, but I think it might make sense. As for how this partitioning would work at the neural layers, that remains to be determined.
Hoping this forces universities to change their archaic PhD system, to adapt something similar to european system. Then people who really have passion for a field and want to have normal job do their research, without working for labs as a cheap labor.
No, but you’ll get PhD-level ReactJS Hello World templates while people will be replaced by those agents. My vision of the future is bleak but I have yet to be proven wrong.
I’ve spent the day working in C++ with a popular, well known framework. Both ChatGPT and Claude have been horrible. They hallucinate APIs, they remove things that were already working for no apparent reason, they severely struggle with basic logic.
Even a big leap in model quality would still be dog shit at these kinds of tasks. I am vastly better at programming than even the best LLMs out there right now. The only real advantages they have over me are that they type a lot more quickly than I do, and their hourly rate is lower.
I’m not saying this is complete BS. I confess I feel a twinge of panic every time I see a headline like this. But years of working with these models and seeing how terrible they are for many tasks makes me skeptical that we’re suddenly about to get AGI.
Especially the way the companies make LLMs appear more human. It will be the same with "PhD level" ... utterly useless in terms of evolution or cognitive performance. Efficient and fast scripts that can take any request and data and give a precise result is all super-agents are good at.
And that's a lot. And we are happy.
The rest is just pretty standard PR & content media for another 10 years.
I'm easily in the top 95-99th percentile of people I know and probably most people in general in most respects. Net worth, IQ, height, athletic achievements...
A good 10-20 years ago I anticipated these AI moments, and even some of the specifics like our use of ANNs(when everyone thought they were a failure), and yet I failed to capitalize on it.
It's hard to describe just how defeated and irrelevant these news articles make me feel.
I don't say this to brag or anything, I keep this account pseudo anonymous. Some follow up points and questions
- What hope do the normies have?
- Does anyone expect our representatives to do anything?
LLM agents will lose track of what they are trying to do after couple of trials. That's something that would differentiate human PhD is that while not fast or always creative, they have better attention memory span.
reply