There is an unavoidable endgame as academic fields evolve: groups whose ideas have stopped generating cutting edge advancements have to go deeper into rhetoric to compensate, arguing that the cutting edge is actually what they were talking about all along. The need to constantly justify your existence distracts from using your methods to contribute to problems people care about, and your part of the field gets further behind.
The result of the last 5 years is like seeing a world class professional try to solve problems in his sleep. He can do many amazing things way better than any human, but still can't reason coherently since he is asleep. An awake mouse is still much better than a sleeping human at many problems, so there are limits to what you can do with this approach.
It applies pretty dang well. The first image generator I remember was literally titled Deep Dream. As far as I can tell, the architecture being used for this stuff doesn't allow for self-reflection or focused learning. All training examples go into a global melting pot, there's no history or "story".
It's becoming pretty apparent that deep-learning is equivalent to what people do. Get a bunch of examples, practice a lot, and skills become automatic and pop out of the subconscious on demand. Human magic is in building a library of these skills and choosing what to use when.
Getting to general intelligence is going to require generalizing specific intelligence. Doesn't matter how much compute you throw at it, the system needs to be situated in time and capable of linking events together.
It might be as simple as chaining current goals together and applying them internally. "Create memory from image", "find memories that match input", "draw output from memories". Maybe that's what midjourney is doing...
Its possible, but current developments looks very much like what you would expect to get from a smart and smarter but still sleeping agent. If deep learning can solve the awake part then it is very likely that there is a technique we are completely missing.
But in a way that would be exciting, we have already models that work like a sleeping genius programmer, or a sleeping author etc, if we can somehow get those to wake up then we would have some form of AGI.
> A third possibility, which I personally have spent much of my career arguing for, aims for middle ground: “hybrid models”
So this person witnessed the last 5 years of DL advancements and didn’t update their beliefs at all?