I have a very opinionated opposing view of this research. A lot of research in this direction is working on raising the floor. Basically they just want the robot to handle a large variety of simple tasks and environments. Fair enough most industrial robots can't handle the smallest changes. But many times they implicitly make the assumption that raising the floor will also raise the ceiling. They assume, if it can generalize at 90%, it might also be able to do far more dextrous tasks that humans can. I think this is completely false, at best if we could do dextrous tasks in 1 environment this can transfer them to other environments with a presumably lower efficiency.
On the other hand, I think a more promising direction is to raise the ceiling of robot arm manipulation sky high. OpenAI kind of did this with Dactyl but I would like to see more of it. Can we get robotic arms to tie a shoelace, knit, perform pottery etc (with an arm like morphology, no special mechanisms). I think this can actually then lead to large scale generalization, kind of what we are seeing happening with NERF's now. I would like a robot arm NERF, overfit to one hard task but reproduces it with human like precision and dexterity. Deepminds approach to me (with GATO, robocat) seems like a red herring, they will never reach the kind of results we want from our arms.
Deepmind is google, and google suffers from chronical dabbling and never shipping products, I'd be surprised if they even care much about generalizing to useful tasks.
And about floor vs ceiling, what's really important is robustness, only robust robots can be deployed in the wild, at this time Dactyl with all its fingers is still too difficult to control, RoboCat got the grippers right, the problem really is they are again doing cute things with large models instead of raising robustness.
On the other hand, I think a more promising direction is to raise the ceiling of robot arm manipulation sky high. OpenAI kind of did this with Dactyl but I would like to see more of it. Can we get robotic arms to tie a shoelace, knit, perform pottery etc (with an arm like morphology, no special mechanisms). I think this can actually then lead to large scale generalization, kind of what we are seeing happening with NERF's now. I would like a robot arm NERF, overfit to one hard task but reproduces it with human like precision and dexterity. Deepminds approach to me (with GATO, robocat) seems like a red herring, they will never reach the kind of results we want from our arms.