There's a neat argument against these models doing interpolation: the manifold of the data is so sparse that it's infinitesimally unlikely for a good predictor to be doing interpolation between existing points on the manifold.

https://arxiv.org/abs/2110.09485

Yes, nice, I know some of these words

Maybe what transformers actually transform are us..

Autobots, print out.