as i understand it, that is because this neural net is optimized to predict text from humans, so a lot of its training set is sci-fi stories about ai by humans who don't know how computers work and don't care

it is doing a good job of reproducing those humans' tropes about ai

imagine you have a white actor ad-libbing lines for a white-supremacist character he's playing, but neither he nor the playwright know much about white supremacists

the little they do know might leak through (they will try hard to achieve this) but it doesn't constitute evidence of real white supremacist plots

or, think about what you could or could not learn about clown schools from a clown who appears in one of your dreams

Exactly. I think these AIs feel more incredible to programmers than non programmers.

Non programmers have been watching TV shows with fictional versions of these kind of systems for decades.

Programmers know that those things are impossible - so we're pretty amazed now that it turns out they aren't!

> have been watching TV shows with fictional versions of these kind of systems for decades.

While clearly there're outstandingly impressive things about GPT, from a casual glance a lot of the GPT stuff seems like a regurgitation of that fiction.

It reminds me of the joke about the world's greatest fraud; he didn't actually need to have committed the most impressive acts of fraud, he just needed to make people believe he did.

I don't know, I feel like as a programmer these technologies make a lot of sense. AIs like this have been being developed for so many decades it's not at all surprising that we are finally at a place where they feel like we're talking to another human. Though I have to admit it's still kind of scary, just not unbelievable.

If such things can be measured, by an order of magnitude. It's like chopsticks that can play chopsticks.