> Well, `cp` would go over that data even faster, but depending on what retention/conclusion is reached from that it may or may not be impressive.
Sure, but it would be a level zero on that list, right?
I'd say even Google would be #0.
> A two years old can definitely say stupid stuff, or have wildly incomplete/incorrect models of their reality, but can most certainly already think and reason, and update their internal models at any point.
I think that this presumes a certain definition of "think" and "reason". Monsters under the bed? To move from concrete examples to the abstract, from four apples to the idea of four?
Imagine a picture of a moon's orbit around the parent planet and the planet's orbit around a star, first at one time of year, then again 60° later, the circular orbits of each drawn clearly, with the two positions of the moon's orbits aligned at the top of the image; exaggerate the scale for clarity, and find it in an astronomy book — my peers at age 6 or 7 thought it was a picture of a mouse.
Imagine teachers and an ambulance crew explaining to the class how blood is donated, showing that they're putting a bag up the teachers sleeves and explaining how they'll demonstrate this by taking "blood" (fake? No idea at this point) from that bag. Everyone's looking, we see it go up the sleeve. We see the red stuff come out. Kid next to me screams "they're killing her!". Rather than say "we literally saw the bag go up the sleeve", 5-year-old-me tried to argue on the basis that killing a teacher in front of us was unlikely — not wrong, per say, but a strange argument and I wondered even at the time why I made it.
Are these examples of "reason"? Could be. But, while I would say that we get to the "children say funny things" *with far fewer examples than the best AI*, it doesn't seem different in kind to what AI does.
> LLMs are definitely a novel tool when it comes to finding information based on some high-ish level patterns (over exact string match, or fuzzy match), and they are very good at transforming between different representations of said data, with minimal (and hard limited) reasoning capabilities, but I have never seen evidence of going any further than that.
Aye. So, where I'm going with #2 and #3: even knowing what the question means well enough to respond by appropriately gluing together a few existing documents correctly, requires the AI to have created a vector space of meaning from the words — the sort of thing which word2vec did. But:
To be able to translate questions into answers when neither the question nor the answer are themselves literally in the training set, requires at least #2. (If it was #1, you might see it transition from "Elizabeth II was Queen of the UK" to "Felipe VI is King of Spain" via a mid-point of "Macron is Monarch of France").
For #3, I've tried the concrete example of getting ChatGPT (free model a few months back now) to take the concept of the difference between a racoon and a wolf and apply this difference again on top of a wolf, and… well, their combination of LLM and image generator gave me what looked like a greyhound, so I'm *not* convinced that OpenAI's models demonstrate this in normal use — but also, I've seen this kind of thing demonstrated with other models (including Anthropic, so it's not a limit of the Transformer architecture) and the models seem to do more interesting things.
Possibly sample bias, I am aware of the risk of being subject to a Clever Hans effect.
For #4, this seems hard to be sure it has happened when it seems to have happened. I don't mean what word2vec does, which I realise now could be described in similar language, as what word2vec does is kinda a precursor to anything at least #1. Rather, what I mean, in a human, would seem like "spots a black swan before it happens". I think the invention of non-Euclidian geometry might count, but even then I'm not sure.
Sure, but it would be a level zero on that list, right?
I'd say even Google would be #0.
> A two years old can definitely say stupid stuff, or have wildly incomplete/incorrect models of their reality, but can most certainly already think and reason, and update their internal models at any point.
I think that this presumes a certain definition of "think" and "reason". Monsters under the bed? To move from concrete examples to the abstract, from four apples to the idea of four?
Imagine a picture of a moon's orbit around the parent planet and the planet's orbit around a star, first at one time of year, then again 60° later, the circular orbits of each drawn clearly, with the two positions of the moon's orbits aligned at the top of the image; exaggerate the scale for clarity, and find it in an astronomy book — my peers at age 6 or 7 thought it was a picture of a mouse.
Imagine teachers and an ambulance crew explaining to the class how blood is donated, showing that they're putting a bag up the teachers sleeves and explaining how they'll demonstrate this by taking "blood" (fake? No idea at this point) from that bag. Everyone's looking, we see it go up the sleeve. We see the red stuff come out. Kid next to me screams "they're killing her!". Rather than say "we literally saw the bag go up the sleeve", 5-year-old-me tried to argue on the basis that killing a teacher in front of us was unlikely — not wrong, per say, but a strange argument and I wondered even at the time why I made it.
Are these examples of "reason"? Could be. But, while I would say that we get to the "children say funny things" *with far fewer examples than the best AI*, it doesn't seem different in kind to what AI does.
> LLMs are definitely a novel tool when it comes to finding information based on some high-ish level patterns (over exact string match, or fuzzy match), and they are very good at transforming between different representations of said data, with minimal (and hard limited) reasoning capabilities, but I have never seen evidence of going any further than that.
Aye. So, where I'm going with #2 and #3: even knowing what the question means well enough to respond by appropriately gluing together a few existing documents correctly, requires the AI to have created a vector space of meaning from the words — the sort of thing which word2vec did. But:
To be able to translate questions into answers when neither the question nor the answer are themselves literally in the training set, requires at least #2. (If it was #1, you might see it transition from "Elizabeth II was Queen of the UK" to "Felipe VI is King of Spain" via a mid-point of "Macron is Monarch of France").
For #3, I've tried the concrete example of getting ChatGPT (free model a few months back now) to take the concept of the difference between a racoon and a wolf and apply this difference again on top of a wolf, and… well, their combination of LLM and image generator gave me what looked like a greyhound, so I'm *not* convinced that OpenAI's models demonstrate this in normal use — but also, I've seen this kind of thing demonstrated with other models (including Anthropic, so it's not a limit of the Transformer architecture) and the models seem to do more interesting things.
Possibly sample bias, I am aware of the risk of being subject to a Clever Hans effect.
For #4, this seems hard to be sure it has happened when it seems to have happened. I don't mean what word2vec does, which I realise now could be described in similar language, as what word2vec does is kinda a precursor to anything at least #1. Rather, what I mean, in a human, would seem like "spots a black swan before it happens". I think the invention of non-Euclidian geometry might count, but even then I'm not sure.