I agree with your general point: it is a mistake to say "these two things are mysterious, therefore they are the same".
That said:
> LLMs have no experience, they exist crystalised and unchanging in an abstract eternal now, so they literally can't understand anything.
Being crystalised and unchanging, doesn't tell us either way if they do or don't "understand" anything — if it did, then I could only be said to "understand" whatever I am at some moment actually experiencing, so it would not be allowed to say, for example, that I can understand "water in my mouth" because my memory of previous times I had water in my mouth seem to be like that.
They're definitely not "like us", but that's about all I can say with confidence, and it's a very vague statement.
That said:
> LLMs have no experience, they exist crystalised and unchanging in an abstract eternal now, so they literally can't understand anything.
Being crystalised and unchanging, doesn't tell us either way if they do or don't "understand" anything — if it did, then I could only be said to "understand" whatever I am at some moment actually experiencing, so it would not be allowed to say, for example, that I can understand "water in my mouth" because my memory of previous times I had water in my mouth seem to be like that.
They're definitely not "like us", but that's about all I can say with confidence, and it's a very vague statement.