I think there is another aspect to human thinking other than system 1/system 2 though, which is the abstract world model humans have.
system 1 / 2 is more like the process, while the world model is the actual data being 'processed'.
And I think basically, humans have a much simplified, 'low dimensional' world model that consists of a set of objects (let's call them patterns), and then a "list" of essential properties that those objects have, that leads to a constraint on how each object can behave in the world model as a whole.
And this is sort of hierarchical or at least, we can zoom in and out in detail depending on the level of knowledge we have about a particular pattern.
So problem 1 is: It's not clear to me that text or any sort of data would contain all the necessary constraints so that any particular prompt would result in a world model that exactly takes into account the constrains of each object and 2) Even if was, I'm not sure the process of step by step thinking (system1/2) about each object and computing world states could occur in current architectures. This is especially important for computing a set of objects, then abstracting the result, then doing another round of computing with that result, or something like this.
I'm not hard set on this but this is my current thinking.
And I think basically, humans have a much simplified, 'low dimensional' world model that consists of a set of objects (let's call them patterns), and then a "list" of essential properties that those objects have, that leads to a constraint on how each object can behave in the world model as a whole.
And this is sort of hierarchical or at least, we can zoom in and out in detail depending on the level of knowledge we have about a particular pattern.
So problem 1 is: It's not clear to me that text or any sort of data would contain all the necessary constraints so that any particular prompt would result in a world model that exactly takes into account the constrains of each object and 2) Even if was, I'm not sure the process of step by step thinking (system1/2) about each object and computing world states could occur in current architectures. This is especially important for computing a set of objects, then abstracting the result, then doing another round of computing with that result, or something like this.
I'm not hard set on this but this is my current thinking.