I don't think people really appreciate how simple ARC-AGI-1 was, and what solving it really means.
It was designed as the simplest, most basic assessment of fluid intelligence possible. Failure to pass signifies a near-total inability to adapt or problem-solve in unfamiliar situations.
Passing it means your system exhibits non-zero fluid intelligence -- you're finally looking at something that isn't pure memorized skill. But it says rather little about how intelligent your system is, or how close to human intelligence it is.
> designed as the simplest, most basic assessment of fluid intelligence possible.
This was the goal, but that doesn't say what the test itself is. Try to get a human to solve this problem without their visual cortex, they couldn't do it. Stating your goal for a thing, doesn't make the thing that goal.
AI researchers designing intelligence tests are like programmers designing their own cryptography.
How about we have people skilled in neuropsychology, psychometrics and cognitive psychology do what they are good at.
> How about we have people skilled in neuropsychology, psychometrics and cognitive psychology do what they are good at.
Disagree. The thing that we will eventually call AGI will not be human. No need to have human-specific evaluations unless you’re aiming for an artificial human and not just an artificial intelligence.
> But why ignore a huge body of research in how to write scientific tests of intelligence and cognition?
Not saying to ignore it, but we are not dealing with humans. Those tests may give misleading results as you're proposing to use them outside of their design envelope. This is an area of research in itself.
I don't think people really appreciate how simple ARC-AGI-1 was, and what solving it really means.
It was designed as the simplest, most basic assessment of fluid intelligence possible. Failure to pass signifies a near-total inability to adapt or problem-solve in unfamiliar situations.
Passing it means your system exhibits non-zero fluid intelligence -- you're finally looking at something that isn't pure memorized skill. But it says rather little about how intelligent your system is, or how close to human intelligence it is.