>> My perception is ARC tried it the other way for years but a lot of academics and AI pundits ignored or dismissed it without ever meaningfully engaging with it.
Your perception is very wrong and the likely reason is that as you say you're not an academic researcher. ARC made a huge splash with the original Kaggle competition a few years ago and it drew in exactly the kind of "academic researcher" you seem to be pointing to: those in university research groups who do not have access to the data and compute that the big tech companies have, and who can consequently not compete in the usual big data benchmarks that are dominated by Google, OpenAI, Meta, and friends. ARC, with its (unfair) few-shot tasks and constantly changing private test set, is exactly the kind of dataset that that kind of researcher are looking for, something that is relatively safe from big tech deep neural nets. Even the $1 million prize seems specially designed to be just enough to draw in that crowd of not super-rich academics while leaving corporate research groups insufficiently motivated.
Besides which, I won't name names but one of the principal researchers in the winning system is just one of those academics. I don't know which is the period you mean ARC was ignored by the academic community but that particular researcher was in a certain meeting of like-minded academics two years ago where one of the main areas of discussion was in short "how to beat ARC and show that our stuff works".
Your perception is very wrong and the likely reason is that as you say you're not an academic researcher. ARC made a huge splash with the original Kaggle competition a few years ago and it drew in exactly the kind of "academic researcher" you seem to be pointing to: those in university research groups who do not have access to the data and compute that the big tech companies have, and who can consequently not compete in the usual big data benchmarks that are dominated by Google, OpenAI, Meta, and friends. ARC, with its (unfair) few-shot tasks and constantly changing private test set, is exactly the kind of dataset that that kind of researcher are looking for, something that is relatively safe from big tech deep neural nets. Even the $1 million prize seems specially designed to be just enough to draw in that crowd of not super-rich academics while leaving corporate research groups insufficiently motivated.
Besides which, I won't name names but one of the principal researchers in the winning system is just one of those academics. I don't know which is the period you mean ARC was ignored by the academic community but that particular researcher was in a certain meeting of like-minded academics two years ago where one of the main areas of discussion was in short "how to beat ARC and show that our stuff works".