> 10000 samples are nothing compared to 2^100 possible outputs. It is absolutely, definitely not a "brute search". Testing a small fraction of possibilities (e.g. 0.000001%) is called heuristics, and that's what people use too.
Brute searching literally means generating solutions until one works. Which is exactly what is being done here.
> Please learn a bit of combinatorics.
Don't be condescending - I understand the problem space just fine. Fine enough to realise that the problem was constructed specifically to ensure that "solutions" such as this just won't work.
Which is why this "solution" is straight-up broken (doesn't meet the target, exceeds the computationally bounds, etc).
> It's ridiculous to demand GPT-4 performs as good as a human.
Wasn't the whole point of this prize to spur interest in a new approach to learning? What does GPT-[1234] have to do with the contest rules? Especially since this solution broke those rules anyway?
> Obviously its vision is much worse and it doesn't have 'video' and physics priors people have, so it has to guess more times.
That's precisely my point - it has to guess. Humans aren't guessing for those types of problems (not for the few that I saw anyway).
I think to be clear, brute force generally means an iterative search of a solution space. I don't think that's what this system is doing, and it's not like it's following some search path and returning as early as possible.
It's similar that a lot of wrong answers are being thrown up, but I think this is more like a probabilistic system which is being pruned than a walk of the solution space. It's much smarter, but not as smart as we would like.
> I think to be clear, brute force generally means an iterative search of a solution space.
Sure, but not an exhaustive one - you stop when you get a solution[1]. Brute force does not require an exhaustive search in order to be called brute-force.
GP was using the argument that because it is not exhaustive, it cannot be brute-force. That's the wrong argument. Brute-force doesn't have to be exhaustive to be brute-force.
A brute force search can be expected to find a solution after a more thorough search of the space of possibilities. If it really is only searching 0.000001% of that space before finding solutions, then some structure of the problem is guiding the search and it's no longer brute force.
> Therefore, brute-force search is typically used when the problem size is limited, or when there are problem-specific heuristics that can be used to reduce the set of candidate solutions to a manageable size.
> One way to speed up a brute-force algorithm is to reduce the search space, that is, the set of candidate solutions, by using heuristics specific to the problem class.
I mean, the approach under discussion is literally exactly this.
Now, Mr "ACM ICPC, studied algorithms for years", where's your reference that reducing the solution space using heuristics results in a non-brute-force algorithm?
You're asking for a definition of exhaustive search. Exhaustive search, by definition, goes through the entire search space. That's what word exhaustive means.
For a reference, check Cormen's "Introduction to Algorithms". Every mention of brute-force search is specifically to exhaustive search is which not feasible for bigger spaces.
> I mean, the approach under discussion is literally exactly this.
It's literally not. It DOES NOT REDUCE the candidate set. It generates most likely candidates, but it doesn't reduce anything.
You lack basic understanding. Solutions are pixels grids, not Python programs. There's no search over pixel grids in the article. Not every search is exhaustive search.
This is like saying theoretical physicists are "brute-forcing" physics by generating candidate theories and testing them. Ridiculous.
This was extremely cringe worthy to read. You are confidently wrong about this. You are trying to redefine well established terminology. I don't care about whatever random wiki page you might find to "support your claims". Anybody who has worked a lot on algorithms (including myself) knows what brute force means and this is not it.
Also: lol at your "who hasn't" comment. Because you clearly haven't.
Sure, here's definition for "brute force" from university textbook material written by pllk, who has taught algorithms for 20 years and holds a 2400 rating on Codeforces:
"Complete search is a general method that can be used to solve almost any algorithm problem. The idea is to generate all possible solutions to the problem using brute force ..."
And a bit further down chapter 5:
"We can often optimize backtracking by pruning the search tree. The idea is to add ”intelligence” to the algorithm so that it will notice as soon as possible if a partial solution cannot be extended to a complete solution. Such optimizations can have a tremendous effect on the efficiency of the search."
Your mistake is that you for some reason believe that any search over solution space is a brute force solution. But there are many ways to search over a solution space. A "dumb search" over solution space is generally considered to be brute force, whereas a "smart search" is generally not considered to be brute force.
Ok now I think I understand what causes your confusion. When an author writes "One way to speed up a brute-force algorithm ..." you think that the algorithm can still be called "brute force" after whatever optimizations were applied. No. That's not what that text means. This is like saying "One way to make a gray car more colorful is by painting it red". Is it still a gray car after it has been painted red? No it is not.
Brute searching literally means generating solutions until one works. Which is exactly what is being done here.
> Please learn a bit of combinatorics.
Don't be condescending - I understand the problem space just fine. Fine enough to realise that the problem was constructed specifically to ensure that "solutions" such as this just won't work.
Which is why this "solution" is straight-up broken (doesn't meet the target, exceeds the computationally bounds, etc).
> It's ridiculous to demand GPT-4 performs as good as a human.
Wasn't the whole point of this prize to spur interest in a new approach to learning? What does GPT-[1234] have to do with the contest rules? Especially since this solution broke those rules anyway?
> Obviously its vision is much worse and it doesn't have 'video' and physics priors people have, so it has to guess more times.
That's precisely my point - it has to guess. Humans aren't guessing for those types of problems (not for the few that I saw anyway).