I once tried 3 hours to get ChatGPT to write a correct cubic interpolation function. Everything it wrote was either not cubic or not an interpolation function (the resulting curve didn't pass through the input points).
Not sure what your point is. I probably couldn't write a correct cubic interpolation function in 3 hours if you stuck me in an empty room and kept giving me the same instruction with a small change every time. But I've definitely written software as complex as this project basically from memory with only the occasional glance at some framework docs.
Leetcode is a meaningless metric when evaluating application developers.
This was an example of how I gave a LLM clear instructions on a function that is known to the internet. If I told you get to use google you probably could solve this "leetcode" task in 10 minutes.
So I picked a random known algorithm and asked ChatGPT to write a function with it, just to test it.
I agree that leetcode is a useless metric, but nobody informed me that something simple like cubic interpolation belongs to this.category now. Maybe I should re-evaluate my career.
I'm just saying that writing a small function that has to be perfectly correct is a very different task from from writing a small web app where the space of possible solutions is incredibly large.
LLMs don't have perfect recall, so it has to reconstructtthe answer from the information it retained from training. Your example is like asking it math problems: unless it had both seen and retained the specific one you've asked it, it can't answer correctly because doesn't have the capability to reason about the problem and solve it. But something like writing a web app is closer to writing a story, where each next block it spits out doesn't have to be exactly correct, it just has to get the job done.
As for leetcode, I guess it depends on your field, but my logic is this: if I came to an interview and was told to write a microservice that receives a POSTed file, extracts some data and puts it in a DB, that's a solid work-relevant skill test. But if they asked to to write some interpolation functions, I'd consider that a leetcode interview - closer to math than programming, not a good indicator of job ability, not representative of the kind of work I'll be doing.
So yeah.