The problem with your example of applying analyzing something as complex and esoteric as a codebase is that LLMs cannot reason they simply return a response based on statistical inference, so unless you followed a standard like PSR for PHP and implemented it to a 't' it simply doesn't have to context to do what you're asking it to do. If you want an LLM to be an effective programmer for a specific application you'd probably need to fine tune and provide it instructions on your coding standards.
Basically, how I've become successful using LLMs is that I solve the problem at a 9,000ft view, instruct the LLM to play different personas, have the personas validate my solution, and then instruct the LLM step-by-step to do all of the monkey work. Which doesn't necessarily always save me time upfront but in the long run it does because it makes fewer mistakes implementing my thought experiment.
Fair enough, I might be asking too much indeed, and may not be able to come up with an idea how LLMs can help me. For me, writing code is easy as soon as I understand the problem, and I sometimes spend a lot of time trying to figure out a solution that fits well within the context, so I thought I could ask an LLM what different things do and mean to help me understanding the problem surface better. Again, I may not understand something, but, at this point, I don’t understand what’s the value of code generation after I know how to solve a problem.
Do you happen to have a blog post or something showing a concrete problem that LLM helped you to solve?
Basically, how I've become successful using LLMs is that I solve the problem at a 9,000ft view, instruct the LLM to play different personas, have the personas validate my solution, and then instruct the LLM step-by-step to do all of the monkey work. Which doesn't necessarily always save me time upfront but in the long run it does because it makes fewer mistakes implementing my thought experiment.