That makes sense, LLM training data probably has tons of common problems in it, but maybe only a few or no instances of really niche, difficult ones. So it just comes up with a bunch of garbage.

My experience has been similar, it is amazing for stuff I am a beginner at but kinda useless for my actual work. It was invaluable today when I was trying to grasp CA zoning laws, but its almost useless for coding.

This also points to why it will never (imo) be "intelligent". It will never be able to take all its knowledge and use that to solve a problem it doesn't have training data for.