Ive noticed the same and wonder if this is the natural result of public codebases on average being simpler since small projects will always outnumber bigger ones (at least if you ignore forks with zero new commits)

If high quality closed off codebases were used in training, would we see an improvement in LLM quality for more complex use cases?