I don't think one should underestimate the incentives at play here though. Complexity sells not just in literal money, but in career prospects too and so on. It's really bad incentives all around in favor of complexity.
Complexity sells in terms of dopamine. "Look at this incredibly complicated thing I made, that I understand and you don't! Aren't I brilliant!" "You must be - I can't understand it at all."
People get emotional rewards from themselves from making something work that is at the limit of the complexity that they can handle. They often also get emotional rewards from others for making things that others can't understand. (They shouldn't, but they often do.)
They should, because succeeding at something near the limit of your ability is how you expand your ability. But that isn't the best for production systems.
But that's the problem right, in any other industry self-serving pursuits without proper motivation is rightly seen as wholly unprofessional. But somehow our field has come to not only accept it, but encourage it.
Imagine a carpenter going: - "Yeah, so I wanted to test the limits of my abilities so I made the shed nuclear bomb proof, and yeah, that'll be $100k, now I gotta go to the reinforced concrete conference, and do you mind if I use this shed as a reference when applying to the nuclear shelter company that I actually want to work for?"
This carpenter would of course never be hired again. But our industry is overflowing with nuclear bomb proof sheds without much push back. It's incredibly unprofessional.
It is a sign of a newbie (nothing terribly wrong but most people grow out of it).
It may require thinking hard to get code that is simple (like with DRY, KISS principle should be followed in moderation. It is all about tradeoffs as usual).
