Because not every engine slaps a fancy ECS tag front and center, the pattern evolved concurrently in industry (and we all had games to ship).
If you look really carefully you'll find structured column databases have similar properties since when you optimize for speed cache misses are your number one enemy.
Yes, I understand that ECS is probably good for more low-level stuff like particle-like systems. But what about gameplay-code? Unity ECS is mostly about that, to think in a data-oriented way about your gameplay programming. The thing that bothers me is why huge commerical engine like Unreal Engine or idTech don't even lift a finger about that?
About structured column databases, the same thing could be said about Lisps like Clojure which match the RMDB data model. But people still find it hard to grok it, hence the low adoption rate.
The "S" in unreal is a tick. It's been about a decade since I last worked with Unreal but there was very much the concept of systems that processed batches of things in areas where performance was a concern.
Keep in mind that ECS is a design pattern and just like any design pattern applied blindly you can end up in a worse place than you started. A FactoryDispatcherQueuePatternImpl helps no one despite using many design patterns. When it comes to ECS it's much more about the spirit than the letter of the law.
Not so. The physics system works on all physics components. The renderer system works on all rendering components. The networking system work on all components that are enabled for network replication.
You mean the big commercial game engines? They took a while time to adjust because, by definition, they're big. Harder to move the entire ecosystem at once.