Another facet is that, despite the waste, a lot of people are strongly motivated by competition to do better. Look at how Mir has spurred Wayland to get its act in gear - or how Gnome tried fixing Gnome 3 when Unity happened, or clang vs gcc, or how Go + Rust / Ruby + Python / Java + C# are competitive in the language space, or how Backbone competes with Angular.
All over the open source space, there are almost always two opposing forces in each camp, sometimes more, but whenever the competition dries up (VLC?) innovation slows down. VLC has become extremely refined as a media player, and I ask nothing more from it, but it hasn't tried any real radical UI departures or UX shifts and its mobile version is still slow on the uptake. I imagine if it had a media player competitor it would improve faster.
So the duplicated effort, although a "waste" of time, seems to drive developers to work harder, which produces better products in the end.
All over the open source space, there are almost always two opposing forces in each camp, sometimes more, but whenever the competition dries up (VLC?) innovation slows down. VLC has become extremely refined as a media player, and I ask nothing more from it, but it hasn't tried any real radical UI departures or UX shifts and its mobile version is still slow on the uptake. I imagine if it had a media player competitor it would improve faster.
So the duplicated effort, although a "waste" of time, seems to drive developers to work harder, which produces better products in the end.