The macro economy is dominated by gigantic conglomerates that try to own and integrate a significant amount of their entire supply chains - think General Electric, Mitsubishi, Samsung. Small to Medium Enterprises have a few niches where they work (restaurants and other service industries being the largest by far), but they play a relatively small role and are not normally part of any supply chain, they only really work at the very end of the chain.
Similarly, building a whole product out of pure microservices with teams that barely coordinate with each other beyond maintaining their APIs doesn't really work. There is very much a place for such microservices for certain well understood problems (authentication being probably the clearest), but for most domain specific work you need far more coordination than that to truly evolve a complex product.
For example, you can't build a computer game in this model. A modern 3D engine requires far more than 4-8 people, and there is no clean separation between pieces to afford this. And even if there were, there is no One True Way to build one: the teams designing the 3D engine need to communicate with the teams building the game engine and the game designers themselves and the artists to achieve a cohesive whole. A 100-person project just can't (always) be modeled as 15 4-8-person projects.
> the teams designing the 3D engine need to communicate with the teams building the game engine and the game designers themselves and the artists to achieve a cohesive whole.
Sure, just as they need to communicate to the farmer what food needs to be grown in order to ensure that the right food shows up on their dinner table. These are solved problems, though. We already know how to do incredibly complex things at scale.
No, it is nowhere near like that. Food is a particularly bad example, as the food industry is extremely wasteful overall, with a huge percentage of all food produced being thrown away at all levels of the production chain all the way to the final consumer.
In addition, most companies that package food in various ways, including major restaurants, either own or contract farming business for specific quantities of specific crops ahead of time, spending large amounts of time to negotiate contracts to this effect, and sometimes working even closer when a particular result is required.
If you find yourself craving Mongolian food one night, you may often simply find that none is available, at any price you are willing to pay, unless you worked with someone to make some months in advance. And conversely if you open a Mongolian restaurant, you may just not find any customers for it and waste significant money on it. (substitute Mongolian with any other food that is not commonly eaten or even thought about in your area)
The point is that you do need explicit integration and cooperation. And you often have way more than 4-8 people working on one of these contracts to even define the terms.
And again, most real companies own most of their supply chain, they don't depend on a hundred different suppliers at every level like you seem to be suggesting. And they do this because it's FAR more efficient to have a large monolithic team build something together than to rely on dozens of small teams competing in a market to build the same thing.
> The point is that you do need explicit integration and cooperation.
Obviously. Like we said, no 4-8 person team is building anything by themselves. For one, most obviously, they'd starve to death if they even tried. Integration and cooperation are literally what defines an economy.
> most real companies own most of their supply chain
As opposed to fake companies, that have no supply chain, with them being fake and all?
> because it's FAR more efficient to have a large monolithic team build something together
How can you know? The problems needed to be solved to see that through were stated to be unsolved. Was there some breakthrough in the past 10 hours that you haven't mentioned and the rest of us missed?
> As opposed to fake companies, that have no supply chain, with them being fake and all?
I should have said "companies that can afford to".
> How can you know? The problems needed to be solved to see that through were stated to be unsolved. Was there some breakthrough in the past 10 hours that you haven't mentioned and the rest of us missed?
Because I can look at the price of goods versus their cost. A company that owns the production chain for a sub-component is paying the cost of that component, while a company that doesn't pays the market price.
Also, markets only work if you have competing suppliers for the same product. This in itself is extremely wasteful for software, so trying to maintain an internal market of software where multiple teams build the same service and compete with each other on who has a better version is crazy when there is any alternative.
Again we can see this from the software market: most companies not only don't build an internal market in house to have, say, the best OS, but they actually found that even supporting an external market is wasteful. Instead of buying Solaris or Windows or OS/2, they started collaborating on Linux (and clang, and MySQL, and Kubernetes and others).
But these are not built as a marketplace of microservices. They are built as monoliths with large teams working to add features and collaborating relying on a hierarchy of maintainers to merge them together in a cohesive whole, or on a more elaborate committee system.
So again, I don't see any proof whatsoever that building a company or a product in general out of a marketplace of dozens or hundreds of very small teams has ever worked beyond maybe some niche cases.
> Because I can look at the price of goods versus their cost. A company that owns the production chain for a sub-component is paying the cost of that component, while a company that doesn't pays the market price.
I'm not sure I follow. Let's say Acme Inc. produces a widget that is a subcomponent of their larger product. When that subcomponent is within the control of a team of four who works for Acme Inc., the company has to pay the market price (meaning a markup margin?), but when that team is joined with another team who also works for Acme Inc., then they only have to pay the cost?
1. What difference does that make? Even if a markup really was paid on paper, it's just to Acme Inc. itself. You haven't changed anything, practically speaking.
2. If an external entity is willing to pay more for the component, which is what I think you are trying to suggest with market price, Acme Inc. is paying the opportunity cost when keeping it internally and thus is still paying the market price. Are you under the impression that there is a free lunch here? There is not.
> trying to maintain an internal market of software where multiple teams build the same service and compete with each other on who has a better version is crazy
Just as it would be crazy in the macroeconomy. The idea that competition is necessary for markets to work is misguided. You don't need a second road running parallel to the first. One road is just fine. Competition only comes into being when someone thinks they can serve the customer better by doing things differently. Which is a useful property of human dynamics, to be sure, but sometimes things are already as good as anyone is able to imagine.
> they started collaborating on Linux
As individual teams, with another team headed by Linus who accepts the services of those other teams as seen fit. Another great example of the service economy in action. No need for a monolith – if such a thing were even possible, but as pointed out at the beginning of this, of which I agree, we don't actually know how to build monoliths, no matter how great they sound in theory. There are outstanding problems not yet solved.
> They are built as monoliths with large teams working to add features and collaborating relying on a hierarchy of maintainers
A monolith, but also hierarchal? Uhh...
> I don't see any proof whatsoever that building a company or a product in general out of a marketplace of dozens or hundreds of very small teams has ever worked beyond maybe some niche cases.
I suppose you can take the perspective that nothing works, but given the outstanding problems not solved to allow anything else, something that doesn't work beats nothing at all. Worse is better, perhaps.
I think this whole thread may be at least partly a misunderstanding.
The way I see it, the discussion began by saying that any team (but in context, referring to a whole company) that grows past a certain size needs a traditional hierarchical system of layers of management to function. Then, someone said that a team/company should just not grow past 4-8 people, and thus you avoid the need of hierarchical management.
My point is that this idea makes no sense. A company can't function as a composite of hundreds or thousands of sled organizing small teams without any management to coordinate them.
My understanding is that you claimed that such a thing is possible, and that in fact it is how the free market and thus the economy of most nations actually functions, without any hierarchy.
I was then rebutting this by showing that the economy does form hierarchies (with large buyers essentially viewing small sellers as subordinates, not independent market actors, when they're not outright buying them).
So, I would argue, the economy is not generally organized as a bunch of independent small orgs, but as a series of huge conglomerates that either own their suppliers, or at least dictate market terms for some period with contracts. The idea of a free market where buyers and sellers meet and negotiate prices based on supply and demand is largely a myth, with some limited exceptions.
As such, the hierarchy in how Linux is built is in line with my original idea: not supply and demand based, but instead organized in a relatively rigid hierarchy.
In support of why I'm claiming that this system of hierarchies arises due to efficiency, I gave the example of how much it costs a company to buy a piece they need versus how much it costs them to produce it internally. This is meant to show that a supply&demand based self-organizing market (the flat, many small teams without managers model) is less efficient than an integrated supply chain (the top-down hierarchical model, with layers of management).
> and that in fact it is how the free market and thus the economy of most nations actually functions, without any hierarchy.
There is a hierarchy of sorts (realistically, something more like a cyclic graph, but close enough for all intents and purposes). The customer gets food from the grocery store -> who gets food from a wholesaler -> who gets food from a processor -> who gets food from the farmer. But at no point, despite being an exceptionally complex system, do you need to exceed a small number of people per team.
> thus you avoid the need of hierarchical management.
Right, so I expect this is where the miscommunication is happening. You're not wrong that the consumer of food manages the grocery store. Absolutely that is true. Of course it is. And in that sense, some kind of hierarchical (or whatever we want to see the graph as) management will emerge. That is a necessary precondition for an economy to exist.
Contextually, though, management was seen in a more vileness way, which does tend to happen when communication overhead starts to overburden the people, leaving actors to be faceless "cogs". This is what small teams avoid.
> but as a series of huge conglomerates
Which don't operate as monoliths. Internally they function as a series of smaller businesses, and inside that even smaller business, until you whittle the teams down into a small number of people. Maybe not exactly 4-8, but certainly within that order of magnitude. A person can only communicate with so many other people before all hours of the day are exhausted.
> The idea of a free market where buyers and sellers meet and negotiate prices based on supply and demand is largely a myth
Okay... Sure? Is there some reason for us to randomly start talking about a "free market" here? It was never mentioned before.
> not supply and demand based
I don't follow. Supply and demand is not some kind of foundation on which something is based. It is observance of social behaviour, typically that of humans, but applies even to animals. Supply and demand is every bit as much present in the development of Linux as everything else humans do with other humans. It is fundamental.
> but instead organized in a relatively rigid hierarchy.
Instead? They are not at odds with each other.
> I gave the example of how much it costs a company to buy a piece they need versus how much it costs them to produce it internally.
Which didn't seem to serve a purpose. If all else is equal, it costs the same. Presumably you are referring to a case where you can buy a subcomponent externally for $10 or produce it internally for $5? But given that buyers are willing to pay $10, you lose $5 every time you keep it for internal use. So, in both cases, it costs you $10. There is no free lunch.
> This is meant to show that a supply&demand based self-organizing market (the flat, many small teams without managers model) is less efficient than an integrated supply chain
The data doesn't support this, though. Statistically, smaller businesses in coordination with each other are more efficient. You'll notice large businesses only become large when they can build some kind of moat. In the case of software, that mostly means hiding behind copyright and patent protections.
> You're not wrong that the consumer of food manages the grocery store.
I think this is the key to our miscommunication. I don't call that management. That is what I am referring to as the free market model of self-organization: one customer doesn't dictate what the convenience store does, they only give very indirect signals that the convenience store can choose to respond to or not (say, this customer didn't buy any cheese; they can drop the price of cheese, or drop cheese from their inventory, or keep offering cheese and hope that others will buy it).
In contrast, a rigid hierarchy is what happens inside most corporations: your manager tells you to make cheese, you either make cheese or you get fired. Your manager is telling you to make cheese because a different team is planning to make cheese toast, and the manager wants to ensure they have cheese available. I would refer to this as a planned economy model, and call this business a monolith in the sense that, while it has internal parts, they are "fixed in place", they don't get to decide what to work on.
Similarly, Apple's relationship to its suppliers is closer to the planned economy model than to the free market: Apple doesn't go shopping around various suppliers and getting some milk here and some cheese there. They investigate some market and pick some suppliers, and form contracts with them that demand specific amounts at specific times at specific prices. After signing the contract, the supplier is not free to take Apple's feedback or balance it with their own desires or other customers: they have to execute on the contract or risk penalties and fines. They are now effectively under Apple's management, for at at least some portion of their business, and not operating as an independent market agent that uses pricing signals to elaborate their strategy.
Of course I do agree that at no step of the way is there a 1000 person team working on something. But neither is anyone else: the discussion we are having, as I see it, is whether you can organize 1000 people into many small teams that only coordinate voluntarily, where team A decides what to work on themselves and team B decides whether to use team A's work or not; or you need to impose a more rigid management hierarchy that tells team A to work on this and team B to work use team A's output.
The second model is FAR more pervasive in how companies organize internally. And even in the economy at large, contrary to what you're saying, large organized groups like Samsung easily outcompete a network of small businesses. There are almost 0 industrial consumer products supplied by a myriad of small manufacturers, in fact. From phones to cars to appliances, everything is produced by huge conglomerates. And they don't have any special moats, only money and the inherent efficiencies of being able to run a planned economy internally.
Similarly, building a whole product out of pure microservices with teams that barely coordinate with each other beyond maintaining their APIs doesn't really work. There is very much a place for such microservices for certain well understood problems (authentication being probably the clearest), but for most domain specific work you need far more coordination than that to truly evolve a complex product.
For example, you can't build a computer game in this model. A modern 3D engine requires far more than 4-8 people, and there is no clean separation between pieces to afford this. And even if there were, there is no One True Way to build one: the teams designing the 3D engine need to communicate with the teams building the game engine and the game designers themselves and the artists to achieve a cohesive whole. A 100-person project just can't (always) be modeled as 15 4-8-person projects.