You can't have competing companies on rails. Each companies having their own networks would be prohibitively expensive, or they have to share rails between each other which poses significant issues.
You're also left with the issue of unprofitable stations, which is a lot of them.
If you want an example of what a really privatized, deregulated rail system would look like, read about France in the early 20th century. By the 30s, shareholders were begging the state to nationalize their money-burning companies. And it did.
Rail should be a public service. As one, its benefits to society are immense.
You can, Japan does it and they are considered best in the world. There are at least 10 different train companies in Tokyo, (probably more like 20 but I don't know all the smaller ones). Similarly in Osaka, Kyoto, etc....
> Rail should be a public service.
No, it should be run for profit but the system put in place well designed so serving more customers better means the company does better.
Japan accomplishes this AFAIK, by letting the train companies buy/own/build other things. For example, the Tokyu company that runs some of the trains that come into Shibuya station owns the building Google's Japan office is in. They also own the famous 109 building (10 = to, 9 = kyu). And 10 or 15 other buildings in Shibuya. They run grocery stores at probably 50% of their stations. They also build apartments.
If their line gets sucky people spread the word "don't live on that line, it sucks, it's too crowded and the trains are late" and all their business suffer and people start moving out. So they have an incentive to keep their lines better than the other lines.
They also compete on destinations. For example Shibuya to Yokohama. You can take a Tokyu line and you can also take a JR line. They go slightly different routes but end up at the same place. Similarly if you're in Kyoto and want to go to Osaka. You can take JR, or Hankyu, or Hanshin, 3 separate train companies all of which run multiple lines. You'll see ads for the multiple different competing trains to get you from downtown Tokyo to Narita and Hanada airports.
All of that is only possible because of network effects, and a nationalized JR built most of that network.
And honestly, I think you’re overstating the degree of choice you have as a rider. Different lines carved out their corners and companies avoid each other. Service level is generally high, to the point that it’s a non factor. You take whichever train is cheaper/faster, and that’s it.
The problem is that rail has huge positive externalities that are not captured by the operators. You can address that by letting operators speculate on real estate, or you could do it with taxes.
When JR was nationalised, the Japanese central gov't very carefully controlled where private lines could be built. This partly explains why no private lines ran inside of Yamanote line in Tokyo.
Also, the OP specifically mentioned Tokyo, Osaka, and Kyoto. These three cities are special because they have exceptionally high ratio of private train lines. (Yokohama should also be included.) Each of these train lines needed to apply with the Japanese central gov't to have their routes approved. If the competition to the JR national line was deemed too high, the proposal was denied.
Another thing that is frequently overlooked by outsiders who try to explain "rail success" in Japan: For more than 50 years, there was/is a tax deduction where employers buy a monthly rail/bus pass for employees. This was/is not restricted to JR national lines only -- private lines also qualify. As a result, this programme generates huge demand for rail service in Japan. I assume that Switzerland must have something very similar, as the outcome is incredibly similar (rail crazy by any measurement).
Open access operators have generally not been allowed to directly compete with franchised Train Operating Companies (TOCs).
Many of Britain's railway routes are profitable, but open access operators aren't allowed to compete with franchised TOCs for a slice of those profits. As a prerequisite for approval, the Department of Transport requires open access operators to show that their services would create additional value of their own, not just abstract existing revenue.
Lumo, an open access operator and a part of First group, run trains between London and Edinburgh. They stop at Morpeth for no other reason than to fulfil their requirement to create value, in this case by stopping at a station previously underserved by the DfT franchise of the route. In other words, Lump had to make compromises in return for being allowed to get in on the action on one of the profitable railway lines.
I'd be confident in claiming that there has never been any true free market competition for British passenger rail services since privatization. The existence of private capital doesn't automatically create free markets!
Even if they did have their own networks, you'd just get what NTL (now Virgin Media) did in the 90s: build a few tiny networks in the most profitable places. There probably wouldn't even be rail in the Highlands or Wales.
You're also left with the issue of unprofitable stations, which is a lot of them.
If you want an example of what a really privatized, deregulated rail system would look like, read about France in the early 20th century. By the 30s, shareholders were begging the state to nationalize their money-burning companies. And it did.
Rail should be a public service. As one, its benefits to society are immense.