Which is why you get network topologies other than 'just' fat tree in HPC networks:
* https://www.hpcwire.com/2019/07/15/super-connecting-the-supe...
* https://en.wikipedia.org/wiki/Torus_interconnect
https://en.wikipedia.org/wiki/High-performance_computing