I don't particularly like a centralized model either. Federation is much more difficult, but can create ecosystems which can survive longer/better than a single company can.

But on the specific issue of phone numbers as ID, they have been making some substantial progress on a technological solution to address this, specifically without running central servers.

https://signal.org/blog/secure-value-recovery/