If you can convince your contacts to use it, and they're on platforms with good clients, XMPP with OMEMO ticks all the boxes, and has since 4-5 years ago.
It's properly federated, with a diversity of client and server implementations (something that will likely never happen with Matrix due to the excessive complexity and poor design of the protocol), servers are easy to deploy and don't need a lot of system resources, and it has strong security.
The main issue is that there are only a few really good clients, since network effects convinced everyone to use inferior (but better marketed) messaging systems. Conversations (on Android) is excellent, though.
You mention the "excessive complexity" of the matrix protocol, so I am a bit surprised that you advertise XMPP as an alternative. I never have looked at either protocols, but I remember that in the days when jabber looked like the next thing, it was strongly criticised for being overly complex and in particular using xml being unsuitable for a chat protocol. So I'm wondering if something changed and if you can I've examples of what is complex in matrix but easy in XMPP?
The use of XML is a bit unfortunate, but these days there are a huge range of mature XML libraries for any language, with a variety of different styles of API to suit the programmer / program design, so the use of XML doesn't really affect implementation that much. Not to mention there are decent XMPP libraries for many languages, so only the application-specific logic needs to be implemented.
XMPP has a clear design based around general concepts, that make it useful for a wide variety of messaging-type applications. The criticism I hear often seems to come from people approaching it as an IM protocol, and then being upset that they need to learn some general XMPP concepts to understand how IM is implemented on top of XMPP. The reality is that it's a general protocol, which can be used to implement a variety of different real-time messaging applications. That generality can cause people to think it's overcomplex if they approach it as if it's designed purely for IM.
You'll often find XMPP as the underlying tech for IoT real-time communication, or for signaling/presence (as an alternative to SIP) for videoconferencing (for example, Jitsi Meet uses XMPP for signaling). In this way, XMPP is quite widely used, but often not in a way that people are aware of.
By contrast, Matrix appears to attempt a clean-sheet design without learning any of the lessons of the many protocols that came before it. The protocol is heavy on special cases, has many serious inefficiencies (many coming from an apparent desire to make the protocol "more Web"), and especially on the server side requires a large amount of implementation work simply to get to a basic level of functionality (as opposed to XMPP, where a simple server for a basic set of functionality can be written in a very small amount of code, especially if you make use of an existing XML library).
I've written code for both protocols (as well as many other protocols with similar goals) and will do my best to avoid touching Matrix ever again.
It's properly federated, with a diversity of client and server implementations (something that will likely never happen with Matrix due to the excessive complexity and poor design of the protocol), servers are easy to deploy and don't need a lot of system resources, and it has strong security.
The main issue is that there are only a few really good clients, since network effects convinced everyone to use inferior (but better marketed) messaging systems. Conversations (on Android) is excellent, though.