It really isn't that hard to get a single binary that works across glibc-based distros. Just compile against the oldest version of glibc/Xlib/etc. you want to support and bundle all non-system libraries, statically linking them with symbols hidden if possible.

We are not talking out of our ass here - I myself do this for all the software I maintain and just works. Tons of programs are released this way.