Or just don't use floats at all. In many cases fixed-point makes more sense.
The only problem with that is that there's no support in most programming languages for either and no hardware-support for decimal floating points in common hardware.
Floating point numbers are a pretty generic concept. But there are infinite possibilities of fixed point ones, with completely different qualities. It's just not viable to implement them in hardware.
I especially said hardware for decimal floats and didn't include fixed point. You don't need special hardware for fixed point. Addition and subtraction are the same as if you would do it with integers, for multiplication and division you also need to shift additionally.
With floating point you also have endless possibilities how to define them. Next to the standard IEEE ones you for instance have 80 bit ones you had to use on intel CPUs. With fixed point arithmetic you don't really have standards but common ones like Q16.16
The only problem with that is that there's no support in most programming languages for either and no hardware-support for decimal floating points in common hardware.