Math is much more than arithmetic as it is taught. Understanding arithmetic through the lens of relationships (as you stated) is no different than algebra, geometry, trigonometry, calculus, differential equations...
Arithmetic is important, especially in applications. But I wouldn't say it forms the backbone of all mathematics. You can do symbolic calculations just fine without any actual numbers. And a number of fields don't even involve any numbers at all, not even in applications.
Math is much more than arithmetic as it is taught. Understanding arithmetic through the lens of relationships (as you stated) is no different than algebra, geometry, trigonometry, calculus, differential equations...
Indeed, it forms the backbone of them all.