I wouldn't call the mapping necessarily arbitrary. The causal/information processes that your nerves connect to may have some mathematical structure that distinguishes them.

There's the symmetry theory of valence which proposes that symmetric (in some strict mathematical sense) processes feel good, and vice versa with bad qualia.

But this is only one of many subproblems: https://opentheory.net/wp-content/uploads/2016/11/Eight-Prob...