> Apply these assumptions to finding an answer to the
> question of what you should make for dinner, and you'll
> quickly see there are problems.
And, to amplify your point a bit, there are problems with A/B testing for which of my children I should love the most, or what I find beautiful or interesting; or should I A/B test out whether enslaving others works out well for me?
I chose extreme examples, but the point is that there are many, many things in human experience that don't lend themselves to easy, simplistic rules. It's genuinly hard to work through a lot of a lot of the issues that people face in real life.
With that said, it's it's also important to work through and to understand objective data as best as possible. For clearly defined and nicely-behaving problems objective data is certainly the way to go. The problem is that a lot of the problems people actually face aren't so easy to understand, and aren't so well-behaved.
Here are some dubious assumptions:
* There is an optimal answer.
* Human preferences are transitive (if I prefer A over B and B over C, then I prefer A over C).
* Human preferences are internally consistent.
* Human preferences are stable over time.
Apply these assumptions to finding an answer to the question of what you should make for dinner, and you'll quickly see there are problems.