It will never happen, but that's not the reason. The reason is that voting data is an extremely intimate reflection of users' feelings and beliefs. None of us would want that picture of ourselves to become public, and I shudder to think of what people would do to each other with the information.
I agree with this. But do you think only showing the first X votes could help resolve it? If you want to keep your upvote private, don't upvote if there are < X votes.