Mostly, yes. but there were clearly common people in history that were described as 'somewhat eccentric' but still managed a normal-ish life otherwise. The chances are most of those were people who would today be labelled 'high functioning autistic' or 20 years ago would have been labelled as having aspergers.

What money/class-status afforded was the ability to be more eccentric without being pushed into an institution.