This argument only works if you assume that people who go vegan enjoys food less than they did before they went vegan. The reality is that most vegans report enjoying food much more. The reason for that is that they are forced to experiment with different flavors and get more creative, which expands their culinary horizons.
This is a faulty argument. People who join various religious sects say the same exact things.
Overlooking whether vegans or religious sects are good, bad or whatever, what really happens is probably that people find other vegan people, become part of a community and enjoy that.
It's the same cargo cult process that makes everyone want an Iphone. I dread the day when a high percentage or, god forbid, majority of people start looking down on those dirty meat-eaters because "won't anyone think of the animals?".