"Over one third of my reputation was "earned" from me doing absolutely nothing for over two years. Indeed I went from the top 4% of contributors at my time of departure to the top 3%, despite, you know, me not doing anything."

I don't see the problem here. He's not getting points for doing nothing. He's getting points for something he did in the past. Sort of like royalties.