Very, very true. Brunei and Kuwait are terrible offenders - huge sections of the population (more than half in Brunei) aren't citizens and can't be, no matter how many generations they've been there.
Think about big that has to be - it's more than any of the most developed EU states to the rest of the EU (how many foreign workers are in the UK? Germany? A lot. And they probably pay much better). It's more than comes out of Russia, when half the GDP of some of the ex-Soviet states is Russian remittances.
So while maybe you're entitled to certain things as a citizen, a huge amount of what makes that possible is done by people who aren't.