> If "the poor" move to less dense areas then who will wait your tables, pick up your garbage, clean your offices, cook your food, check out your groceries, etc?

That is not the reason people stay in HCOL areas. There is value for them there, or the cost to leave seems higher than staying.