Did both for quite some time: Working remotely means getting thing done, getting into flow, and being productive. Working on-site means building strong relationships to people which might be key for one's further career development. I guess latter is more important when it is about increasing the personal market value.

Not saying that you can't build stromg work relations remotely but it's just different.