Again, this is not my argument - just one that I've seen brought up in this context. It's also not completely wrong.
While it's possible you, your ancestors, or your countrymen did not directly or indirectly benefit a single cent from a transfer of wealth via colonialism or oppression of non-Europeans, I do honestly doubt that would be true for most people of European descent.
Nobody from east of Italy and German proto-state did not benefit from it, also Scandinavia.
If you want to include all the beneficiaries, you should include coastal Africans and all the traders including Arab and Jewish ones. Things will get messy. If you also talk about slavery then you also should not forget all the white slaves of the same time period.
I think that the most unfortunate fact about the end of American slavery was that the oppressed did not overpower the oppressors. At least not directly. As such there was no real liberation.
I have been thinking if it would have been best if all the black people would have been deported back to Africa. It would have given greater dignity that I feel is something that is really missing in the larger picture. Imagine if nobody in US was there because their ancestors were a former slave.
Try to put it into your head - most white people did not benefit from the oppression of the black people.
I did not, my ancestors did not, my countrymen did not.
I do not share any collective guilt with some American slave owners, or French colonists.
Like it was told, this policy is racist. Whoever came up with it is racist. Whoever defends it is racist.