Commenting specifically on the black names example: I am skeptical because they replicated the results of a study on humans based on the Implicit Association Test. I would be generous in calling that the Myers-Briggs of this generation, but it's probably even less valuable other in that it's free to take online so a huge data set is easily available.
Just because humans are biased doesn't mean we have to make our algorithms biased too. This is especially important now that AI and ML are starting to be deployed in non-CS cases.
Like, the IAT has problems, but it has been shown to have some predictive validity in behaviour. I am entirely unsurprised that the IAT finding replicated in this data, and it actually supports the structural racism explanation propounded by one of the measure's harshest critics.
To be fair to the IAT, while it has very low test-retest reliability and lots and lots of state variance, there's definitely something in the reaction time differences that is meaningful. How meaningful it is, and whether or not it makes any practical difference remains to be determined.
