> But somehow when a mechanically computed algorithm is used as a step in a process, all of this sort of reasoning goes out the window and now it's scary
Humans following procedure can be analyzed in meaningful ways by non-technical third parties to figure out where a failure point is
Black box program algorithms cannot. They require experts to analyze, who are also often the people who own the algorithm and wrote it in the first place
These algos are part of a process, so subject to the same kind of treatment. The car was a "black box" as far as its social impact when introduced, yet we managed to figure it out.
And a lot of forensic techniques, like fingerprints, flame spreading, graphology and others are epistemologically less sound than the kind of algo discussed in this article yet people seem comfortable with them. At least phrenology has been abandoned (AFAIK...eek)
Humans following procedure can be analyzed in meaningful ways by non-technical third parties to figure out where a failure point is
Black box program algorithms cannot. They require experts to analyze, who are also often the people who own the algorithm and wrote it in the first place