The perils of Big Data: How crunching numbers can lead to moral blunders
Considering data at a distance makes it perilously easy to overlook the stories the data does not tell. What would a strategic management consultancy have done if they had been handed the data of wealthy slaveholders? Would they have suggested ways to tweak profits? Or perhaps recommended lobbying Congress to prevent abolition? Hopefully not.
Machine Bias
If computers could accurately predict which defendants were likely to commit new crimes, the criminal justice system could be fairer and more selective about who is incarcerated and for how long. The trick, of course, is to make sure the computer gets it right. If it’s wrong in one direction, a dangerous criminal could go free. If it’s wrong in another direction, it could result in someone unfairly receiving a harsher sentence or waiting longer for parole than is appropriate.
https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing