A tragic scandal at the UK Post Office highlights the need for legal change, especially as organizations embrace artificial intelligence to enhance decision-making.
Short of a floating point bug, computers don’t make mistakes. They do exactly what they’re programmed to do. The issue is the people developing them are fallible and QC has gone out the window globally, so you’re going to get computers that operate as good as the Devs and QC are.
There’s always small hardware quirks to be accounted for, but when we are talking about machine learning, which is not directly programmed, it’s less applicable to blame developers.
The issue is that computer system are now used to whitewash mistakes or biases with a veneer of objective impartiality. Even an accounting system’s results are taken as fact.
Consider that an AI trained with data from the history of policing and criminal cases might make racist decisions, because the dataset includes a plenty of racist bias, but it’s very easy for the people using it to say “welp, the machine said it so it must be true”. The responsibility for mistakes is also abstracted away, because the user and even the software provider might say they had nothing to do with it.
I the example you gave I would actually put the blame the software provider. It wouldn’t be ridiculously difficult to anonimize the data, get rid of name, race, gender, and leave only the information about the crime committed, the evidence, any extenuating circumstances, and the judgment.
It’s more difficult then simply throwing in all the data, but it can and should be done. It could still contain some bias, based on things like the location of the crime. But the bias would be already greatly reduced.
Short of a floating point bug, computers don’t make mistakes. They do exactly what they’re programmed to do. The issue is the people developing them are fallible and QC has gone out the window globally, so you’re going to get computers that operate as good as the Devs and QC are.
There’s always small hardware quirks to be accounted for, but when we are talking about machine learning, which is not directly programmed, it’s less applicable to blame developers.
The issue is that computer system are now used to whitewash mistakes or biases with a veneer of objective impartiality. Even an accounting system’s results are taken as fact.
Consider that an AI trained with data from the history of policing and criminal cases might make racist decisions, because the dataset includes a plenty of racist bias, but it’s very easy for the people using it to say “welp, the machine said it so it must be true”. The responsibility for mistakes is also abstracted away, because the user and even the software provider might say they had nothing to do with it.
I the example you gave I would actually put the blame the software provider. It wouldn’t be ridiculously difficult to anonimize the data, get rid of name, race, gender, and leave only the information about the crime committed, the evidence, any extenuating circumstances, and the judgment.
It’s more difficult then simply throwing in all the data, but it can and should be done. It could still contain some bias, based on things like the location of the crime. But the bias would be already greatly reduced.
Perfectly good computers do make random bit flip mistakes, and the smaller they get the more issues we will see with that.
Even highly QA’d code like they put on the space shuttle put 5 redundant computers in to reduce the chance they all fail.
Not every piece of software is worth the resources to do that though. If your game crashes just restart it.