Bearing the Moral Weight: What's Lost When AI Decides?

Abstract

This paper argues that there are some cases where having a decision made by a non-human entity (even if human experts agree that it is the right decision) would itself be an injustice. The reasons why a human decision is warranted go beyond the current limitations of artificial intelligence (e.g., embedded bias, lack of explainability, and inaccuracy). Instead, in cases where someone’s fundamental human interests are at stake (especially life and liberty interests), respect for human dignity requires a moral agent owning the decision, what I characterize as “bearing the moral weight.” Following Darwall, I ground my argument in the importance of shared vulnerability and in the demands of recognition respect. While one might think the primary issue here is about ensuring accountability for mistakes, my concern goes deeper. I argue that we have a responsibility for one another. Outsourcing this responsibility to AI in high-stakes cases undermines our sense of effective responsibility for our communities and runs the risk of degrading human dignity. This is especially problematic in the current political climate given decreases in social and institutional trust. Rather than continue to improve ourselves and commit to genuinely seeing one another, turning to AI in cases like these suggests giving up on the potential for us to do so and of our responsibility to try. For these reasons, I maintain that we should be leery about championing growth in AI in areas of heightened human vulnerability without more clear-eyed public conversations about the possible loss.