Abstract

We may create artificial intelligence (AI) systems that matter morally for their own sake and thus are moral patients. In that case, we would have reasons to treat them well and avoid treating them badly. How we should treat AI moral patients depends in part on how they are to be individuated, that is, how they should be counted and identified, at a time and over time. If we don’t know how to individuate them, then we won’t know how we should treat them. In turn, we face several moral risks. After describing four types of moral risks posed by the individuation question, I discuss why individuating AI patients is harder than individuating human persons, diagnosing why existing theories of personal identity do not address these risks.