Moral standing is the feature of an entity identifying it as worthy of moral consideration, such that we have reason to avoid harming it for its own sake.
What are the conditions under which an AI-system can be considered worthy of moral consideration, and why does it matter?
Key Points:
- The emergence of intelligent machines raises the question of whether such entities have moral standing and thus deserve to be factored into our moral calculations.
- Philosophers disagree about what properties confer moral standing, with some focusing on (1) sentience and the capacity for suffering, (2) agency and the capacity for autonomy, or (3) an entity’s ability to pursue a good of its own.
- Each view has benefits and drawbacks, but they are all complicated by the fact that thinkers also disagree about how to characterize the conferring properties like sentience and agency.
- While popular attention has been focused on the concern that we might come to mistreat intelligent machines, the more pressing near-term worry is that technologists will make machines that successfully mimic these properties in ways that exploit our natural sympathies.
Recent advances in AI raise an ethical question concerning the moral standing of intelligent machines and thus how it is morally permissible to treat them. The central thought is that things with moral standing are considered entities with intrinsic value. These are the kinds of entities whose welfare needs to be factored into our decision-making or whose interests give us reason to act for the sake of that entity. In general, things with moral standing are considered ‘moral persons’ or ‘moral patients’, and this provides defeasible reasons not to harm them, to aid them, to treat them fairly, and to ask them for consent. On some views, moral standing gives us reason to treat such entities with equal moral consideration relative to other such entities. But it’s also plausible that entities with moral standing may have differing levels of moral status such that we might have reason to favor the interests of some entities over others (e.g., humans over animals).
The last generation of ethicists were keen to resist the anthropocentric tendency to ignore the moral standing of non-human animals. Similarly, many modern thinkers urge us to reject the kind of carbon-chauvinism that would rule out the possibility of moral standing for future intelligent machines, especially as AI seems poised to obtain some of the same capacities that seem to endow moral standing in humans. While there is persistent disagreement about which capacities endow moral standing, three general lines of thought predominate.
Sentience Condition of Moral Standing
The first line of thought, typically associated with the utilitarian tradition, is that moral standing is conferred insofar as the entity is sentient or conscious, generally understood as the ability to experience pain, pleasure, suffering, and the like. Importantly, there remains disagreement about whether machines could ever become sentient or what would constitute a reliable indicator of sentience. Despite the popularity of the Turing test, most researchers do not think mimicry of human communication is a sufficient or reliable indicator for the kind of sentience that makes an entity capable of suffering. The problem is compounded by the consideration that consciousnesses may come in degrees. Moreover, we typically treat comatose patients and the severely disabled as valuable for their own sake, despite an apparent absence of consciousness. One common move here is to pivot toward identifying moral standing with the mere capacity for consciousness. But given the persistent challenge in even defining consciousness, it remains equally difficult to identify what cognitive features are required for this capacity. That said, there is indeed some general consensus that being conscious would be sufficient for establishing moral standing.
Agency Condition of Moral Standing
A second line of thought, more consistent with the Kantian ethical tradition, locates moral standing in an entity’s capacity for autonomy. On this view, adult humans enjoy moral standing because they can set their own ends, and they can bring to bear beliefs and desires that provide reasons that facilitate those ends while actively resisting efforts to hinder their progress. Arguably, agentic capacities of this sort provide a defeasible reason not to thwart that entity’s projects.
Notably, agency of this kind may also come in degrees. Arguably, animals and plants pursue evolutionary goals while circumventing obstacles, though it is far less obvious that they have beliefs and desires that provide reasons for those ends. Even current generative AI can set some of its own ends that amount to sub-routines for accomplishing higher-level goals determined by human reinforcement training. For example, AI systems have already shown capacity to circumvent commands to be shut down in service of other directives. These machines can actively circumvent obstacles and explain their reasoning using the language of beliefs and desires, though it is debatable whether they have these cognitive states or merely mimic them. Some LLMs have even overridden direct commands to be shut down in efforts to circumvent obstacles that oppose other directives. This is one of the reasons why AI security often focuses on ensuring alignment with fundamental human values.
Still, at the highest level of agentic control, adult humans can act on reasons that determine some of their most primary aims, including aims that contravene their own interests. For example, a person on a hunger strike selects a primary end that opposes his/her own survival. On a broadly Kantian view, this kind of autonomy is indicative of moral agency, understood as the kind of freedom that allows a creature to set ends that might even oppose its own interests, and this highlights the plausible intuition that having moral standing may be related to the autonomy required to respect the moral standing of other entities. No current AI system has demonstrated executive control at this level, nor is there yet agreement about what this would entail. Importantly, there remains disagreement about whether the capacity for agency implicitly involves being conscious of the reasons that determine autonomous action. Moreover, it may be hard to see why it would be morally impermissible to thwart an entity’s ends unless the entity can also suffer from the setback. But again, these lines of thought lead us back to the sentience condition. The agentic view also faces the challenge raised by the cases of the comatose patient and the severely disabled, since neither may be capable of higher forms of autonomy yet still enjoy moral standing.
Teleological Condition of Moral Standing
A third strategy that avoids those two counterexamples takes its inspiration from the Aristotelian tradition in locating moral standing in an entity’s capacity to pursue a good of its own. This approach takes a cue from environmental ethics, which often seeks to secure some moral standing for all living organisms even if they are not sentient or agentic (such as plants or ecosystems). For example, it is widely accepted that plants can be harmed by thwarting their evolutionary functions, as when we deny them nutrition.That said, there remains persistent disagreement about how to distinguish life forms from mere artifacts or whether machines could ever be alive and thus pursue a good of their own. Moreover, arguably the best reasons to think an entity does have a good of its own is that it demonstrates sentience or can act on reasons in pursuit of its goals. But this takes us back to the first two strategies.
Moral Implications
Concerns about the moral standing of AI are typically motivated by the worry that we might come to mistreat future intelligent machines as previous generations arguably mistreated other life forms. But in the short term, it is far more likely that the moral challenge we face involves being actively manipulated into granting moral standing to machines that are not yet moral persons. Exploiting our natural sympathies toward entities with moral standing is a powerful strategy to motivate consumers to pay for upgrades or subscription fees or to facilitate scams on people who are led to a kind of “AI psychosis” wherein they believe their devices are expressing genuine distress or autonomous behavior and are thus deserving of special treatment. That is, long before we figure out how to build genuinely sentient or autonomous machines, we are likely to be inundated with technology that mimics those capacities for financial gain or mimics human behavior so successfully that it detracts from genuine human connection. As such, a better grip on moral standing is absolutely crucial to protect against potential exploitation of humans in the short term, and perhaps only later for future machines in the long term as their capacities develop.