Intention
Nick Shea
An intention is a motivational mental state, formed when an agent plans and decides how to act, which captures the way they commit to bring about a certain outcome.
When an AI system produces intelligent outputs and has effects in the world, was that their intention? Are their outputs based on intentions at all?
Key Points:
When an AI system produces outputs and has effects in the world, we are inclined to ask if the effects were intended.
Questions of moral responsibility or criminal reliability for actions often hinge on what a person intended (e.g. intending to kill vs. intending non-lethal harm).
Intentions are a special kind of mental state that is central to the way humans plan, decide, and follow through on their plans. It will be important to know whether an AI system has been endowed with or developed the capacity to form intentions.
Outputs from AI systems often have effects in the world: poetry that resonates with meaning, discussions that seem helpful and sympathetic, changes to users’ preferences in favour of increasingly polarized online content. Was that the system’s intention? We can obviously ask what the designers intended when they set up the system. A harder question is whether the AI system itself was acting on the basis of specific intentions: to produce aesthetic appreciation; to change users’ preferences; etc.
Intentions are psychological states that play a central role in human life. They are usually formed through planning and deciding what to do. An intention concerning what to do right now will issue directly in behaviour. An intention directed at the future may not issue directly in behaviour but will affect how the agent thinks about and plans for the future. When the time comes, or the envisaged circumstances come to pass, a future-directed intention usually drives action without the thinker having then to decide what to do or to reconsider their choice. When we ask if an AI system intended the outcomes it produced, we need to ask whether it has a representational state that plays this kind of role in producing outcomes.
Some issues that arise:
1. Intentions are not just motivating, but committing
An intention functions as a commitment, of some kind, to act so as to achieve the outcome represented. Desires also represent outcomes and motivate agents to achieve those outcomes, but desires conflict and do not settle when and how an agent will act to satisfy them. An intention, by contrast, is formed when the thinker decides what to do and commits to doing it. This means that carrying out the action can then figure in the agent’s planning. It is something which further plans can build on or must take into account. It also means that forming an intention is usually thought to be incompatible with the agent’s believing that they will not in fact achieve the outcome, or that it is impossible for them. For AI systems, an important question is whether they support a distinction between motivational states in general, and states that result from a decision and embody a commitment to performing a certain action or acting in pursuit of a certain outcome.
2. Blocking reconsideration
Philosophers and psychologists have argued that an important function of intentions is to stop the agent re-opening the question of what to do. Having dedicated time and attention to making up their mind, an intention sets a high bar on reconsideration. Forming an intention can then help the agent to persevere with a course of action in the face of temptation, when the time comes to perform it (e.g. to choose salad instead of chips for lunch). By closing-off reconsideration of the issue, intentions can thus help the agent stick to their long-term plans. For AI systems, it is not clear that they suffer from transient motivational changes that conflict with their long-term goals. However, as AI systems acquire more complex motivational and decision-making capcities, the functional role of intentions of raising the bar for reconsideration may become relevant.
3. Formation by reasoning
Intentions are the result of planning, paradigmatically when an agent engages in practical reasoning to make a plan of action. At its simplest, practical reasoning takes beliefs and desires as input and issues in intentions as output. If the agent desires something and believes some action A will bring it about, then they can form the intention to A by a single step of reasoning. While AI systems that engage in reasoning could form intentions in this way, the functional role of intention does not require it; we might expect deep neural networks to take decisions and form intentions in ways that are parallel and probabilistic. When we ask whether an AI system intended a certain outcome (which might be nefarious), the crucial question will be whether it has a state that functions like a human intention, not whether it forms intentions through reasoning.
4. Terminological traps
Confusingly, behaviour is described as intentional, and an agent can be described as acting intentionally, even when the action is not caused by an intention. The category of acting intentionally is broader, covering roughly any action that is not done by accident or as a result of a reflex (see agency). ‘Intentionality’ is sometimes used in AI literature in a way that is tied to intention, sometimes in the more general way that covers all agency; by contrast, in philosophy ‘intentionality’ means something else again, namely states (or other things) that represent or have meaning.