Superintelligence is the capacity for general reasoning and problem solving at a level that exceeds ordinary humans.
Are current AI systems superintelligent? If not, will artificial superintelligence follow from current developments in deep learning?
Key Points:
- When a single computational system like a large language model (LLM) can solve a wide range of problems as well as individual humans, it arguably displays artificial general intelligence (AGI).
- This raises the prospect of artificial systems whose general intelligence significantly exceeds that of humans.
- We need to clarify whether superintelligence involves reasoning in the same way as humans but better (e.g. faster, with fewer errors) or whether it involves qualitatively different capacities.
What is superintelligence? Like intelligence itself, the concept of superintelligence is somewhat hard to pin down. But we get some purchase on it by considering what sorts of mental powers are outside the realm of current human capability.
To say what superintelligence is it is useful to first say what human intelligence is [cross ref: intelligence]. Humans possess a capacity for problem solving and reasoning which is significantly more generally-applicable than that found in other animals. The emphasis on ‘general’ here is crucial, because whereas many of our animal brethren may display intelligent behavior within their ecological niche, this behavior often does not generalize to other contexts. Human beings, on the other hand, are able to generalize things we learn in one environment to others. Compared to many other creatures, we can do a lot of thinking quite fast. It is in this sense that humans are more intelligent than snails and that octopuses are more intelligent than sea slugs. Furthermore, we reason in a way that is logically valid, a form of reasoning that can be applied to any domain.
But human reasoning capacities are also held back by all sorts of performance issues—our minds are imperfect machines. For instance, there is a limit to the number of things human beings can attend to, or hold in working memory, at any given time (the number typically cited is seven, plus or minus two). One thing we might mean by superintelligence, then, is a human intelligence that is unburdened by the sorts of issues that arise in discussions of the competence-performance distinction. Competence refers to those things we are theoretically able to do given our cognitive endowment: what is in-principle possible given our underlying knowledge of rules (e.g. those involved in constructing and interpreting grammatical sentences). Performance, on the other hand, refers to our actual deployment of these capacities in the real world. Some of the best known examples of this distinction come from the study of recursion and composition in language—in theory, we can produce and, importantly, understand infinitely long meaningful sentences generated with recursive tools:
- Jane believes it’s raining.
- John believes Jane believes it’s raining.
- Jane believes John believes Jane believes it’s raining.
- John believes Jane believes John believes Jane believes it’s raining.
- Jane believes John believes Jane believes John believes Jane believes…
But most of us tap out pretty early when it comes to being able to understand or do anything with such sentences. Processing lots of information is hard work, and there are limits on—for instance—the number of bits of information we can hold in verbal working memory at any given time.
A key way of thinking about superintelligence, then, is in terms of a sufficient relaxation of these kinds of performance constraints. The borders between enhanced intelligence and superintelligence will be somewhat vague, and this vagueness will extend across multiple dimensions. For example, working memory is the capacity a human or animal has to hold information that can be attended to and actively manipulated. Humans have a tightly-limited capacity to hold information in working memory at a given time. A system with the capacity to hold much more information in working memory at a given time might be thought of as possessing an extraordinary or superhuman capability. But is it beyond the realm of possible human enhancement? It is unclear where we would draw the line between enhanced human intelligence and the kind of capacity which would make a sufficiently enhanced human qualify as superintelligent.
There are other ways of thinking about superintelligence: for instance, in terms of a kind of unburdened computational ability that systematically and robustly outperforms humans on thinking tasks, but bears little resemblance to human competences. That is, a sort of alien intelligence that we are nevertheless able to recognize as vastly computationally more powerful than our own.
Some issues that arise:
1. It’s also important to say something about what superintelligence is not. When I ask Chat GPT to briefly summarize and comment on key debates in the last 70 years of formal language theory, it does so much faster than I ever could. Does this make Chat GPT a superintelligence? No. It’s better than I am at this one sort of thing, but that’s about it: and if you spend enough time chatting with Chat GPT about philosophy you’ll quickly find that while it’s quite fast, it’s not very deep. But superintelligence is about general skills. Sometimes superintelligence is even described in terms of an intelligence that is far superior to human intelligence in every way. While this may be too stringent, it points to the kind of generality required for superintelligence.
2. The prospect of superintelligence raises the key question of whether superintelligent machines would have greater agency than humans.
This depends on whether superintelligence entails greater capacities for things like empathy, affect, and emotion. And that depends on the extent to which these things can be enhanced through enhancements to cognitive / computational ability. A superintelligence might be a cold and calculating supercomputer with little respect for human life—but it might also be warm, benevolent, and forgiving.
3. Superintelligence underpins the most influential concern about the safety of AI: Is superintelligence inevitable and will superintelligence lead to an ‘intelligence explosion’? Superintelligence is related to what is sometimes called an ‘intelligence explosion’ and more cryptically ‘the singularity’. An intelligence explosion refers to a hypothetical event in which an artificial intelligence achieves the ability to design and produce artificial intelligences even more powerful than itself, entering a positive feedback loop of unchecked and unstoppable self-improvement. Such an event–cryptically referred to as ‘the singularity’–would represent an unprecedented change in the structure and governance of human lives.