Levels of AGI for Operationalizing Progress on the Path to AGI — part 1/3

Cleaned-text code points [0, 14980). Bibliography omitted here; source.md includes it.

# Position: Levels of AGI for Operationalizing Progress on the Path to AGI

###### Abstract

We propose a framework for classifying the capabilities and behavior of Artificial General Intelligence (AGI) models and their precursors. This framework introduces levels of AGI performance, generality, and autonomy, providing a common language to compare models, assess risks, and measure progress along the path to AGI. To develop our framework, we analyze existing definitions of AGI, and distill six principles that a useful ontology for AGI should satisfy. With these principles in mind, we propose “Levels of AGI” based on depth (performance) and breadth (generality) of capabilities, and reflect on how current systems fit into this ontology. We discuss the challenging requirements for future benchmarks that quantify the behavior and capabilities of AGI models against these levels. Finally, we discuss how these levels of AGI interact with deployment considerations such as autonomy and risk, and emphasize the importance of carefully selecting Human-AI Interaction paradigms for responsible and safe deployment of highly capable AI systems.

###### Keywords:

## 1 Introduction

Artificial General Intelligence (AGI) is an important and sometimes controversial concept in computing research, used to describe an AI system that is at least as capable as a human at most tasks. Given the rapid advancement of Machine Learning (ML) models, the concept of AGI has grown from a subject of philosophical debate, to one which also has near-term practical relevance. Some experts believe that “sparks” of AGI ( Bubeck et al., 2023 ) are already present in the latest generation of large language models (LLMs); some predict AI will broadly outperform humans within about a decade ( Bengio et al., 2023 ) ; some even assert that current LLMs are AGIs ( Agüera y Arcas & Norvig, 2023 ) .

The concept of AGI is important as it maps onto goals for, predictions about, and risks of AI:

Goals : Achieving human-level “intelligence” is an implicit or explicit north-star goal for many in our field, from the 1956 Dartmouth AI Conference ( McCarthy et al., 1955 ) that kick-started the modern field of AI, to today’s leading AI research firms, whose mission statements include goals such as “ensure transformative AI helps people and society” ( Anthropic, 2023a ) and “ensure that artificial general intelligence benefits all of humanity” ( OpenAI, 2023 ) .

Predictions : The concept of AGI is related to a prediction about progress in AI, namely that it is toward greater generality, approaching and exceeding human generality. Additionally, AGI is typically intertwined with a notion of “emergent” properties ( Wei et al., 2022 ) , i.e. capabilities not explicitly anticipated by the developer. Such capabilities offer promise, perhaps including abilities that are complementary to typical human skills, enabling new types of interaction or novel industries. Such predictions about AGI’s capabilities in turn predict likely societal impacts; AGI may have significant economic implications, i.e., reaching the necessary criteria for widespread labor substitution ( Ellingrud et al., 2023 ; Dell’Acqua et al., 2023 ; Eloundou et al., 2023 ) , as well as geo-political implications relating not only to the economic advantages AGI may confer, but also to military considerations ( Kissinger et al., 2022 ) .

Risks : Lastly, AGI is viewed by some as a concept for identifying the point when there are extreme risks ( Shevlane et al., 2023 ; Bengio et al., 2023 ) , as some speculate that AGI systems might be able to deceive and manipulate, accumulate resources, advance goals, behave agentically, outwit humans in broad domains, displace humans from key roles, and/or recursively self-improve.

In this position paper, we argue that it is critical for the AI research community to explicitly reflect on what we mean by “AGI,” and aspire to quantify attributes like the performance, generality, and autonomy of AI systems. Shared operationalizable definitions for these concepts will support: comparisons between models; risk assessments and mitigation strategies; clear criteria from policymakers and regulators; identifying goals, predictions, and risks for research and development; and the ability to understand and communicate where we are along the path to AGI.

## 2 Defining AGI: Case Studies

Many AI researchers and organizations have proposed definitions of AGI. In this section, we consider nine prominent examples, and reflect on their strengths and limitations. This analysis informs our subsequent introduction of a two-dimensional, leveled ontology of AGI.

Case Study 1: The Turing Test. The Turing Test ( Turing, 1950 ) is perhaps the most well-known attempt to operationalize an AGI-like concept. Turing’s “imitation game” attempts to operationalize the question of whether machines can think, and asks a human to interactively distinguish whether text is produced by another human or by a machine. The test as originally framed is a thought experiment, and is the subject of many critiques ( Wikipedia, 2023b ) ; in practice, the test often highlights the ease of fooling people ( Weizenbaum, 1966 ; Wikipedia, 2023a ) rather than the “intelligence” of the machine. Given that modern LLMs pass some framings of the Turing Test, it seems clear that this criteria is insufficient for operationalizing or benchmarking AGI. We agree with Turing that whether a machine can think, while an interesting philosophical and scientific question, seems orthogonal to the question of what the machine can do; the latter is much more straightforward to measure and more important for evaluating impacts. Therefore we propose that AGI should be defined in terms of capabilities rather than processes 1 1 1 As research into mechanistic interpretability ( Räuker et al., 2023 ) advances, it may enable process-oriented metrics. These may be relevant to future definitions of AGI. .

Case Study 2: Strong AI – Systems Possessing Consciousness . Philosopher John Searle mused, “according to strong AI, the computer is not merely a tool in the study of the mind; rather, the appropriately programmed computer really is a mind, in the sense that computers given the right programs can be literally said to understand and have other cognitive states” ( Searle, 1980 ) . While strong AI might be one path to achieving AGI, there is no scientific consensus on methods for determining whether machines possess strong AI attributes such as consciousness ( Butlin et al., 2023 ) , making this process-oriented framing impractical.

Case Study 3: Analogies to the Human Brain. The original use of the term “artificial general intelligence” was in a 1997 article about military technologies by Mark Gubrud ( Gubrud, 1997 ) , which defined AGI as “AI systems that rival or surpass the human brain in complexity and speed, that can acquire, manipulate and reason with general knowledge, and that are usable in essentially any phase of industrial or military operations where a human intelligence would otherwise be needed.” This early definition emphasizes processes (rivaling the human brain in complexity) in addition to capabilities; while neural network architectures underlying modern ML systems are loosely inspired by the human brain, the success of transformer-based architectures ( Vaswani et al., 2023 ) whose performance is not reliant on human-like learning suggests that strict brain-based processes and benchmarks are not inherently necessary for AGI.

Case Study 4: Human-Level Performance on Cognitive Tasks. Legg ( Legg, 2008 ) and Goertzel ( Goertzel, 2014 ) popularized the term AGI among computer scientists in 2001 ( Legg, 2022 ) , describing AGI as a machine that is able to do the cognitive tasks that people can typically do. This definition notably focuses on non-physical tasks (i.e., not requiring robotic embodiment as a precursor to AGI). Like many definitions of AGI, this framing presents ambiguity around choices such as “what tasks?” and “which people?”.

Case Study 5: Ability to Learn Tasks. In The Technological Singularity ( Shanahan, 2015 ) , Shanahan suggests that AGI is “artificial intelligence that is not specialized to carry out specific tasks, but can learn to perform as broad a range of tasks as a human.” An important property of this framing is its inclusion of metacognitive capabilities (learning) as a requirement for AGI.

Case Study 6: Economically Valuable Work. OpenAI’s charter defines AGI as “highly autonomous systems that outperform humans at most economically valuable work” ( OpenAI, 2018 ) . This definition has strengths per the “capabilities, not processes” criteria, as it focuses on performance agnostic to underlying mechanisms; further, this definition offers a potential yardstick for measurement, i.e., economic value. A shortcoming of this definition is that it does not capture all of the criteria that may be part of “general intelligence.” There are tasks associated with intelligence that may not have a well-defined economic value (e.g., artistic creativity or emotional intelligence). Such properties may be indirectly accounted for in economic measures (e.g., artistic creativity might produce books or movies, emotional intelligence might relate to the ability to be a successful CEO), though whether economic value captures the full spectrum of “intelligence” remains unclear. Another challenge with framing AGI in terms of economic value is the implied need for deployment in order to realize that value, whereas a focus on capabilities might only require the potential for an AGI to execute a task. We may develop systems that are technically capable of performing economically important tasks but don’t realize that economic value for varied reasons (legal, ethical, social, etc.).

Case Study 7: Flexible and General – The “Coffee Test” and Related Challenges. Marcus suggests that AGI is “shorthand for any intelligence (there might be many) that is flexible and general, with resourcefulness and reliability comparable to (or beyond) human intelligence” ( Marcus, 2022b ) . This definition captures both generality and performance (via the inclusion of reliability); the mention of “flexibility” is noteworthy, since, like the Shanahan formulation, this suggests that metacognitive capabilities, such as the ability to learn new skills, are necessary to make an AI system sufficiently general. Further, Marcus proposes five tasks to gauge success (understanding a movie, understanding a novel, cooking in an arbitrary kitchen, writing a bug-free 10,000 line program, and converting natural language mathematical proofs into symbolic form) ( Marcus, 2022a ) . Accompanying a definition with a benchmark is valuable; however, more work would be required to make this benchmark comprehensive. While failing some of these tasks may indicate a system is not an AGI, it is unclear that passing them is sufficient for AGI status. In Section 5 , we further discuss the challenge in developing a set of tasks that is both necessary and sufficient for capturing the generality of AGI. We also note that one of Marcus’ proposed tasks, “work as a competent cook in an arbitrary kitchen” (a variant of Steve Wozniak’s “Coffee Test” ( Wozniak, 2010 ) ), requires robotic embodiment; this differs from other definitions that focus on non-physical tasks 2 2 2 Though robotics might also be implied by the OpenAI charter’s focus on “economically valuable work,” OpenAI shut down its robotics research division in 2021 ( Wiggers, 2021 ) , suggesting this is not their intended interpretation. .

Case Study 8: Artificial Capable Intelligence. Suleyman proposed the concept of “Artificial Capable Intelligence (ACI)” ( Mustafa Suleyman and Michael Bhaskar, 2023 ) to refer to AI systems with sufficient performance and generality to accomplish complex, multi-step tasks in the open world. More specifically, Suleyman proposed an economically-based definition of ACI skill that he dubbed the “Modern Turing Test,” in which an AI would be given $100,000 of capital and tasked with turning that into $1,000,000 over a period of several months. This framing is more narrow than OpenAI’s definition of economically valuable work and has the additional downside of potentially introducing alignment risks ( Kenton et al., 2021 ) by only targeting fiscal profit. However, a strength of Suleyman’s concept is the focus on performing a complex, multi-step task that humans value. Construed more broadly than making a million dollars, ACI’s emphasis on complex, real-world tasks is noteworthy, since such tasks may have more ecological validity than many current AI benchmarks; Marcus’ aforementioned five tests of flexibility and generality ( Marcus, 2022a ) seem within the spirit of ACI, as well.

Case Study 9: SOTA LLMs as Generalists. Agüera y Arcas and Norvig ( Agüera y Arcas & Norvig, 2023 ) suggested that state-of-the-art LLMs (e.g. mid-2023 deployments of GPT-4, Bard, Llama 2, and Claude) already are AGIs, arguing that generality is the key property of AGI, and that because language models can discuss a wide range of topics, execute a wide range of tasks, handle multimodal inputs and outputs, operate in multiple languages, and “learn” from zero-shot or few-shot examples, they have achieved sufficient generality. While we agree that generality is a crucial characteristic of AGI, we posit that it must also be paired with a measure of performance (i.e., if an LLM can write code or perform math, but is not reliably correct, then its generality is not yet sufficiently performant).

## 3 Defining AGI: Six Principles

Reflecting on these nine example formulations of AGI (or AGI-adjacent concepts), we identify properties and commonalities that we feel contribute to a clear, operationalizable definition of AGI. We argue that any definition of AGI should meet the following six criteria:

1. Focus on Capabilities, not Processes. The majority of definitions focus on what an AGI can accomplish, not on the mechanism by which it accomplishes tasks. This is important for identifying characteristics that are not necessarily a prerequisite for achieving AGI (but may nonetheless be interesting research topics). This focus on capabilities implies that AGI systems need not necessarily think or understand in a human-like way (since this focuses on processes); similarly, it is not a necessary precursor for AGI that systems possess qualities such as consciousness (subjective awareness) ( Butlin et al., 2023 ) or sentience (the ability to have feelings), since these qualities have a process focus.

2. Focus on Generality and Performance. All of the above definitions emphasize generality to varying degrees, but some exclude performance criteria. We argue that both generality and performance are key components of AGI. In Section 4 we introduce a leveled taxonomy that considers the interplay between these dimensions.