Consciousness in Artificial Intelligence: Insights from the Science of Consciousness — part 1/14

Cleaned-text code points [0, 15269). Bibliography omitted here; source.md includes it.

Consciousness in Artificial Intelligence: Insights from the Science of Consciousness Patrick Butlin 1 1 1 Joint first authors and corresponding authors (patrickbutlin@gmail.com, rgblong@gmail.com) Robert Long 1 1 1 Joint first authors and corresponding authors (patrickbutlin@gmail.com, rgblong@gmail.com) Eric Elmoznino Yoshua Bengio Jonathan Birch Axel Constant George Deane Stephen M. Fleming Chris Frith Xu Ji Ryota Kanai Colin Klein Grace Lindsay Matthias Michel Liad Mudrik Megan A. K. Peters Eric Schwitzgebel Jonathan Simon Rufin VanRullen Abstract

Whether current or near-term AI systems could be conscious is a topic of scientific interest and increasing public concern. This report argues for, and exemplifies, a rigorous and empirically grounded approach to AI consciousness: assessing existing AI systems in detail, in light of our best-supported neuroscientific theories of consciousness. We survey several prominent scientific theories of consciousness, including recurrent processing theory, global workspace theory, higher-order theories, predictive processing, and attention schema theory. From these theories we derive ”indicator properties” of consciousness, elucidated in computational terms that allow us to assess AI systems for these properties. We use these indicator properties to assess several recent AI systems, and we discuss how future systems might implement them. Our analysis suggests that no current AI systems are conscious, but also suggests that there are no obvious technical barriers to building AI systems which satisfy these indicators. 1 1 1 A previous version of this sentence read ”…but also shows that there are no obvious barriers to building conscious AI systems.” We have amended it to better reflect the messaging of the report: that satisfying these indicators may be feasible. But satisfying the indicators would not mean that such an AI system would definitely be conscious.

Authors

Patrick Butlin* , Future of Humanity Institute, University of Oxford Robert Long* , Center for AI Safety Eric Elmoznino , University of Montreal and MILA - Quebec AI Institute Yoshua Bengio , University of Montreal and MILA - Quebec AI Institute Jonathan Birch , Centre for Philosophy of Natural and Social Science, London School of Economics and Political Science Axel Constant , School of Engineering and Informatics, The University of Sussex and Centre de Recherche en Éthique, University of Montreal George Deane , Department of Philosophy, University of Montreal Stephen M. Fleming , Department of Experimental Psychology and Wellcome Centre for Human Neuroimaging, University College London Chris Frith , Wellcome Centre for Human Neuroimaging, University College London and Institute of Philosophy, University of London Xu Ji , University of Montreal and MILA - Quebec AI Institute Ryota Kanai , Araya, Inc. Colin Klein , School of Philosophy, The Australian National University Grace Lindsay , Department of Psychology and Center for Data Science, New York University Matthias Michel , Center for Mind, Brain and Consciousness, New York University Liad Mudrik , School of Psychological Sciences and Sagol School of Neuroscience, Tel-Aviv University and CIFAR Program in Brain, Mind and Consciousness Megan A. K. Peters , Department of Cognitive Sciences, University of California, Irvine and CIFAR Program in Brain, Mind and Consciousness Eric Schwitzgebel , Department of Philosophy, University of California, Riverside Jonathan Simon , Department of Philosophy, University of Montreal Rufin VanRullen , Centre de Recherche Cerveau et Cognition, CNRS, Université de Toulouse

Details

Authorship statement: PB and RL are joint first authors. PB and RL planned and coordinated the project and formulated the core ideas. PB drafted the majority of the report with substantial contributions from RL. EE wrote the first drafts of sections 3.1.2 and 3.1.3, GD wrote the first draft of section 4.1.2, and GL wrote the box on attention. All authors participated in workshops where we planned the report, developed the ideas and reviewed drafts. Authors other than PB, RL and EE are listed in alphabetical order.

Acknowledgements: The authors would like to thank: Nick Bostrom, who proposed the project to PB and RL and contributed in the early stages; Tim Bayne, Matt Botvinick, David Chalmers, Hakwan Lau, Matt McGill and Murray Shanahan, who attended workshops or took part in other discussions contributing to the preparation of the report; Xander Balwit for her help as a research assistant in the final stages of the project; and Charlie Thompson for his help with graphics and formatting.

Funding: • RL and PB ran workshops for the report that were supported by Effective Ventures and the EA Long-Term Future Fund.

• EE was supported by the FRQNT Strategic Clusters Program (Centre UNIQUE) and a Vanier Doctoral Canada Graduate Scholarship.

• AC was supported by European Research Council grant (XSCAPE) ERC-2020-SyG 951631.

• YB, AC, GD and JS were supported by a grant from Open Philanthropy.

• JS was supported by a grant from FRQ and a grant from SSHRC

• MM was supported by the Templeton World Charity Foundation, as part of the grant ‘Analyzing and Merging Theories of Consciousness’, at the Center for Mind, Brain, and Consciousness (NYU).

• SMF was supported by a Wellcome/Royal Society Sir Henry Dale Fellowship (206648/Z/17/Z).

• SMF, LM and MAKP were supported by Fellowships from the CIFAR Program in Brain, Mind and Consciousness.

• XJ was supported by IVADO.

• CK was supported by Templeton World Charity Foundation grant TWCF-2020-20539.

The authors have no conflicts of interest to report.

Executive Summary

The question of whether AI systems could be conscious is increasingly pressing. Progress in AI has been startlingly rapid, and leading researchers are taking inspiration from functions associated with consciousness in human brains in efforts to further enhance AI capabilities. Meanwhile, the rise of AI systems that can convincingly imitate human conversation will likely cause many people to believe that the systems they interact with are conscious. In this report, we argue that consciousness in AI is best assessed by drawing on neuroscientific theories of consciousness. We describe prominent theories of this kind and investigate their implications for AI.

We take our principal contributions in this report to be:

1. Showing that the assessment of consciousness in AI is scientifically tractable because consciousness can be studied scientifically and findings from this research are applicable to AI;

2. Proposing a rubric for assessing consciousness in AI in the form of a list of indicator properties derived from scientific theories;

3. Providing initial evidence that many of the indicator properties can be implemented in AI systems using current techniques, although no current system appears to be a strong candidate for consciousness.

The rubric we propose is provisional, in that we expect the list of indicator properties we would include to change as research continues.

Our method for studying consciousness in AI has three main tenets. First, we adopt computational functionalism , the thesis that performing computations of the right kind is necessary and sufficient for consciousness, as a working hypothesis. This thesis is a mainstream—although disputed—position in philosophy of mind. We adopt this hypothesis for pragmatic reasons: unlike rival views, it entails that consciousness in AI is possible in principle and that studying the workings of AI systems is relevant to determining whether they are likely to be conscious. This means that it is productive to consider what the implications for AI consciousness would be if computational functionalism were true. Second, we claim that neuroscientific theories of consciousness enjoy meaningful empirical support and can help us to assess consciousness in AI. These theories aim to identify functions that are necessary and sufficient for consciousness in humans, and computational functionalism implies that similar functions would be sufficient for consciousness in AI. Third, we argue that a theory-heavy approach is most suitable for investigating consciousness in AI. This involves investigating whether AI systems perform functions similar to those that scientific theories associate with consciousness, then assigning credences based on (a) the similarity of the functions, (b) the strength of the evidence for the theories in question, and (c) one’s credence in computational functionalism. The main alternative to this approach is to use behavioural tests for consciousness, but this method is unreliable because AI systems can be trained to mimic human behaviours while working in very different ways.

Various theories are currently live candidates in the science of consciousness, so we do not endorse any one theory here. Instead, we derive a list of indicator properties from a survey of theories of consciousness. Each of these indicator properties is said to be necessary for consciousness by one or more theories, and some subsets are said to be jointly sufficient. Our claim, however, is that AI systems which possess more of the indicator properties are more likely to be conscious. To judge whether an existing or proposed AI system is a serious candidate for consciousness, one should assess whether it has or would have these properties.

The scientific theories we discuss include recurrent processing theory, global workspace theory, computational higher-order theories, and others. We do not consider integrated information theory, because it is not compatible with computational functionalism. We also consider the possibility that agency and embodiment are indicator properties, although these must be understood in terms of the computational features that they imply. This yields the following list of indicator properties:

We outline the theories on which these properties are based and describe the evidence and arguments that support them in section 2 of the report, as well as explain the formulations used in the table.

Having formulated this list of indicator properties, in section 3.1 we discuss how AI systems could be constructed, or have been constructed, with each of the indicator properties. In most cases, standard machine learning methods could be used to build systems that possess individual properties from this list, although experimentation would be needed to learn how to build and train functional systems which combine multiple properties. There are some properties in the list which are already clearly met by existing AI systems (such as RPT-1, algorithmic recurrence), and others where this is arguably the case (such as the first part of AE-1, agency). Researchers have also experimented with systems designed to implement particular theories of consciousness, including global workspace theory and attention schema theory.

In section 3.2, we consider whether some specific existing AI systems possess the indicator properties. These include Transformer-based large language models and the Perceiver architecture, which we analyse with respect to the global workspace theory. We also analyse DeepMind’s Adaptive Agent, which is a reinforcement learning agent operating in a 3D virtual environment; a system trained to perform tasks by controlling a virtual rodent body; and PaLM-E, which has been described as an “embodied multimodal language model”. We use these three systems as case studies to illustrate the indicator properties concerning agency and embodiment. This work does not suggest that any existing AI system is a strong candidate for consciousness.

This report is far from the final word on these topics. We strongly recommend support for further research on the science of consciousness and its application to AI. We also recommend urgent consideration of the moral and social risks of building conscious AI systems, a topic which we do not address in this report. The evidence we consider suggests that, if computational functionalism is true, conscious AI systems could realistically be built in the near term.

## 1 Introduction

In the last decade, striking progress in artificial intelligence (AI) has revived interest in deep and long-standing questions about AI, including the question of whether AI systems could be conscious. This report is about what we take to be the best scientific evidence for and against consciousness in current and near-term AI systems.

Because consciousness is philosophically puzzling, difficult to define and difficult to study empirically, expert opinions about consciousness—in general, and regarding AI systems—are highly divergent. However, we believe that it is possible to make progress on the topic of AI consciousness despite this divergence. There are scientific theories of consciousness that enjoy significant empirical support and are compatible with a range of views about the metaphysics of consciousness. Although these theories are based largely on research on humans, they make claims about properties and functions associated with consciousness that are applicable to AI systems. We claim that using the tools these theories offer us is the best method currently available for assessing whether AI systems are likely to be conscious. In this report, we explain this method in detail, identify the tools offered by leading scientific theories and show how they can be used.

We are publishing this report in part because we take seriously the possibility that conscious AI systems could be built in the relatively near term—within the next few decades. Furthermore, whether or not conscious AI is a realistic prospect in the near term, the rise of large language model-based systems which are capable of imitating human conversation is likely to cause many people to believe that some AI systems are conscious. These prospects raise profound moral and social questions, for society as a whole, for those who interact with AI systems, and for the companies and individuals developing and deploying AI systems. Humanity will be better equipped to navigate these changes if we are better informed about the science of consciousness and its implications for AI. Our aim is to promote understanding of these topics by providing a mainstream, interdisciplinary perspective, which illustrates the degree to which questions about AI consciousness are scientifically tractable, and which may be a basis for future research.

In the remainder of this section, we outline the terminology, methods and assumptions which underlie this report.

### 1.1 Terminology

What do we mean by “conscious” in this report? To say that a person, animal or AI system is conscious is to say either that they are currently having a conscious experience or that they are capable of having conscious experiences. We use “consciousness” and cognate terms to refer to what is sometimes called “phenomenal consciousness” (Block 1995). Another synonym for “consciousness”, in our terminology, is “subjective experience”. This report is, therefore, about whether AI systems might be phenomenally conscious, or in other words, whether they might be capable of having conscious or subjective experiences.