Consciousness in Artificial Intelligence: Insights from the Science of Consciousness — part 9/14

Cleaned-text code points [123164, 139057). Bibliography omitted here; source.md includes it.

To avoid this incompatibility, indicators concerning agency, perspective, or self-maintenance should be formulated “narrowly”, in ways that do not make reference to external conditions. For instance, rather than saying that a system is more likely to be conscious if it pursues goals by interacting with its environment, we can say that it is more likely to be conscious if it learns from feedback and selects actions in such a way as to pursue goals by interacting with its environment. Similarly, we can say that a system is embodied if it has a model of how its outputs affect its inputs, which represents some systematic effects, and uses this model in perception or control. There is a sense in which this system might misrepresent itself as an embodied agent, even if it has learnt this model, because it may be that the apparently systematic contingencies suggested by its past observations are mere coincidences. However, only embodiment conditions on which this is enough for consciousness are consistent with computational functionalism. Notably, this account of embodiment allows that systems controlling virtual avatars can count as embodied.

In this section, we have seen several indicators which could be added to our list. These include: being an agent; having flexible goals or values; being an intentional agent; having a perspective; having a body; and being self-maintaining. However, there are also reasons to exclude some of these. We already have an indicator requiring a relatively sophisticated form of agency with belief-like representations: indicator HOT-3, derived from PRM. An intentional agency indicator would be too similar to this one. And although a narrow formulation of a self-maintenance indicator is possible, which would be compatible with computational functionalism, this would be contrary to the spirit of the philosophical theories which emphasise self-maintenance. The ideas of being an agent and having flexible goals are closely related, and the main way in which flexible goals add to the case for consciousness is through the argument for centralisation and a common motivational currency, so we combine these in one indicator.

We, therefore, adopt the following two further indicators of consciousness:

AE-1 Agency: Learning from feedback and selecting outputs so as to pursue goals, especially where this involves flexible responsiveness to competing goals

AE-2 Embodiment: Modeling output-input contingencies, including some systematic effects, and using this model in perception or control

#### 2.4.6 Time and Recurrence

Human conscious experience seems to be highly integrated over time. We seem to undergo experiences that are themselves extended in time, and constitute conscious perception of temporally-extended phenomena, such as when we hear a bird’s song or watch it fly from one perch to another. What’s more, from the time we wake up each day we seem to experience a flow of several hours of continuous, integrated experience. Whether or not this feature of our experience should be explained in terms of memory (Dainton 2000, Phillips 2018b), our conscious experiences also seem to be deeply influenced by our memories. This integration might be seen as a reason to doubt that consciousness is possible in ANNs whose activity consists of temporally discrete forward passes; the point is particularly vivid if we imagine that there are long intervening periods (weeks, years) between passes. However, several responses can be made to this objection.

One response is that it is not obvious that consciousness is necessarily integrated over time. Patients with dense amnesia seem to have a series of brief, disjointed experiences (Wilson et al. 1995). These experiences may not be static, but it seems possible to imagine a conscious being that had only a succession of brief, static, discrete experiences. There is a question about what would make it the case that these were the experiences of a single subject, but it is not obvious either that integration over time is the only possible grounds for the persistence of the subject of conscious experience, or that there could not be conscious systems without a persisting subject.

Furthermore, it is possible that the apparent temporal continuity of human experience is illusory. Philosophers and scientists defend a range of views about how our experiences are generated over time (Dainton 2023), but these include the view that we undergo discrete, static experiences in rapid succession, generated by sampling from unconscious activity (VanRullen 2016, Herzog et al. 2020). Prosser (2016) calls this the “dynamic snapshot” view. The content of discrete experiences may be informed by unconscious retention of information, and be capable of representing change despite being static. For example, when watching a bird I might undergo a brief static visual experience representing not only that the bird is in a certain location, but that it is moving in a certain way. On this model, there may be no special integration between successive experiences, since the appearance of such integration can be explained by their being sampled at a fairly high frequency from smoothly-changing unconscious perceptual activity which in turn reflects smooth continuous change in the environment.

Despite these points, it does seem that algorithmic recurrence (i.e. recurrence as it is usually understood in machine learning; see section 2.1.3) is likely to be necessary for conscious experience with a human-like temporal character. For conscious experience to represent change or continuity in the environment in any way, information about the past must be retained and used to influence present processing. This is a further reason to take algorithmic recurrence to be an indicator of consciousness, as expressed in our indicator RPT-1. Integration over time, and, therefore, recurrent processing, is also emphasised by the midbrain and sensorimotor theories, the UAL framework and some PP theorists.

### 2.5 Indicators of Consciousness

Each of the theories and proposals which we have discussed in sections 2.1-2.4 is of some value in assessing whether a given AI system is likely to be conscious, or how likely it is that a conscious system could be built in the near future. They are each supported by evidence and arguments that have some force. We, the authors of this report, have varying opinions on the strength of the evidence and arguments supporting each theory, as well as varying background views about consciousness that influence our assessments of the likelihood of near-term AI consciousness. Here we summarise the findings of section 2 by giving a list of indicators of consciousness drawn from the theories and proposals we have discussed.

Our claim about these indicators is that they jointly amount to a rubric, informed by current views in the science of consciousness, for assessing the likelihood of consciousness in particular AI systems. Systems that have more of these features are better candidates for consciousness. Theories of consciousness make stronger claims than this, such as that some of these are necessary conditions for consciousness, and that combinations are jointly sufficient. We do not endorse these stronger claims, but we do claim that in using these indicators one should bear in mind how they relate to theories and to each other—some combinations of indicators will amount to more a compelling case for consciousness than others. The extent to which these indicators are individually probability-raising also varies, with some being plausibly necessary conditions that do not make consciousness significantly more likely in isolation (perhaps including RPT-1, GWT-1 and HOT-1).

## 3 Consciousness in AI

What do the findings of section 2 imply about consciousness in current and near-future AI systems? In this section, we address this question in two ways. First, in section 3.1 we discuss the indicator properties in turn, asking how they could be implemented in artificial systems. Second, in section 3.2 we examine several existing AI systems as case studies illustrating both how the indicators should be used, and how our method evaluates some current systems. We discuss large language models and the Perceiver architecture (Jaegle et al. 2021a, b) with a particular focus on global workspace theory, and consider whether any of a selection of recent systems—PaLM-E (Driess et al. 2023), a “virtual rodent” (Merel et al. 2019) and AdA (DeepMind Adaptive Agents Team 2023)—are embodied agents.

Reflecting on how to construct systems with the indicator properties, and on whether they are present in current systems, illustrates some crucial lessons of our work. One is that assessing whether a system possesses an indicator property typically involves some interpretation of the description of the property; the descriptions of indicator properties, like the theories from which they are drawn, contain ambiguities that possible implementations draw out. To describe indicator properties with so much precision that this kind of interpretation is not needed would mean going well beyond the claims made by scientific theories of consciousness, and these more precise claims would also tend to be less well-supported by the available empirical evidence. We hope that interdisciplinary research on consciousness, which brings together neuroscientists and AI researchers, will result in greater precision in theories of consciousness and the development of empirical methods that can provide evidence supporting more precise theories.

A second lesson from the discussion in this section is that in assessing AI systems for consciousness, it is not always sufficient to consider the system’s architecture, training and behaviour. For example, we may know that a system is a recurrent neural network and has been trained via RL to successfully control a virtual body to perform a task, and yet not know whether its performance relies on a learned model of output-input contingencies. In such a case, we may not know whether the system satisfies our embodiment condition. Interpretability methods, such as examining the information encoded in hidden layers (Olah et al. 2018), would be required to determine whether the system has acquired such a model in the course of training.

The third lesson is that, despite the challenges involved in applying theories of consciousness to AI, there is a strong case that most or all of the conditions for consciousness suggested by current computational theories can be met using existing techniques in AI. This is not to say that current AI systems are likely to be conscious—there is also the issue of whether they combine existing techniques in the right ways, and in any case, there is uncertainty about both computational functionalism and current theories—but it does suggest that conscious AI is not merely a remote possibility in the distant future. If it is possible at all to build conscious AI systems without radically new hardware, it may well be possible now.

### 3.1 Implementing Indicator Properties in AI

#### 3.1.1 Implementing RPT and PP

We begin our investigation of the indicators by discussing the two indicators taken from RPT, algorithmic recurrence and perceptual organisation, along with the PP indicator (i.e., indicators RPT-1, RPT-2 and PP-1). The reason for including PP here is that, as we will see, recent studies have found that using predictive coding in computer vision can facilitate processing that is more sensitive to global features of visual scenes, in contrast to the local-feature sensitivity of feedforward convolutional neural networks which perform well in classification tasks.

Algorithmic recurrence (RPT-1) is a feature of many deep learning architectures, including recurrent neural networks (RNNs), long short-term memory networks (LSTMs) and gated recurrent unit networks (GRUs) (LeCun et al. 2015). Building systems that possess indicator property RPT-1 is, therefore, straightforward. Although they are less widely used, there are also methods for implementing predictive coding (which is a form of algorithmic recurrence) in artificial systems (Lotter et al. 2017, Oord et al. 2019, Millidge et al. 2022). These systems meet indicator PP-1. Furthermore, recurrent neural networks trained on prediction tasks and optimised for energy efficiency self-organise into distinct populations of “prediction” and “error” units (Ali et al. 2022).

Turning to perceptual organisation (RPT-2), artificial vision models such as deep convolutional neural networks (DCNNs), are already both highly successful and often claimed to be good models of human vision (Kietzmann et al. 2019, Lindsay 2021, Mehrer et al. 2021, Zhuang et al. 2021). However, the claim that current systems are good models of human vision has recently been criticised (Bowers et al. 2022, Quilty-Dunn et al. 2022), and, more to the point, the human-level performance in visual object recognition that DCNNs achieve does not entail that they represent organised visual scenes. Bowers et al. (2022) and Quilty-Dunn et al. (2022) both cite evidence that DCNNs trained to classify objects are more sensitive to local shapes and textures than to global shapes, and tend to ignore relations between parts of objects, suggesting that they do not employ representations of integrated scenes. Conwell and Ullman (2022) found that the recent image generation model DALL-E 2 performed poorly when prompted to generate a scene with objects arranged in unfamiliar ways.

From the point of view of RPT, these points might be taken to show that the models in question are capable of categorising features of visual stimuli, a function which is said to be performed unconsciously in humans, but are not capable of further functions up to and including the generation of organised, integrated representations of visual scenes, some of which may require consciousness (Lamme 2020). However, other current systems, including predictive coding networks, do perform some of these further functions.

PredNet, a predictive coding network trained to predict the next frame of video inputs, is notable partly because success at this task seems to require representation of the objects that make up a scene and their spatial relations (Lotter et al. 2017). Furthermore, there is evidence that PredNet units respond to illusory contours in the Kanizsa illusion, the perception of which depends on inferring the presence of objects from the wider context (Lotter et al. 2020). Extending this finding, Pang et al. (2021) used a technique for adding feedback predictive coding connections to a feedforward DCNN and found further evidence that this predictive coding network, but not the initial feedforward model, was sensitive to the Kanizsa illusion. However, Lamme (2020) claims that the Kanizsa illusion requires only perceptual “inference”, not perceptual organisation as understood by RPT.

Beyond predictive coding, systems of other kinds have been developed specifically to represent objects and their relations in visual scenes. MONet, which uses a recurrent attention mechanism and a variational autoencoder to decompose and reconstruct scenes, is one example (Burgess et al. 2019). In this system, the attention network picks out individual objects in turn for identification and reconstruction by the variational autoencoder. Another system, the Object Scene Representation Transformer, is trained to predict the appearance of complex scenes from new angles, a task which (like video prediction) requires the representation of scenes as made up of objects in space (Sajjadi et al. 2022). Representing organised perceptual scenes is an active area of research in machine learning for which several methods have already been developed (Greff et al. 2020).

#### 3.1.2 Implementing GWT