Consciousness in Artificial Intelligence: Insights from the Science of Consciousness — part 11/14

Cleaned-text code points [154001, 169577). Bibliography omitted here; source.md includes it.

While Bayesian inference is intractable in general, several approximate methods exist. For instance, a recent class of models called Generative Flow Networks (GFlowNets) provide a framework for doing approximate Bayesian inference using modern deep neural networks (Bengio et al. 2021, Bengio et al. 2022), and can even be used to jointly train the inference model (first-order network) at the same time as the world-model (second-order network) (Hu et al. 2023, Zhang et al. 2023). This proposed GFlowNet architecture (like other approximate Bayesian inference methods) is again related to predictive processing theories of consciousness, which state that conscious percepts are the brain’s best guesses of the latent causes of its own sensory input (Seth 2021). An important difference in the PRM case, though, is that after the first-order inference system samples perceptual latent variables, they only become conscious if the second-order world model assigns them a high enough probability of being veridical, where either a hard or a soft threshold can be used to produce the “real” tag.

Note that the mechanism described above could also be used to produce internally-generated conscious experiences in the system. For instance, a top-down imagination module might first produce a goal (e.g., visualize an apple) and then drive neural activity accordingly in the first-order network’s sensory and perceptual layers. Following this, the second-order world model would assign a probability of veracity to this activity. If the imagination module is capable of producing coherent enough representations in the sensory and perceptual layers, then the second-order model would assign the signals a “real” tag in the same way as for externally-generated signals, and the imagined content would be consciously experienced. A similar mechanism may exist in the brain, as it has been shown that neural activity in sensory regions is similar during imagination and bottom-up perception and that the degree of similarity correlates with the vividness of the experience (Dijkstra 2019, Dijkstra 2023).

A final possibility is to learn a world model implicitly through adversarial methods. In particular, Generative Adversarial Networks (GANs) (Goodfellow et al. 2014) are a class of methods in which a generator network attempts to sample synthetic data and a discriminator attempts to differentiate real data from synthetic data produced by the generator. The generator is optimized to fool the discriminator into predicting its outputs as “real”, with the result that, if the networks used are powerful enough, the stable solution is for the generator to sample data from the true data distribution. The discriminator in GANs is comparable to the higher-order monitoring mechanism posited by PRM (Lau 2022). A GAN-based implementation of PRM might involve (a) a first-order perception network that produces perceptual representations given sensory data, (b) a generator network that inverts this process by first sampling synthetic perceptual representations and then producing synthetic sensory data given those percepts, and (c) a second-order network that is trained to discriminate between the real sensory-percept tuples produced by the first-order network and the synthetic ones produced by the generator (Gershman 2019). Like in the case of explicit world models, we can subsequently produce the “real” tag of PRM by applying a hard or a soft threshold to the discriminator’s output given representations from the first-order network.

It now remains for us to consider indicator HOT-3, which concerns the consumer of outputs from the metacognitive monitoring mechanism. Indicator HOT-3 states that this should be a “general belief-formation and action selection system” and that there should be a “strong disposition” to update it in accordance with the monitoring mechanism’s outputs. There are dual motivations for this condition: the function of the monitoring mechanism is to serve this consumer, which relies on the accuracy of first-order representations which are tagged as “real”; and the “force” of these inputs to the consumer is intended to explain the force and persistence of conscious experiences.

To implement the consumer, we can instantiate one or more higher-level networks that similarly take only perceptual representations with the “real” tag as input. There might be many ways to accomplish this, but one possibility is to use standard Transformer architectures for the higher-level networks with one small adaptation. In the Transformer, one or more query vectors would be produced by the higher-level network (intuitively representing the current computational goal) that would then attend to first-order perceptual representations depending on the values of their key vectors (intuitively representing the “kind” of percept or its “data type”). In a standard Transformer, each precept would then be modulated based on the similarity between its key and the higher-level network’s query, and then subsequently integrated into the downstream computation. If we wish to additionally make use of the “real” tags produced by the second-order networks, we need only multiply the query-key similarities by the value of the “real” tags for each corresponding percept, which would allow them to effectively serve as prior masks on which precepts may influence computations in the Transformer. Importantly, such an architecture is compatible with predictions made by PRM about the stubbornness of conscious percepts (Lau 2019): even if we are cognitively aware that a percept is inaccurate (e.g., a phantom pain, or a drug-induced hallucination), we still have a strong disposition to integrate it into our higher-level reasoning. With this Transformer-based architecture, conscious experiences cannot be reasoned away. This is because the higher-level network simply takes the percepts modulated by their “real” tags as input, and has no ability to change those “real” tags itself.

This proposal focuses on the second aspect of indicator HOT-3 (the “strong disposition”), rather than the idea that the consumer should be a general belief-formation and action selection system. Philosophers have various ideas about the features a system would need to have to count as forming beliefs, but one plausible necessary condition, which may be sufficient in this context, is that the system engages in instrumental reasoning leading to action selection (see section 2.4.5).

#### 3.1.4 Implementing AST

Two notable studies have implemented simple attention schemas in artificial systems. Wilterson and Graziano’s (2021) system used reinforcement learning on a three-layer neural network with 200 neurons per layer to learn to catch a ball falling on an unpredictable path. The primary input to this system was a noisy “visual” array showing the position of the ball. This array included an “attention spotlight”—a portion of the array with the noise removed—that the system could learn to move. A natural strategy for solving the task was thus for the system to learn to position the spotlight over the ball, so that noise in the visual array would not interfere with its ability to track and catch it. This was facilitated by a second input, an “attention schema” that represented the current position of the spotlight in the array. Wilterson and Graziano found that this system was much more successful in learning to perform the task when the attention schema was available, even though the spotlight remained when the schema was removed.

This very simple system did possess some part of indicator property AST-1, the attention schema, because it used a representation of an attention-like mechanism to control that mechanism, resulting in improved performance. However, this was not a predictive model, and the attention spotlight was substantially different from attention itself because it was a simple controllable feature of the input rather than an internal process with multiple degrees of freedom governing information flow through the system.

More recently, Liu et al. (2023) tested several different systems employing key-query-value attention on multi-agent reinforcement learning tasks. Their systems involved three main elements: multi-head attention layers, a recurrent neural network for “internal control” and a policy network. In the version that they took to most accurately implement an attention schema, the attention layers were applied to inputs to the system and sent information on to the policy network that generated actions, with the internal control network both learning to predict the behaviour of the attention layers and influencing this behaviour. This system performed better than the others, with different architectures made up of the same components, which were tested on the multi-agent RL tasks.

Compared to Wilterson and Graziano’s system, this system had the advantage of using a learnt predictive model of attention, rather than having an infallible representation provided as input. It was still used only for relatively simple tasks in 2D visual environments, and a caveat to this line of research is that attention in AI is not perfectly analogous to attention as understood by neuroscience (see Box 3). However, this system does illustrate a route to constructing systems that possess the attention schema indicator.

#### 3.1.5 Implementing agency and embodiment

The remaining indicators are agency and embodiment. Reinforcement learning is arguably sufficient for agency as we have characterised it (“learning from feedback and selecting outputs so as to pursue goals”), so meeting this part of indicator AE-1 may be very straightforward. Reinforcement learning is very widely used in current AI. To recap the argument for the claim that RL is sufficient for agency, the basic task in RL is for the system to learn to maximise cumulative reward in an environment in which its outputs affect not only the immediate reward it receives but also its opportunities for future reward. RL algorithms are, therefore, designed to be sensitive to the consequences of outputs, making the system more likely to repeat outputs that lead to greater reward over multiple time-steps. This means that there is a contrast between RL and other forms of machine learning. Like systems trained by supervised or self-supervised learning, RL systems gradually come to approximate a desired input-output function. However, in RL what makes this the desired function is that the outputs are conducive to goals that can only be achieved by extended episodes of interaction between the system and the environment. Crucially, RL systems learn these functions by virtue of their sensitivity to relationships between outputs, subsequent inputs, and rewards. Model-based RL systems learn models of these relationships, but model-free systems are also sensitive to them. So RL systems learn and select actions so as to pursue goals (Butlin 2022, 2023).

The second part of indicator AE-1 says that the probability of consciousness is raised to a greater degree if systems exhibit “flexible responsiveness to competing goals”. The example motivating this clause is an animal balancing multiple homeostatic drives: this requires prioritisation which is sensitive to shifting circumstances. Several computational approaches to this problem have been explored, including in the context of reinforcement learning (Keramati & Gutkin 2014, Juechems & Summerfield 2019, Andersson et al. 2019). One proposal is to use multiple modules that learn independently about how to maximise distinct reward functions, with each of these modules assigning scores to possible actions, and then to pick the action with the highest total score (Dulberg et al. 2023). In the homeostatic case, these reward functions might each correspond to a homeostatic drive, with large penalties for allowing any one of them to depart too far from a set point.

Our embodiment indicator, AE-2, states that systems should use output-input models (also known as forward models) for perception or control. There are specific uses of such models, which neuroscientists have identified in humans, that are particularly associated with embodiment. In perception, predictions of the sensory effects of the system’s own actions can be used to distinguish changes to sensory stimulation caused by these actions from those caused by events in the environment. Systems that engage in this form of inference implicitly or explicitly distinguish themselves from their environments. In motor control, forward models can be used for state estimation and feedback control, enabling rapid adjustments of complex effectors. It is important to distinguish these uses of output-input models from other uses, such as planning, which do not imply embodiment.

Learning output-input models for tasks related to perception and control is common, but there are few examples of current AI systems which meet these specific descriptions. For example, video prediction is a much-studied task (Oprea et al. 2020), and this includes predicting how visual input will develop conditional on the system’s outputs (Finn et al. 2016). But this is not sufficient for the kind of use in perception that we have described, which also includes making sense of exogenous changes which are not predictable from outputs. Output-input models are used in model-based reinforcement learning for control, facilitating successful control of quadrotor drones (Becker-Ehmck et al. 2020) and other robots (Wu et al. 2022). But it is uncertain whether the models are used in these cases for control-specific purposes rather than for planning, a topic that we explore further in section 3.2.2.

One good example of recent research which does employ a forward model for a purpose specific to embodied systems is the system described in Friedrich et al. (2021). In this system, Kalman filtering (Todorov & Jordan 2002) was used to combine a forward model with sensory inputs, which were subject to a time delay, in order to estimate the current state of the system in its environment. These estimates were then used in ongoing motor control, although only of relatively simple effectors in virtual environments.

### 3.2 Case Studies of Current Systems

#### 3.2.1 Case studies for GWT

In practice, it is not always immediately obvious whether a given AI system possesses one of the indicator properties. One reason for this is that we have not given absolutely precise definitions for each indicator. Another is that how deep learning systems work, including what they represent at intermediate layers, is often not transparent. In this section and section 3.2.2 we present case studies illustrating the use of the indicators to assess current AI systems. Here we focus on the GWT indicators (GWT-1 through GWT-4), and on two kinds of systems that are notable for different reasons. These systems are Transformer-based large language models (LLMs) such as GPT-3 (Brown et al. 2020), GPT-4 (OpenAI 2023), and LaMDA (Thoppilan et al. 2022), which are notable for their remarkable performance on natural language tasks and the public attention they have attracted; and Perceiver (Jaegle et al. 2021a) and Perceiver IO (Jaegle et al. 2021b), which are notable because Juliani et al. (2022) argue that they “implement a functioning global workspace”. Neither of these kinds of systems was designed to implement a global workspace, but there are arguments to be made that they each possess some of the GWT indicator properties.