Could a Large Language Model be Conscious? — part 4/4

Cleaned-text code points [42182, 52220). Bibliography omitted here; source.md includes it.

Where future LLMs and their extensions are concerned, things look quite different. It seems 
entirely possible that within the next decade, we’ll have robust systems with senses, 
embodiment, world models and self-models, recurrent processing, global workspace, and unified 
goals. (A multimodal system like Perceiver IO already arguably has senses, embodiment, a 
global workspace, and a form of recurrence, with the most obvious challenges for it being world-
models, self-models, and unified agency.). I think it wouldn’t be unreasonable to have a credence 
over 50 percent that we’ll have sophisticated LLM+ systems (that is, LLM+ systems with 
behavior that seems comparable to that of animals that we take to be conscious) with all of these 
properties within a decade. It also wouldn’t be unreasonable to have at least a 50 percent 
credence that if we develop sophisticated systems with all of these properties, they will be 
conscious. Those figures together would leave us with a credence of 25 percent or more. Again, 
you shouldn’t take the exact numbers too seriously, but this reasoning suggests that on 
mainstream assumptions, it’s reasonable to have a significant credence that we’ll have conscious 
LLM+s within a decade.30 
 
30 The philosopher Jonathan Birch (“The Search for Invertebrate Consciousness”, Nous 56:133-53, 2021) 
distinguishes approaches to animal consciousness that are “theory-heavy” (assume a complete theory), “theory-
neutral” (proceed without theoretical assumptions), and “theory-light” (proceed with weak theoretical assumptions). 
One can likewise take theory-heavy, theory-neutral, and theory-light approaches to AI consciousness. The approach 
to artificial consciousness that I have taken here is distinct from these three. It might be considered a theory-
balanced approach, one that takes into account the predictions of multiple theories, perhaps balancing one’s 
credences according to evidence for those theories or according to acceptance of those theories.  
 
One more precise form of the theory-balanced approach might use data about how widely accepted various theories 
are among experts to provide credences for those theories, and use those credences along with the various theories’ 
predictions to estimate probabilities for AI (or animal) consciousness. In a recent survey of researchers in the 
science of consciousness (Jolien C. Frankel et al, “An academic survey on theoretical foundations, common 
assumptions and the current state of consciousness science”, Neuroscience of Consciousness, issue 1, 2022), just 
over 50% of respondents indicated that they accept or find promising the global workspace theory of consciousness, 
while just under 50% indicated that they accept or find promising the local recurrence theory (which requires 
recurrent processing for consciousness). Figures for other theories include just over 50% for predictive processing 
theories (which do not make clear predictions for AI consciousness) and for higher-order theories (which require 
self-models for consciousness), and just under 50% for integrated information theory (which ascribes consciousness 
to many simple systems but requires recurrent processing for consciousness). Of course turning these figures into 
collective credences requires further work (e.g. in converting “accept” and “find promising” into credences), as does 
applying these credences along with theoretical predictions to derive collective credences about AI consciousness. 
Still, it seems not unreasonable to assign a collective credence above one in three for each of global workspace, 
recurrent processing, and self models as requirements for consciousness.. 
 
What about biology as a requirement? A 2020 survey of professional philosophers (David Bourget and David 
Chalmers, “Philosophers on Philosophy: The 2020 PhilPapers Survey”, Philosophers’ Imprint, 2023), around 3% 
accepted or leaned toward the view that current AI systems are conscious, with 82% rejecting or leaning against the 
view and 10% neutral. Around 39% accepted or leaned toward the view that future AI systems will be conscious, 
with 27% rejecting or leaning against the view and 29% neutral. (Around 5% rejected the questions in various ways, 
e.g. saying that there is no fact of the matter or that the question is too unclear to answer). The future-AI figures 
might tend to suggest a collective credence of at least one in three that biology is required for consciousness (albeit 
among philosophers rather than consciousness researchers). The two surveys have less information about unified 
agency and about sensory grounding as requirements for consciousness.

One way to approach this is via the “NeuroAI” challenge of matching the capacities of various 
non-human animals in virtually embodied systems.31 It’s arguable that even if we don’t reach 
human-level cognitive capacities in the next decade, we have a serious chance of reaching 
mouse-level capacities in an embodied system with world-models, recurrent processing, unified 
goals, and so on. If we reach that point, there would be a serious chance that those systems are 
conscious. Multiplying those chances gives us a significant chance of at least mouse-level 
consciousness with a decade.32 
We might see this as a ninth challenge: build multimodal models with mouse-level capacities. 
This would be a stepping stone toward mouse-level consciousness and eventually to human-level 
consciousness somewhere down the line. 
Of course there’s a lot we don’t understand here. One major gap in our understanding is that we 
don’t understand consciousness. That’s a hard problem, as they say. This yields a tenth 
challenge: develop better scientific and philosophical theories of consciousness. These theories 
have come a long way in the last few decades, but much more work is needed. 
Another major gap is that we don’t really understand what’s going on in these large language 
models. The project of interpreting machine learning systems has come a long way, but it also 
has a very long way to go. Interpretability yields an eleventh challenge: understand what’s going 
on inside LLMs. 
I summarize the challenges here, with four foundational challenges followed by seven 
engineering-oriented challenges, and a twelfth challenge in the form of a question. 
1. Evidence: Develop benchmarks for consciousness. 
2. Theory: Develop better scientific and philosophical theories of consciousness. 
3. Interpretability: Understand what’s happening inside an LLM. 
4. Ethics: Should we build conscious AI? 
 
5. Build rich perception-language-action models in virtual worlds. 
6. Build LLM+s with robust world models and self models. 
7. Build LLM+s with genuine memory and genuine recurrence. 
8. Build LLM+s with a global workspace. 
9. Build LLM+s that are unified agent models. 
10. Build LLM+s that describe non-trained features of consciousness. 
11. Build LLM+s with mouse-level capacities. 
 
12. If that’s not enough for conscious AI: What’s missing? 
 
31 Anthony Zador et al. Toward next-generation artificial intelligence: Catalyzing the NeuroAI revolution. 
arXiv:2210.08340, 2022. 
 32 At NeurIPS I said “fish-level capacities”. I’ve changed this to “mouse-level capacities” (probably a harder 
challenge in principle), in part because more people are confident that mice are conscious than that fish are 
conscious, and in part because there is so much more work on mouse cognition than fish cognition.

On the twelfth challenge: Suppose that in the next decade or two, we meet all the engineering 
challenges in a single system. Will we then have a conscious AI systems? Not everyone will 
agree that we do. But if someone disagrees, we can ask once again: what is the X that is missing? 
And could that X be built into an AI system? 
My conclusion is that within the next decade, even if we don’t have human-level artificial 
general intelligence, we may well have systems that are serious candidates for consciousness.  
There are many challenges on the path to consciousness in machine learning systems, but 
meeting those challenges yields a possible research program toward conscious AI. 
I’ll finish by reiterating the ethical challenge.33 I’m not asserting that we should pursue this 
research program. If you think conscious AI is desirable, the program can serve as a sort of 
roadmap for getting there. If you think conscious AI is something to avoid, then the program can 
highlight paths that are best avoided. I’d be especially cautious about creating agent models. That 
said, I think it’s likely that researchers will pursue many of the elements of this research 
program, whether or not they think of this as pursuing AI consciousness. It could be a disaster to 
stumble upon AI consciousness unknowingly and unreflectively. So I hope that making these 
possible paths explicit at least helps us to think about conscious AI reflectively and to handle 
these issues with care. 
Afterword  
How do things look now, in July 2023, eight months after I gave this lecture at the NeurIPS 
conference in late November 2022? While new systems such as GPT-4 still have many flaws, 
they are a significant advance along some of the dimensions discussed in this article. They 
certainly display more sophisticated conversational abilities. Where I said that GPT-3’s 
performance often seemed on a par with a sophisticated child, GPT-4’s performance often (not 
always) seems on a par with an knowledgeable young adult. There have also been advances in 
multimodal processing and in agent modeling, and to a lesser extent on the other dimensions that 
I have discussed. I don’t think these advances change my analysis in any fundamental way, but 
insofar as progress has been faster than expected, it is reasonable to shorten expected timelines. 
If that is right, my predictions toward the end of this article might even be somewhat 
conservative.  
 
 
 
33 This final paragraph is an addition to what I presented at the NeurIPS conference.