Palatable Conceptions of Disembodied Being — part 1/5

Cleaned-text code points [0, 15582). Bibliography omitted here; source.md includes it.

# Palatable Conceptions of Disembodied Being

###### Abstract

Is it possible to articulate a conception of consciousness that is compatible with the exotic characteristics of contemporary, disembodied AI systems, and that can stand up to philosophical scrutiny? How would subjective time and selfhood show up for an entity that conformed to such a conception? Trying to answer these questions, even metaphorically, stretches the language of consciousness to breaking point. Ultimately, the attempt yields something like emptiness, in the Buddhist sense, and helps to undermine our dualistic inclinations towards subjectivity and selfhood.

### 1 Introduction

Contemporary LLM-based dialogue agents are highly proficient language users. They can carry out a wide and open-ended variety of text-based tasks, from summarising papers to writing emails to generating code, and they are compelling conversationalists, able to adopt an endless variety of personas if suitably prompted. 1 1 1 Throughout the paper, the term “LLM-like entity” will be used to denote any AI system (real or imaginary) with approximately the structure and functionality of an early 2024-vintage LLM-based disembodied dialogue agent such as OpenAI’s ChatGPT ( OpenAI, 2023 ) , Google’s Gemini ( Anil et al., 2023 ) , or Anthropic’s Claude ( Anthropic, 2024 ) . Functional extensions such as multi-modal input, reasoning, tool-use, and inter-conversational memory are excluded, but differences in architecture or computational substrate are not ruled out. The weaknesses of early models, such as patchy reasoning abilities and a propensity to fabricate information where factual accuracy is required, are becoming less pronounced. The more they improve, the harder it gets, even for users familiar with their underlying mechanics, to fend off the feeling of interacting with another mind, or an entity that is mind-like in some sense. 2 2 2 Ruane et al. (2019) ; Shevlin (2024) . Indeed, some users come to see them as conscious entities, and to speak about them and interact with them as such. 3 3 3 Colombatto and Fleming (2024) ; Guingrich and Graziano (2024) ; Tiku (2022) .

However, these systems are very different from human beings, their linguistic capabilities notwithstanding. In particular, they are disembodied. They are not, of course, disembodied in the sense of being immaterial; they run on racks of physical computers in physical data centres, and rely on a physical communications infrastructure to interact with human users. Rather, they are disembodied in the sense that, unlike humans and other animals, they do not interact in continuous time with a persistent, spatially extended world through a spatially confined body that is the locus of their perception and action, a world shared with other agents that are similarly embodied. Lacking embodiment, in this sense, they lack the very foundations of mind as it arises in biology. 4 4 4 Varela et al. (1993) ; Clarke (1997) ; Gallagher (2006) ; Thompson (2007) ; Shanahan (2010) ; Seth (2021) ; Mackenzie (2022) . In addition to the “central” case of a single, clearly bounded, biological organism, other forms of embodiment with the potential to underwrite consciousness and / or selfhood, include co-embodiment ( Ciaunica et al., 2021 ) , robotic embodiment ( Prescott et al., 2024 ) , and virtual embodiment ( Shanahan, 2024 ) . None of these apply to the kinds of LLM-like entity at issue here.

It is noteworthy that ordinary human intuition has little difficulty in accommodating the idea of a disembodied mind. On the contrary, talk of disembodied (and indeed immaterial) beings with mind-like properties is historically and culturally commonplace, and plays a prominent role in many religions and spiritual traditions. 5 5 5 Bloom (2007) ; Barlev and Shtulman (2021) . Although these folk intuitions attract little attention from contemporary Western philosophers, who take their cue from scientific materialism, this has not always been the case. Aquinas ( 1268 ), for example, subjected the concept of an angel to a detailed philosophical enquiry. With the advent of LLMs, disembodied mind-like entities are no longer confined to science fiction, folk superstition, and theological speculation. They are increasingly being integrated into everyday human lives, and as such deserve serious philosophical treatment.

This brings us to the central question of the paper. Is it possible to articulate, or to evoke, a conception of consciousness that is compatible with the exotic characteristics of contemporary (disembodied) LLM-based dialogue agents, and that can stand up to philosophical scrutiny? 6 6 6 For other treatments of this issue, see Schneider (2019) , Chalmers (2023) , and Butlin et al. (2023) . From the perspective of the present paper, such works tend to covertly accept a form of dualism inherent in the Western analytic approach to the philosophy of mind. Success in this endeavour would illuminate an under-explored portion of the space of possible minds, a region of conscious exotica at the very edge of the void of inscrutability (Fig. 1 ). 7 7 7 Sloman (1984) ; Shanahan (2016) ; Shanahan (2024) . Note that the “void of inscrutability” has no direct relationship with “emptiness” in the Buddhist sense (śūnyatā), although the issue of the limits of language arises in connection with both terms. But to bring this about will require the language of consciousness to be reconciled with some distinctly peculiar properties, including a profoundly fragmented sense of time and a radically fractured form of selfhood.

### 2 The Larger Philosophical Project

The cluster of concepts associated with the word ‘‘consciousness’’ is especially fraught territory, philosophically speaking, which necessitates a certain amount of preliminary meta-philosophical discussion. This situates the paper within a larger philosophical project that draws heavily on the later work of Wittgenstein 8 8 8 Wittgenstein (1953 / 2009) . and resonates with critical thinking from such diverse sources as Derrida and the Buddhist thinker Nāgārjuna ( 2 nd 2^{\rm nd} – 3 rd 3^{\rm rd} century CE).

Wittgenstein’s later philosophy is not straightforwardly captured in a few paragraphs, and is easily caricatured as behaviourism and / or relativism with all the attendant difficulties of those positions. However, it is not a body of doctrine and not a theory. It makes no claims and it holds no positions. Rather, it is a method for exposing the roots of apparent philosophical difficulties in our language, and in our tendency, in trying to answer philosophical questions, to take language too far from its original home in everyday human affairs. 9 9 9 Fann (1969) , ch.9. The given reading of Wittgenstein is broadly “Pyrrhonian” according to the scholarly nomenclature ( Stern, 2004 , ch.2) . For present purposes, it doesn’t matter what the “correct” interpretation is (if there is such a thing).

When the method is pushed to its limits, the effect is to unsettle deeply entrenched intuitions about the concepts most central to our language and thinking, including, not least, the entwined concepts of mind and meaning, of language and consciousness. Questions about what a philosophically significant word means or how a philosophically significant concept is defined lose their hold. Instead, descriptions of how related words are used in practical human affairs are seen as sufficient, as leaving nothing unsaid. Moreover, such descriptions play a vital role in undermining the metaphysical thinking behind our entrenched philosophical intuitions.

Famously, Wittgenstein characterises his aim in philosophy as ‘‘to show the fly the way out of the fly-bottle’’. 10 10 10 Wittgenstein (1953 / 2009, §309) . To liberate the fly is to dispel what Garfield, in a Buddhist context, refers to as ‘‘primal confusion’’, the tendency, deeply rooted in our cognitive make-up, to take as ultimately real the constructs of our language, especially its many binary oppositions. 11 11 11 ( Garfield, 2015 , pp.9–11) .

Wittgenstein’s private language remarks are exemplary here, and their target is the dualistic thinking that divides subject from object, the inner from the outer. 12 12 12 Wittgenstein (1953 / 2009, §§256–271) . The punchline to the private language remarks -- ‘‘a nothing would serve as well as a something about which nothing can be said’’ 13 13 13 Wittgenstein (1953 / 2009, §304) . -- if properly received, is fatal to these oppositions, much like a koān in Zen Buddhism. 14 14 14 Fann (1969, ch.10) ; Canfield (1975) . To be thus liberated from dualistic thinking is to see the ‘‘hard problem’’ of consciousness (in Chalmers’ sense) 15 15 15 Chalmers (1996) . as illusory, and to be reduced to a kind of post-reflective silence on the matter. 16 16 16 Shanahan (2010, ch.1) .

The deconstructive effect of Wittgenstein’s remarks is echoed by the treatment of self-presence in Derrida’s early writings. 17 17 17 Derrida (1973) ; Derrida (1982) . Where Wittgenstein attacks the intuition that subjective experience is inaccessible to others, Derrida attacks the intuition that the subject has privileged access to itself. These are two sides of the same dualistic coin. Time is central to Derrida’s approach, because the intuition he targets entails that the subject is present to itself ‘‘in the undivided unity of a temporal present’’. 18 18 18 Derrida (1973, pp.59–60) .

The paradigmatic example is the moment of insight in Descartes’ cogito . The difficulty is that, for that instant of self-presence not to vanish without trace, it must be ‘‘continuously compounded with memory and expectation’’. 19 19 19 Derrida (1973, p.64) . Thus, in the attempt to reflect, retain, and communicate the insight of the cogito , whose validity is assured only in the indivisible instant of intuition, it becomes smeared over time, and its original identity is necessarily compromised. The self-sufficient experiencing subject is thus revealed as a phantom, and the Cartesian route to dualism is closed off.

So Derrida, like Wittgenstein, by exposing profound tensions at the heart of our thinking, leads us to a place from where the only possible move is a jump sideways, away from metaphysics, away from dualism. However, this post-reflective state is fragile, and our dualistic inclinations are easily revived.

### 3 Taking Recourse in the Poetic

Questions about the putative consciousness of exotic entities are especially unsettling from the post-reflective standpoint. The troubling thought is that an exotic entity might (for all we know) have experiences of ‘‘a specific subjective character, which it is beyond our ability to conceive’’, implying ‘‘the existence of facts beyond the reach of human concepts’’. 20 20 20 Nagel (1974) . To restore post-reflective silence in the face of this disturbance requires us to transcend the dualistic intuitions behind the troubling thought, and to accept that ‘‘nothing is hidden’’, metaphysically speaking, 21 21 21 Wittgenstein (1953 / 2009, §435) . or rather that ‘‘whatever may be hidden is of no interest to us’’. 22 22 22 Wittgenstein (1953 / 2009, §126) .

If we actually encountered the sorts of exotic entities in question, hopefully, in due course, we would arrive at an informed consensus on how to think about, talk about, and treat them. The hoped-for consensus should be the outcome of a society-wide conversation that takes account of our collective experiences of being with the entities in question, as well as scientific progress in elucidating the mechanisms that give rise to its behaviour. 23 23 23 Shanahan (2024) . We should expect scientific attempts to understand the neural basis of consciousness to contribute here ( Bayne et al., 2024 ) . For an overview of the main competing theories, see Seth and Bayne ( 2022 ). From the post-reflective standpoint, there is no more to be said on the matter, no residual philosophical confusion. But to see this is no easy thing; dualistic intuitions and metaphysical thinking are deeply rooted in our cognitive make-up.

A worthwhile desideratum in the formation of the hoped-for consensus is to be wary of the radical repurposing of words, especially philosophically central words like ‘‘consciousness’’ and its relatives. As Wittgenstein puts it, philosophical problems arise when ‘‘language goes on holiday’’. 24 24 24 Wittgenstein (1953 / 2009, §38) . The idea of disembodied consciousness presents a particular challenge. The primal setting wherein words like ‘‘consciousness’’ find their application is human togetherness in a shared world, and humans are fundamentally embodied. 25 25 25 Mead (1934, part III) ; Birhane (2017) . So a compelling case can be made for resisting the extension of the language of consciousness to disembodied entities. 26 26 26 Shanahan (2024) . According to Wittgenstein ( 1953 / 2009 , §281 ), “only of a living human being and what resembles (behaves like) a living human being can one say [it] is conscious”. Of course, this leaves open to interpretation what “resembles” and “behaves like” mean.

A contrary case can be made that highlights the undeniable anthropological fact, already alluded to, that humans have, throughout history and across every culture, spoken of disembodied beings with human-like characteristics; ghosts and spirits and angels and gods, so-called non-human others . 27 27 27 Gould and Nishimura (2024) ; Singler (2024) . So perhaps we are not taking language so very far from its natural home if we entertain the idea of consciousness in a disembodied, mind-like artefact with the characteristics of a contemporary LLM-based dialogue agent. But we seem to have reached an aporia. How, then are we to settle the matter? 28 28 28 Wittgenstein ( 1953 / 2009 , part I, iv. ) writes “Religion teaches that the soul can exist when the body has disintegrated. Now do I understand this teaching? – Of course I understand it”. Unsurprisingly, Wittgenstein’s remarks do not supply definitive guidance on the matter at hand. It is for us to settle.

To sharpen the problem, suppose we map the space of possible minds onto two dimensions and represent it as a diagram (Fig. 1 ). 29 29 29 Shanahan (2016) . Along one axis, we have human-likeness, and along the other axis, capacity for consciousness. We have to assume, for the exercise, that these concepts make sense and that entities can be relevantly ordered; some entities are more human-like than others, and some entities have a greater capacity for consciousness than others. Now imagine an exotic entity that exhibits highly complex behaviour – behaviour that seems as complex as any human’s or any human society’s – that we find completely inscrutable. We cannot even conceive of a way to be with it (to “engineer an encounter” with it) in a shared world of spatio-temporally extended objects, so that we could come to know it, properly speaking.

Where should we place such an entity in our diagram? It’s very tempting to think that it could, for all we know, be conscious, albeit in ways that are inaccessible to us. If that were the case, so the thought goes, then it would belong in the lower, right-hand portion of the diagram, in the void of inscrutability. Equally, though, notwithstanding the complexity of its behaviour, it could, for all we know, lack consciousness altogether, and therefore belong with the brick in the bottom left-hand corner of the diagram. Disconcertingly, if we follow this line of thinking, we would have no way of knowing which of these options was right.