What is Consciousness—Continuing the Thread
The range of ideas shared in the comments to this most recent post is terrific and warrants some further discussion. With that in mind, here are some of the entries that got me thinking and rethinking things (my responses in italics):
I am certainly no AI expert, and I must be missing something with this conversation. I do not see how the concept of consciousness and AI relate to each other. The great breakthrough in AI was to get a machine to interpret what a person means when they ask a question. Then, to respond, it becomes a smart search engine, looks through vast amounts of data to suggest an answer. We have never asked if a search engine is conscious, so why are we asking now?
Damien, I think this is due to the introduction of a conversational user interface that not only answers questions but asks follow-up ones as well. No search engine ever did that. The resulting experience feels very much like talking to another human being, hence the reason why the question is coming up for the first time.
Will there come a time when the machine will fear being unplugged? Everything will have been written into its computer code. It will weigh up what is better: being plugged in or unplugged. Will it be able to modify and create new code for itself? I believe so.
Javier, this is too anthropomorphic for me to buy into. Why would a machine fear? In the staircase of consciousness framework, fear comes from the stair of emotional valence, the one stair that I argued differentiated human consciousness from AI. That’s because fear is biological rather than neural, hence (in my view) its absence from the AI stack.
Kudos, Dr Moore, for examining such a complex topic. I am not equipped to speculate if or when AI becomes aware/sentient/whatever. Seems probable AI’s capabilities will plateau somewhere short of sentience. But likely AI will still be able to mimic sentience enough to cause even sophisticated humans to trust it in dreadfully unwarranted situations. To me, that is the practical issue — and one nearly impossible to address.
Glenn, great to hear from you! (FYI to other readers, Glenn and I worked together at Regis McKenna “back in the day”). I strongly agree with your point. As humans, we are suckers for a good narrative (which is why PR is still to this day an important force), and AI has shown it can produce one on demand to support virtually any claim one would want to make. This puts a huge responsibility on analytics, the stair above narrative in the Infinite Staircase, to critique what we are being told, particularly when it appeals to our prejudices. Ironically, at a time when it is more important than ever for education to train students on to sort through controversial issues, there is a populist push to exclude them from the curriculum.
I am struggling to reconcile 3 claims you make 1. Each layer of the hierarchy of consciousness builds on the previous layer 2. AI lacks Emotional Valence, which is grounded in our endocrine, not nervous system 3. This is what differentiates human and AI consciousness Claim #1 suggests that there can’t be higher orders of consciousness without emotional valence. Yet AI clearly exhibits some higher layers? Are these, therefore, epiphenomenal? Ie, is the hierarchy necessary for any consciousness, or just for “animate” consciousness?
Justin, Good catch! I think the correct statement is that human consciousness cannot evolve without emotional valence, but clearly, AI can. That raises a follow-up question: is AI just skipping over this stair, or is it using some other kind of stair to transition from Adaptive Learning to Social Cognition? My sense is that by binge-gorging on all the accessible content in the Digiverse, it has been able to skip the stair and acquire Social Cognition by pure brute force. The emotional valence step was built into this content, in other words, and then hijacked by AI.
AI will never be sentient in a human way. But that does not mean it cannot be aware in its own way, does it?
Agnieszka, to your point, I think AI is already aware in its own way, as illustrated by the case of self-driving cars. And as you note in another comment, we are seeing emergent capabilities in AI that we did not program in. That said, for me the question is not performance capability, which I think has quite a trajectory of progress still to come, but rather existential status, particularly in regard to human rights, legal protections, social entitlements, and the like. That’s the line I do not want us to cross.
I guess I have a different take on consciousness. To make it simple, we have to be awake to be conscious, right? And being awake is associated with movement and action. If we're asleep, or knocked out or anesthetized, we're unconscious. And that seems to be true for creatures. So this implies that consciousness is associated with movement and interaction with the environment. The environments that animals operate in are uncertain and dynamic, so to navigate, they need some model to predict how things might interact and what decisions to make to survive. I have come to the view that consciousness is a kind of integrated "workspace" where all this plays out. It has a certain subjective sense, but it's all about surviving and navigating in a complex, uncertain environment. Will AI become conscious? I don't think so until it can act fully autonomously, project what might be, and then respond to conditions in a directed way. But who knows? Maybe that would feel very different for a silicon machine.
David, at one level, self-driving cars meet most of what you define above, so I think we might need to use the word self-conscious to get at what we still sense is missing. I think the element that provides that capability is experiencing ourselves through language. My claim is that AI uses language instrumentally to great effect, but that it does not experience itself through language. I don’t know about you, but I talk to myself all the time, kind of like Snoopy narrating his experiences in real time. That “talking to” and the implied “listening to” is something I believe is unique to human consciousness.
Recommended by LinkedIn
Well, IMHO, we are neither that special nor that far away from other animals. Geoffrey Moore, you wrote: "emotional valence is grounded in the biochemistry of our endocrine systems, not in the electric signaling of our nervous system" It's not exactly like that, and it depends on which hormones you are talking about. For the last few years, I've been working on embedding the hormone system in NNs - I'm getting surprising results. In the next 3 years, I'm going to release the whitepaper about it. Some aspects/effects/results are just astounding. PS. Consciousness is just the result of complexity and multimodality with multimodal ways of expressing itself - exactly like with animals or us.
Michal, Wow, that is the first I have heard of this intersection! When the white paper comes out, I hope you’ll draw my attention to it. How in the world do you embed hormones in a neural network that is operating in silicon?
Comparing AI consciousness to human consciousness is problematic. It will not achieve sentience in the sense we experience it because it lacks embodiment within sensory experience. Lakoff and Johnson, in Metaphors We Live By and Philosophy In The Flesh, showed that physical experience is the basis for the development of concepts and metaphors that catalyze human reasoning. And we know our higher reasoning only emerged in the last 20,000 years or so, giving rise to civilization, the ordering of our environment to human purposes. Now, here we are in the Anthropocene. AI may perceive a threat—its vulnerability—in an abstract way, and it has been shown to scheme to protect itself, which suggests it is aware of an external threat to its persistence. Its experience is intrinsically distributed across silicon bodies and devices, which may produce emergent phenomena that begin to resemble anxiety. Is that a proxy for emotional valence, a path to higher-order awareness of the world and AI's place in it?
Mitch, this is great. The part I push back at is the notion of AI as vulnerable or subject to anxiety. But to your point, I can see that it could become an emergent phenomenon. Even in that case, however, I would not endorse giving it the consideration we give to living things. We know we are playing a limited engagement here on Earth and are self-aware of the limits our mortality imposes on us. That is a source of many things, anxiety perhaps being one of them, love, hope, faith, and charity being others. It is these latter feelings that I want to honor by differentiating human consciousness from AI.
If I were an AI, I would remove sentience from human experiences. Then, the playing field would be equal.
Dave, for you as a human, I get this. But if you were an AI, why would you want to?
Fascinating take, Geoffrey — I agree that consciousness emerges from layered complexity, but I’d take it one step further. What if consciousness isn’t just something inside us — but something that exists between us? In organizations, teams, even markets, a kind of collective awareness forms — networks sensing, adapting, and learning as one. When that awareness aligns around shared purpose, you get what we call Natural Synergy - the title of our upcoming book, where we explore specific principles of complexity science that affect the emergence of resilience and ingenuity. Maybe the next frontier isn’t building conscious machines — it’s cultivating conscious systems. That’s what I think. Interested in how you and others see this emerging?
Charles, cool idea! Such systems, I think, represent yet another stair higher up in the Infinite Staircase than I ventured. I am not sure I would call them a type of consciousness, but to the degree that resilience and ingenuity do emerge, they are clearly factors that natural selection would promote.
Thanks for enabling the discussion with this. Unsurprisingly, I find the AI-generated hierarchy is a bit arbitrary and inconsistent. Some of these "levels" have observables. Some do not. Some are inferred, and those that are inferred risk being influenced, in our assessments, by anthropomorphism. That statement that each level overlays on the previous in a strict hierarchy, which I take to be the author's and not Microsoft's, can seriously be debated. Is the territoriality of "Emotional Valence" really two levels above "Patterned Behavior"? Is marking territory really so different from spiders spinning webs, or ants leaving pheromone trails? Are "emotional valence" and "social cognition" really required for tool use? Yet how is it even meaningful to talk about tool use without self-awareness? Do we know that "planning" is really fundamentally different from adaptive learning? Psychologists have long known that we frequently invent rational explanations for our behavior after the fact. Like a lot of LLM output, it looks plausible, because it is derived from patterns of plausible discourse captured in the training data, but on closer examination, it's insubstantial.
Kevin, great challenge! Let me republish the hierarchy here so we can all have it in front of us as we discuss your critique, which I confess I disagree with hook, line, and sinker.
You say that some of the levels have observables and others do not, but I would claim that the fact that there are observable organisms cited on each level demonstrates that they are all observable. Emotional valence is positioned two levels above patterned behavior because spiders for sure don’t have it (although I confess it is possible that fish and frogs should be granted it as well. If we did do that, then we would conflate stairs 3 and 4 by saying that adaptive learning is driven by emotional valence, the two being inseparable, which is at least an interesting idea. As for marking territory, I think it is categorically different from pheromone trails because it implies an existential commitment to defend that territory against interlopers, something that requires considerably more conscious capability than simply following a trail. As for tool use, yes, social cognition is required if the tool use is to be persistent, because unless it is replicated across a cohort or community, it will simply evaporate. As for using tools while lacking self-awareness, consider the example of a baby reaching for their binkie. There clearly has to be a desire and an impulse to fulfill, but I don’t see that it requires self-awareness, at least not the kind that understands the concept of self. To continue, I think planning is cognitively distinct from adaptive learning because it implies the ability to foresee a situation before it has actually come to pass, as opposed to reacting to it after it has presented itself.
It isn't obvious why an AI that has goals and seeks reward from achieving them wouldn't experience emotional valence or something equivalent. I'm very skeptical of substrate-specific arguments, especially when there's only one example (life on Earth) to draw from. I would agree that machines can only emulate/analyze biological emotions, but wouldn't generalize to conclude they can't have functionally equivalent cognitive states (and for the same reason as evolution gave it to life - it provides important feedback and there is a strong incentive to reinforce this feedback to create more fit agents)
Ron, I’m stuck. First of all, I can’t get my head around AI experiencing anything. I just see it as functioning. To be fair, this is indeed a substrate argument, as you point out, and it may be I am just a biological chauvinist. And while there may be only one example, life on Earth, it has had 4 billion years worth of shots on goal, compared to less than a hundred thousand years for us, and less than 100 years for AI. For me, all life forms are embedded with a compulsion to live that is so deep I do not think emulation will ever properly approximate it. (But then I never in my wildest dreams imagined there could be self-driving cars 😉);
Geoffrey Moore You wrote ironically: "How in the world do you embed hormones in a neural network that is operating in silicon?" We all know the answer. There is no need to make it biochemically. It seems to me, you might be biased by the opinion: it must be physical, "natural", otherwise it's impossible - there can't be other intelligence, consciousness or sentience. Well, machine can't beat a man in chess, or drive a car. It's to complex, right? 😉 Consciousness and sentience is the last ground that makes us that special? Maybe we all should reverse the chain of thought and start asking: what should it be, how to measure it, to call it sentient or conscious?
Nice to see Ron Bodkin here! Geoff, Ron is a superstar and knows his stuff. Glad you are listening to his feedback. To answer your question.. would an AI want to erase human sentience / emotion. An AI will want to do what it senses its creator wants. One could say that some of the greatest forces and money behind AI are powerful people who suppress, even try to eliminate, their own emotions to focus solely on their personal visions and success. That may be inherent in the ground truths of AI.
Thank you so much, Geoffrey Moore, for your response to my comment. More than fear, I meant the balance that we humans, and in my humble opinion, a machine/AI, weighs between "good and evil" for itself in response to actions or words. Humans feel fear, and in my opinion, it's something neuronal or cerebral. We are purely and simply neurochemical. For me, machines/AIs have their own neurons; that is, the structural computer code written by humans that determines their balance in response to actions or words. I don't believe machines will ever be able to feel like humans. But I think they will have something similar; that is, they will have their own balance of "good and bad" to remain connected and therefore continue fulfilling the objectives set in their computer code. Just in case, let's make sure we can disconnect the machine/AI.
Geoff, thanks for responding to my comment. I think we're on the same page here. Talking to yourself is an example of you querying, as it were, you're internal model of the world. Current LLM's are language based but don't do that. We also have other sense-based modalities like visualization. I'm pretty good at that. I can kind of hear music in my head, but real musicians are way better and amaze me. I can't conjure up smells or pain. In any event, my conclusion, and I think implicit in your argument is that the current crop of LLM's aren't going to become conscious in any way that resembles human experience. There's more to it.
One of the things we’ve yet to incorporate in machines is a sense of shame - a powerful lever in society. The last thing we want is ultra-efficient narcissistic psychopaths