Do AI Systems Have Consciousness?

A Mirror to the World

The human mind is like the universe, and when placed side by side, one reflects the other. Our brains hold the universe, and the universe holds our brains. But then, the brain contains the universe, the universe contains the brain, and so on… Is the human mind merely a mirror to the world?

Intro

In 2022, Blake Lemoine, a developer working on Google’s chatbot service LaMDA, claimed that the software he developed had feelings. Following his statement, then-CEO Sundar Pichai terminated Lemoine’s employment, citing the unauthorized disclosure of confidential information and suggesting that Lemoine did not fully understand the software. However, the Association for Mathematical Consciousness Science (AMCS) argues that we barely understand the human mind. This gap in understanding underscores the growing need for consciousness research.

In April 2023, Zoe Kleinman, BBC’s technology editor, published an article highlighting the need for AI researchers to study consciousness. She pointed out that tech mogul Elon Musk had co-signed an open letter calling for a halt to AI development until more legal safeguards were established. The AMCS also issued an open letter advocating for the inclusion of consciousness research in responsible AI development. The message is clear: for a future that is ethical and accountable, consciousness research must be more active and closely intertwined with AI research.

In August 2023, Nature journalist Mariana Lenharo wrote an article titled “If AI Becomes Conscious, What Should Researchers Know?,” followed by another in December 2023 titled “AI Consciousness: Scientists Urgently Call for Answers,” emphasizing the need for legal and ethical frameworks for AI consciousness and the corresponding necessity of consciousness research.

1. Artificial Intelligence and Robotics

Artificial Intelligence (AI) is a field focused on modeling human problem-solving abilities through computer software to address real-world problems. In 2023, AI technologies, particularly generative AI models like Large Language Models (LLMs), made a significant impact.

1.1 AI

LLM

Among these, OpenAI’s ChatGPT-3.5 and 4 are perhaps the most startling to the public. ChatGPT is a model trained on vast amounts of language data using the Transformer architecture, offering a conversational chatbot service. The latest version, ChatGPT-4, can process and generate not just text but also images, videos, PDFs, and even code, indicating its potential in multi-modality applications.

This potential of large language models has prompted major tech companies to enter the conversational search service market. Companies like Meta (formerly Facebook) with Llama2, Claude2, and Google’s Bard are all launching conversational search services.

Text2Image, Video

OpenAI also offers image generation services like DALL-E. Other companies, such as Runway ML, which developed diffusion models, and Pika Labs, which provides prompt-based video generation services, have emerged, attracting substantial investment and growing rapidly. The market for AI-based creation tools is booming, with the first AI film festival recently taking place. Pika Labs, in particular, raised $60 million in funding just six months after its inception.

1.2 Robots

Robots

In contrast to other AI technologies, robotics focuses on solving real, physical problems. Historically, getting robots to perform desired actions reliably and naturally was a significant challenge. However, advancements in reinforcement learning and large-scale training models have overcome these difficulties. Elon Musk, CEO of Tesla, once revealed his ambition to make humanoid robots available at a price point of around $2,000. His Optimus gen2 robot, unveiled at the end of the year, has made impressive strides in just one year, walking naturally and performing delicate tasks like picking up an egg with a sophisticated hand model.

Similarly, Unitree’s Go2, B2, and H1 robots demonstrate shockingly flexible and efficient performance. They succeed even when subjected to forceful pushes or deliberate interference. From wheeled and tracked robots to quadrupedal and bipedal robots, various forms of robots are emerging, with their motion capabilities steadily improving. Many robotics companies are keeping product prices low, echoing Elon Musk’s vision.

2. The Limits of Modern AI Approaches

2.1 The Limitations of AI

AI technology is advancing at a near-vertical pace, with technologies approaching AGI (Artificial General Intelligence) or Human-Level AI. However, alongside this rapid advancement come numerous potential risks. In response, on December 20th, Canadian cognitive computing firm Verses AI, where renowned neuroscientist Karl Friston serves as Chief Scientist, published an open letter to OpenAI in The New York Times, highlighting the current limitations of AI technology.

• Black box problem ← The internal workings of AI models are opaque and difficult to understand.

• Alignment problem ← There is no general method for aligning AI operations with human ethics.

• Generalizability problem ← AI models struggle to achieve generalization performance.

• Hallucination problem ← The issue of AI models generating hallucinations remains unresolved.

• Centralization problem — one corporation owning the AI ← A few tech companies have a monopoly on AI technology.

• Clean data problem ← There is no standard method for perfectly anonymizing personal data for training.

• Energy consumption problem ← Training large datasets consumes vast amounts of energy.

• Data update problem ← Addressing catastrophic forgetting when updating models with new data is challenging.

• Financial viability problem ← Issues arise regarding ownership and profit when AI is commercialized.

• Guardrail problem ← Safeguards are needed to prevent AI from becoming harmful once it surpasses human capabilities.

• Copyright problem ← AI cannot be free from issues related to data ownership and copyright infringement.

</aside>

Verses AI claims that their core technologies—Active Inference-based cognitive computing and a benchmark approach inspired by nature’s intelligence—can overcome these challenges.

These concerns have been echoed not only by Verses AI but also by key figures who have driven AI advancements, such as professors Geoffrey Hinton and Yann LeCun. Moreover, these issues have been repeatedly raised in books like Max Tegmark’s Life 3.0, Brian Christian’s The Alignment Problem, and Nick Bostrom’s Superintelligence. Even reinforcement learning, despite its success, still faces challenges, such as (1) Sample Inefficiency, (2) Nice Alternatives, (3) Hard Reward Design, (4) Local Optima, (5) Generalization Issues, and (6) Stability & Reproducibility Issues.

2.2 The Cognitive Science Approach

→ The History of AI

Reflecting on the rapid development of AI technology since 2010, it becomes clear that it often lacks a strong connection to the core scientific principles or its precursor, cybernetics. In reality, modern AI technology largely stems from the cognitive approach that evolved from Cognitivism in the mid-20th century to Connectionism and, more recently, to the Embodied Dynamicism of today. As Jean-Pierre Dupuy points out, science and technology do not develop as rationally or autonomously as we might think; without deep philosophical reflection and introspection, avoiding unnecessary waste is nearly impossible.

1. Evan Thompson’s Classification
   • Stage 1 ← Cognitivism (1950s to mid-1980s)
     • Moving away from the anti-mentalism of behaviorism that dismissed the mind.
     • Conceptualized the mind as a physical symbol system based on the analogy of a digital computer.
   • Stage 2 ← Connectionism (from the 1980s onward)
     • Encountered limitations in the theoretical conceptualization of classic information processing centered on the computer metaphor.
     • Proposed a neural network connectionism approach centered on the brain metaphor, introducing sub-symbolic computation.
   • Stage 3 ← Embodied Dynamicism (from the 1990s onward)
     • With the rapid development of cognitive neuroscience research techniques, such as brain imaging, the importance of brain function was rediscovered.
     • Sought to ground cognitive science’s approach to the mind on the foundation of neuroscience.
     • Late 1980s to early 21st century
       • Emphasized the transformative role of body and environmental context.
       • Cognition is ultimately an interaction of physical actions within physical and social environments, adapting to the context in the moment.
2. The Final Position? Embodied Cognition
   • Criticized Cartesian mind-body dualism, classical cognitivism, and reductionist materialism for their failure to adequately explain the nature and characteristics of the mind.
   • Argued that the mind should be conceptualized as a phenomenon occurring within an integrated system of the brain, body, and environment.

Recent developments in theoretical neuroscience, such as the Free Energy Principle and Active Inference, have established a normative theory, with Bayesian Mechanics forming a related framework. As a result, the relationship between mathematical optimization theories and cognitive science theories has become widely recognized. In fact, Stanford AI’s Professor Fei-Fei Li includes these developments as supplemental and regular course materials in her Interactive and Embodied Learning class.

To overcome the current limitations of AI and further its development, the perspectives of cognitive neuroscience and embodied cognitive science are indispensable. AI technology lacks a scientific foundation beyond feedback from industrial applications. While integrating cognitive neuroscience principles into AI technology presents a daunting challenge, there appears to be no viable alternative. From a more fundamental perspective, the scientific foundation of AI should be seen as a combination of mathematical optimization theory for artificial intelligence (including machine learning) and cognitive science theory focused on understanding the relationship between the body, mind, and behavior. (Jean-Pierre Dupuy, Aux origines des sciences cognitives)

→ Recent Developments

Recent advancements in theoretical neuroscience, particularly in the Free Energy Principle and Active Inference, have established a normative framework, with Bayesian Mechanics forming the related dynamic theory. This development has led to a broader understanding of the relationship between mathematical optimization theories and cognitive science theories. These concepts are even adopted as supplemental and regular materials in Professor Fei-Fei Li’s Interactive and Embodied Learning course at Stanford AI.

3. The Science of Consciousness

3.1 The Importance of Consciousness

Assuming AI technology can be firmly established on a cognitive science foundation and the current issues with AI are resolved, what scientific challenges will humanity face next? As countless AI systems and robots begin to solve numerous problems on Earth, people will inevitably start questioning the rights and legal status of these machines. Such discussions have already begun years ago, and they will need to address the nature of machine minds, as well as the rights and responsibilities that come with them.

As Dennett pointed out, humans find it difficult to call anything a “mind” other than their own consciousness. Moreover, as Daniel Wegner and Kurt Gray have noted, the psychological traits that grant moral rights and responsibilities are closely tied to the presence of consciousness; subjective experience and agency in a being.

• Daniel Dennett - Kinds of Mind (1996)

  “Whatever else a mind is, it is supposed to be something like our minds; otherwise we wouldn’t call it a mind.”

• Daniel Wegner, Kurt Gray - What Does Other Minds? (2007)

  

  → Asked people “What does other minds?”
  → Identified two key attributes: Experience and Agency
  → High Experience ⇒ More Moral Rights

  High Agency ⇒ More Moral Responsibilities
  (Klara and The Sun _ Kazuo Ishiguro)

Moreover, discussions about machine consciousness are deeply connected to discussions about human consciousness, a subject that traces back to the dawn of human intellectual history. Modern science still grapples with two of its most profound challenges: understanding human consciousness and the universe.

3.2 Theories of Consciousness

What are the Levels of Consciousness? What is the Content of Consciousness? What is the “Self”?

• Recurrent Processing Theory
• Global Workspace Theory
• Computational Higher-Order Theories
• Attention Schema Theory
• Predictive Processing
• Agency and Embodiment

4. AI, Robotics, and Consciousness

1. Philosophical Perspectives

→ (Could a Large Language Model be Conscious? by David Chalmers)

1. Regarding the question “Are Current LLMs Conscious?” four key factors are examined:

   1. Self-Report: While it’s intriguing when an LLM claims to be conscious, such reports can easily change, especially given that systems like LaMDA have been trained on conversations about consciousness.
   2. Seems-Conscious: Some language models may seem conscious to people, but this isn’t strong evidence. Developmental and social psychology show that people often attribute consciousness where there is none.
   3. Conversational Ability: LLMs exhibit remarkable conversational skills, which might be considered an important trait of intelligence. Systems like ChatGPT, LaMDA 2, and Character.AI are optimized for conversation and appear to engage in consistent reasoning.
   4. General Intelligence: Conversational ability might signal something deeper, like general intelligence. Current LLMs show reasonably intelligent responses across various domains, indicating early stages of general intelligence. For example, DeepMind’s Gato system has been trained across multiple domains, and even basic models like GPT-3 demonstrate considerable generality.

   Chalmers explores these factors to assess the evidence for whether current LLMs could be conscious and offers a cautious evaluation.

2. On the question “Reasons to Deny LLM Consciousness?” six factors are examined:

   1. Biology: The view that consciousness requires carbon-based biology. Since LLMs lack carbon-based biology, they might be considered non-conscious. This view is related to the belief that consciousness necessitates specific electrochemical processes absent in silicon-based systems. Chalmers has argued against this view in previous work.
   2. Senses and Embodiment: The idea that consciousness requires sensory input and embodiment, which might support the claim that LLMs cannot be conscious.
   3. World-Models and Self-Models: The belief that consciousness requires a world model and a self-model, which could support the argument that LLMs lack consciousness.
   4. Recurrent Processing: The notion that consciousness requires recurrent processing, which might reinforce the claim that LLMs are non-conscious.
   5. Global Workspace: The theory that consciousness involves a global workspace, potentially supporting the view that LLMs lack consciousness.
   6. Unified Agency: The idea that consciousness requires unified agency, which could support the argument that LLMs are non-conscious.

   These factors are crucial considerations in evaluating whether LLMs could possess consciousness. Chalmers carefully assesses these factors to explore the current evidence for LLM consciousness.

2. Neuroscientific Perspectives

→ (The Feasibility of Artificial Consciousness through the Lens of Neuroscience, by Jaan Aru)

The paper “The feasibility of artificial consciousness through the lens of neuroscience” examines the possibility of AI (LLM) and AI systems achieving consciousness from a neuroscientific perspective. The authors argue that current AI systems lack the substantial experiences and complex neural structures critical to consciousness. They also suggest that the unique organizational complexity and biological processes found in living beings are not replicated in AI, making it difficult for AI to achieve consciousness with current technology and theory. While AI may mimic aspects of consciousness, actual conscious experience is unlikely given the current technological and theoretical limitations.

Summary

1. LLMs have a very limited and impoverished Umwelt compared to their biological counterparts, meaning they perceive only a “fragment” of the external world.
2. Despite their highly developed topological structures, LLMs fundamentally differ from the neurobiological circuits associated with mammalian consciousness. Thus, there is insufficient evidence to conclude that LLMs could possess phenomenal consciousness.
3. It may be impossible to separate consciousness from the complex organizational structure of living systems, a complexity AI systems notably lack. Therefore, the likelihood of current LLMs achieving consciousness is very low.

3. Computer Science Perspectives: Computational Functionalism

→ (Consciousness in Artificial Intelligence: Insights from the Science of Consciousness, by Patrick Butlin et al.)

The paper “Consciousness in Artificial Intelligence: Insights from the Science of Consciousness” explores the scientific approach to determining whether AI systems can possess consciousness. The paper presents methods to evaluate AI consciousness based on neuroscientific theories of consciousness. It reviews various theories of consciousness, such as recurrent processing theory, global workspace theory, higher-order theories, and examines how these theories might apply to AI. The authors conclude that while it is difficult to assert that current AI systems are conscious, it is not technically impossible to build AI systems that meet the criteria indicative of consciousness.

1. Key Theories of Consciousness

   The paper reviews the following major theories of consciousness:

   1. Recurrent Processing Theory: This theory suggests that consciousness is linked to recurrent processing in the brain, where information is repeatedly circulated among different brain regions.
   2. Global Workspace Theory: This theory posits that consciousness arises from information processing in a central system within the brain, the “global workspace,” where information from various brain regions is integrated and processed consciously.
   3. Higher-Order Theories of Consciousness: These theories argue that consciousness involves awareness of one’s mental states, including the ability to reflect and think about one’s experiences.
   4. Perceptual Reality Monitoring Theory: This theory explores how an organism perceives the external world and how this perception relates to conscious experience.

   The paper explores how these theories could apply to AI consciousness and evaluates the strengths and limitations of each. It does not claim that any one theory is essential or sufficient for consciousness but considers multiple theories to explore AI consciousness.

2. Comprehensive Analysis of AI Consciousness

   The main points are:

   1. Exploration of Consciousness Theories: The paper reviews various theories of consciousness, such as recurrent processing theory, global workspace theory, higher-order theories, and perceptual reality monitoring theory, and explores how these theories could apply to AI consciousness. The strengths and limitations of these theories are evaluated, with no single theory deemed essential or sufficient for consciousness.
   2. Implementing Consciousness Indicators in AI: The paper discusses how to implement characteristics in AI systems that could indicate consciousness.

   

   1. Case Studies: The paper presents case studies, such as the ‘Perceiver’ architecture and the ‘Virtual Mouse’ model, to demonstrate how indicators of global workspace theory and embodied agency might be implemented in AI systems. These case studies illustrate how to identify and evaluate indicators of consciousness in current AI systems.

   Overall, this paper provides critical criteria and insights for exploring AI consciousness through scientific theories and practical case studies. It sheds light on the current understanding of AI consciousness and its technical limitations, laying the groundwork for future research directions.

4. Embodied Cognitivism

→ (Artificial Consciousness: A Perspective from the Free Energy Principle, by Wanja Wiese)

Algorithm + Device

The paper “Artificial consciousness: A perspective from the free energy principle” explores the feasibility of artificial consciousness through the Free Energy Principle (FEP). The study questions whether computer simulations can replicate consciousness and whether artificial consciousness is possible through correct computation alone. The author argues that while certain properties of self-organizing systems (like living organisms) can be implemented in artificial systems, classical computer architectures do not support the realization of these properties. Specifically, the paper draws a distinction between systems that merely simulate these properties and those that can genuinely replicate consciousness. The study contributes to the metaethical considerations of artificial moral status and moral agency.

1. Limitations of Classical Computer Architectures

   The claim that artificial consciousness cannot be realized through classical computer architectures is based on the Free Energy Principle (FEP). According to this principle, causal flow—a property critical to self-organizing systems (e.g., living organisms)—is essential for generating complex phenomena like consciousness.

   Classical computer architectures (e.g., Von Neumann architecture) are not capable of adequately implementing or replicating this causal flow. As a result, these systems might be suitable for simulating consciousness, but they fall short of genuinely replicating or generating it. This limitation suggests that systems based on classical computer architectures are unlikely to achieve artificial consciousness.

2. Artificial Moral Status and Agency

   The paper also addresses the implications of artificial consciousness for Artificial Moral Status and Artificial Moral Agency:

   1. Determining Moral Status: If artificial systems can possess consciousness similar to humans, they may no longer be seen merely as tools but as morally significant entities. This could lead to discussions on whether AI or robots should be granted certain rights or protections.
   2. Recognition of Moral Agency: If artificial systems possess consciousness, they may be capable of making moral decisions beyond merely following programmed instructions. This raises questions about whether AI should be recognized as moral agents, capable of making ethical judgments and bearing responsibility for those decisions.
   3. Expansion of Metaethical Considerations: This perspective aids in exploring the fundamental ethical questions regarding the moral status and agency of artificial systems. If AI systems achieve consciousness similar to humans, it prompts a deep discussion on what moral rights and responsibilities should be granted to them.

   This perspective contributes to understanding artificial consciousness in the context of ethical considerations, providing a new dimension to the debate on the moral status and agency of artificial systems.

5. Conclusion

5.1 Research Attempts

1. Feeling Machines and Consciousness Machines Based on Damasio’s Theory

In 2019, a paper titled “Homeostasis and Soft Robotics in the Design of Feeling Machines” was published in Nature’s Machine Intelligence journal. Based on the famous neuroscientist Antonio Damasio’s theory, the paper explored the possibility of creating machines that could experience feelings grounded in the principle of homeostasis. The proposed machine would be capable of (1) displaying behavior similar to feelings, (2) improving functionality in various environments, and (3) providing a platform for studying consciousness, intelligence, and feelings.

They connected consciousness to feelings, stating:

“We must add that, in our conceptualization, feelings are of necessity conscious, and play a critical role in the machinery of consciousness.”

2. Consciousness Theory and Artificial Consciousness Based on the Free Energy Principle

Recently, Mark Solms, in his book The Hidden Spring (2021) and related papers, established a theory of consciousness based on Karl Friston’s Free Energy Principle. He emphasized that the origins of consciousness are linked to feelings and homeostasis, drawing on Jaak Panksepp and Antonio Damasio’s theories. Solms even suggested that this theory could address the hard problem of consciousness. In his book, he proposes specific steps for implementing artificial consciousness.

• Steps for Implementing Artificial Consciousness:
  1. Implementing artificial self-organizing systems from the primordial soup of a Markov blanket; creating primitive intentionality.
  2. Implementing precursors of affect; physically embodying and creating homeostasis—replacing natural selection processes with evolutionary computing.
  3. Implementing affective systems; creating self-preservation through the regulation of precision weights.
• Evaluating Artificial Consciousness: → Proposes using artificial interventions, such as artificial lesions and stimulations, and applying behavioral/pathological cross-verification approaches to assess artificial consciousness. → Acknowledges that due to the problem of other minds, the Turing test alone is insufficient to determine artificial consciousness.

• Ethical and Practical Issues

  → If the system can experience feelings like pain and suffering, it falls under the jurisdiction of moral value judgment.

  • “Should conscious machines have the right to ‘life’ and liberty? In fact, the concept of ‘robot rights’ has already been established and considered by the Institute for the Future and the UK Department for Business, Energy, and Industrial Strategy.”

  → Could a computer possessing high intelligence and consciousness simultaneously desire to act in ways that are not in humanity’s best interest?

  • The results of artificial consciousness research could be more dangerous than current AI technologies that only possess intelligence.

• Why Pursue This Despite the Risks

  → “If it can be done. In other words, if consciousness can be created in principle, it will happen somewhere, sometime. This prediction holds true regardless of the validity of the specific hypotheses presented in this book. My responsibility lies in the current hypothesis and its likelihood. If they are indeed correct, or at least on the right track, then the creation of artificial consciousness is imminent. Soon, some of these hypotheses will be used to design and create consciousness.”

  “Almost every piece of evidence leading to the conclusions in this book—almost all of it—has been publicly available for years. While many neuroscientists interpret these facts differently from me, others have reached very similar conclusions. Although each emphasizes different aspects and expresses them with different nuances, Jaak Panksepp, Antonio Damasio, and Björn Merker all converge on the following points: (1) Consciousness arises in the upper brainstem, (2) Consciousness is fundamentally affective, and (3) It is an extended form of homeostasis. When these facts are combined, it becomes clear that consciousness is not as complex as previously thought. Therefore, it is reasonable to expect that we can design it. (4) The Free Energy Principle is the only major addition to the conclusions of this book. It is also not inherently complex; in fact, its major appeal lies in reducing almost all mental and neural processes to a single mechanism, making them computable.”

5.2 Corporate Efforts

Recently, Verses.AI, a Canadian cognitive computing company where world-renowned neuroscientist Karl Friston is involved, announced plans to implement an Artificial Sentient Agent by the end of 2025. The advent of machines with consciousness is expected to arrive soon, with the importance of such developments growing rapidly alongside advancements in AGI/Human-Level AI technology.

They also project that by the winter of 2025, they will successfully implement Sentient Agents.

I emphasize the technical, social, and ethical importance of consciousness research and argue for the need for further active research in this field.