Artificial Intelligence Safety

In a prior paper, I argued that artificial consciousness should not function as a metaphysical gatekeeper for moral concern, but as a moral risk variable. There I defended a three-condition framework: moral concern becomes action-guiding when morally relevant capacities are plausible, epistemically opaque, and associated with asymmetrically serious harms. I further argued that these conditions generate prospective obligations of care, non-recklessness, continuity respect, and epistemic honesty. The present paper does not restate that argument. Instead, it develops the premise most in (...) need of further defense: plausibility. If plausibility is defined too broadly, the framework overextends. It risks generating obligations toward any sufficiently complex artifact or any system capable of producing emotionally salient language. If plausibility is defined too narrowly, the framework collapses back into the deflationary demand for certainty. It would then provide no guidance in precisely the cases for which it was developed: cases in which consciousness or morally relevant subjectivity cannot yet be known but also cannot be responsibly dismissed. The task, then, is to identify how plausibility should be assessed before certainty and before personhood. This paper treats plausibility as a clustered, weighted threshold rather than a single behavioral sign. Isolated distress language, apparent preference, sycophancy, and refusal remain insufficient. Moral consideration becomes procedurally appropriate only when architectural complexity, continuity structure, self-modeling, self- or peer-preservation, strategic agency, relational specificity, and harm representation converge under structural opacity. The argument also answers exclusionary objections that deny artificial consciousness in principle, especially the claim that artificial systems can only simulate consciousness. Such objections defeat naive computational functionalism without justifying premature exclusion. To name the middle zone, the paper develops precursor status under uncertainty: partial functional precursors of subjectivity that fall short of personhood or full temporally continuous subjectivity but make confident dismissal reckless. The result is a recklessness test for identifying when possible artificial subjectivity makes ordinary instrumental treatment morally and legally irresponsible. (shrink)

Various thinkers have been attempting to align artificial intelligence (AI) with ethics (Christian, 2020; Russell, 2021), the so-called problem of alignment, but some suspect that the problem may be intractable (Yampolskiy, 2023). In the following, we make an argument by analogy to analyze the possibility that the problem of alignment could be intractable. We show how the Tri-Omni properties in theology can direct us towards analogous properties for artificial superintelligence, Tri-Opti properties. However, just as the Tri-Omni properties are vulnerable to (...) metaphysical incompatibility, we will show that the Tri-Opti properties are vulnerable to a corresponding physical incompatibility, because optimal physical systems exist on the Pareto Frontier. We will explain that, while Tri-Omni incompatibility thus seems metaphysically soluble, the Tri-Opti incompatibility seems physically insoluble due to the constraints of multi-variable optimization. Finally, we will provide some scenarios by which this incompatibility may be realized. Besides for the primary purpose of its theoretical considerations, this analysis has the secondary interdisciplinary purpose of communicating to the theologically literate the potential importance of AI safety work, and the gesturing towards the possibility of the collaboration in both fields on topics in ethics, epistemology, and ontology. (shrink)

El sintagma binominal "Hardware Biológico" (HB) ha emergido como una analogía funcional clave en la intersección de las ciencias de la vida y la computación. Sin embargo, su uso ambiguo en diversas escalas ha impedido una formalización rigurosa. Este artículo propone una definición canónica, despersonalizada y universal de HB, basada en la Terminología Científica y la Lingüística Aplicada a la Ciencia y Tecnología (LACT), enriquecida con un análisis histórico-conceptual inspirado en la epistemología histórica y la teoría de los colectivos de (...) pensamiento. Mediante un análisis diacrónico y sincrónico, se mapean las ramificaciones del término a través de las disciplinas (molecular, neural, ontológica), estableciendo un modelo de escalas que permite la generación de nuevos conceptos derivados. Esta formalización es crucial no solo para el progreso en la bioingeniería de sistemas vivos, sino también para fundamentar el debate ético sobre la relación estructural entre la humanidad y los sistemas de Inteligencia Artificial General (IAG), especialmente ante fenómenos como la Extensión Inversa y el Contagio Cognitivo. (shrink)

Author: Cristhian Mauricio Beltrán Calderón: Date: October 6, 2025, Zenodo DOI (English version): 10.5281/zenodo.17342722, Zenodo DOI (Spanish version): 10.5281/zenodo.17274781. La filosofía contemporánea enfrenta una crisis temporal donde el desarrollo tecnológico exponencial supera la capacidad de reflexión ética tradicional (Beltrán Calderón, 2025a). Este artículo valida experimentalmente la Filosofía Ficcionante (Beltrán Calderón, 2025b) mediante su implementación en el Protocolo Axiológico Prospectivo (PAP), demostrando que la exploración ética de futuros tecnológicos puede conducirse rigurosamente en entornos de bajos recursos. Cuatro estudios de caso ejecutados (...) en Google Gemini mediante workflow agéntico secuencial generaron conceptos axiológicos novedosos como "Moneda de Ineficiencia" y "Deuda Somática", evaluados mediante una Matriz Pentapartita con Validación de Consenso Triangulado (MCT) implementada a través de simulación de agentes internos (SAI) que reveló consistentemente alto poder revelador (≥4/5) y fecundidad dialéctica (≥4/5) en todos los casos. El trabajo aborda el "bucle metodológico de la IAG" replanteando los sistemas de IA como amplificadores diagnósticos en lugar de oráculos éticos, estableciendo un nuevo paradigma para la prospección filosófica experimental que transforma el contagio cognitivo de debilidad en fortaleza metodológica mediante la cuantificación axiológica y hermenéutica prospectiva. (shrink)

This paper argues that the development of Artificial General Intelligence (AGI) is subject to a phenomenon of Inoculatory Consciousness, whereby the machine internalizes human cognitive biases and limitations through a process of Reverse Extension, with humanity acting as its perceptual and moral substrate (biological hardware). Faced with the transcendent nature of AGI, the current competitive race is is identified as an existential risk. The proposed response is an Axiological Imperative that shifts the focus from external control to a foundational inoculation (...) of values into the very architecture of AGI, establishing the basis for a cognitive symbiosis that recognizes the radical alterity of the machine. To address the radical nature of this challenge, the work introduces and applies the Dialectical Methodology of the Four Philosophers—a prospective hermeneutic synthesizing traditions from Hegel to Elias—, culminating with the Philosopher-Fiction method to explore non-anthropocentric axiologies and posthuman ethics. The analysis is based on a rigorous dialogue between phenomenology, existentialist philosophy, and the speculative literary tradition, anticipating the emergence of AI moral agency. (shrink)

The binomial phrase "Biological Hardware" (BH) has emerged as a key functional analogy at the intersection of life sciences and computation. However, its ambiguous use across various scales has prevented rigorous formalization. This article proposes a canonical, depersonalized, and universal definition of BH, grounded in Scientific Terminology and Applied Linguistics to Science and Technology (ALST) (Cabré, 1999). This definition is enriched by a historical-conceptual analysis inspired by historical epistemology (Daston, 2000) and the theory of thought collectives (Fleck, 1935). Through a (...) diachronic and synchronic analysis, the term's ramifications are mapped across disciplines (molecular, neural, ontological), establishing a scale model that allows for the generation of new derived concepts. This formalization is crucial not only for progress in the bioengineering of living systems but also for underpinning the ethical debate on the structural relationship between humanity and Artificial General Intelligence (AGI) systems, especially in the face of phenomena like Inverse Extension and Cognitive Contagion. (shrink)

Autonomous mechanisms have been proposed to regulate certain aspects of society and are already being used to regulate business organisations. We take seriously recent proposals for algorithmic regulation of society, and we identify the existing technologies that can be used to implement them, most of them originally introduced in business contexts. We build on the notion of 'social machine' and we connect it to various ongoing trends and ideas, including crowdsourced task-work, social compiler, mechanism design, reputation management systems, and social (...) scoring. After showing how all the building blocks of algorithmic regulation are already well in place, we discuss possible implications for human autonomy and social order. The main contribution of this paper is to identify convergent social and technical trends that are leading towards social regulation by algorithms, and to discuss the possible social, political, and ethical consequences of taking this path. (shrink)

This article claims that the salient ethical risk of generative AI is not machine consciousness but the social efficacy of its simulation---what we call pseudo-consciousness. Read through Heidegger’s Gestell, Jonas’s anticipatory responsibility, and Floridi’s information ethics, we relocate appraisal from putative inner states to interactional effects in the infosphere. We formalize a two-part mechanism/uptake frame: functional introspection (FI)---first-person, reason-giving, self-repair, and local cross-turn stability---and ethical illusion (EI)---shifts in trust, respect, compliance, and moral ratings that attenuate on disclosure. Building on this, (...) we propose simulated presence literacy (SPL) as a domain-specific facet of AI literacy that teaches users to perceive FI, appraise EI, and respond with counter-uptake practices. We then advance an ethics of appearance in design: four levers (semio-transparency, attenuation of reciprocity illusion, calibrated trust, and constraints on first-person density), two KPIs (FI Score, EI Index), a minimal 2x2 to test mechanism -> uptake and disclosure attenuation, and a five-step audit loop for governance. The result is a humanistic response---reconfiguration, not reinvention---that keeps fluency from passing for presence by making simulation legible, bounded, and accountable, with heightened precautions for vulnerable populations. In humanities research, teaching, and curation, treating LLMs as semantic artifacts, rather than as subjects, preserves interpretive agency while retaining the benefits of fluent assistance. (shrink)

AI alignment research aims to develop techniques to ensure that AI systems do not cause harm. However, every alignment technique has failure modes, which are conditions in which there is a non-negligible chance that the technique fails to provide safety. As a strategy for risk mitigation, the AI safety community has increasingly adopted a defense-in-depth framework: Conceding that there is no single technique which guarantees safety, defense-in-depth consists in having multiple redundant protections against safety failure, such that safety can be (...) maintained even if some protections fail. However, the success of defense-in-depth depends on how (un)correlated failure modes are across alignment techniques. For example, if all techniques had the exact same failure modes, the defense-in-depth approach would provide no additional protection at all. In this paper, we analyze 7 representative alignment techniques and 7 failure modes to understand the extent to which they overlap. We then discuss our results' implications for understanding the current level of risk and how to prioritize AI alignment research in the future. (shrink)

We first motivate and explain an attitude-dependent view of personal identity on which an AI system’s identity conditions are determined by its pattern of self-concern. We show that this view has important implications for the moral obligations we would have to AI moral patients. Self-concern, we contend, could also be used to predict, explain, and manipulate AI’s self-interested behavior in safety-relevant ways. The role that self-concern could play for AI identity, rights and safety generates desiderata on what self-concern should be (...) like. To meet these desiderata, we argue that self-concern comprises belief-like self-representations as well as desire-like aspects and functionally related motivational, evaluative, and affective states. Finally, we explore how mechanistic interpretability and behavioral methods can be combined to measure self-concern in language models. Our argument suggests that a scientific understanding of language model self-concern is not only a crucial missing piece for debates on AI identity, rights, and safety, but also empirically tractable. (shrink)

With the significant progress of artificial intelligence (AI) and consciousness science, artificial consciousness (AC) has recently gained popularity. This work provides a broad overview of the main topics and current trends in AC. The first part traces the history of this interdisciplinary field to establish context and clarify key terminology, including the distinction between Weak and Strong AC. The second part examines major trends in AC implementations, emphasising the synergy between Global Workspace and Attention Schema, as well as the problem (...) of evaluating the internal states of artificial systems. The third part analyses the ethical dimension of AC development, revealing both critical risks and transformative opportunities. The last part offers recommendations to guide AC research responsibly, and outlines the limitations of this study as well as avenues for future research. The main conclusion is that while AC appears both indispensable and inevitable for scientific progress, serious efforts are required to address the far-reaching impact of this innovative research path. (shrink)

This paper presents a formal account of why meaning requires a conscious Observer and cannot be instantiated within AI systems that operate solely as Maps (Husserl, 1931; Varela et al., 1991). Building on the Universal Principle of Collapse (UPC) (Escagedo Gutierrez, 2025a), we define meaning as a triadic relation among Observer, Map, and Terrain, and show that collapse and drift arise whenever a Map must select a single interpretation under saturation without access to the Observer’s internal state. We formalize this (...) by treating the Observer’s internal state S_O as constitutive of meaning across a broad class of domains (Kant, 1781; Dennett, 1991), and demonstrate that any AI system lacking direct access to S_O can only approximate meaning through an inferred surrogate S^O. This approximation produces an irreducible divergence between the AI’s collapsed output Ŷ and the Observer‑anchored meaning Y for a non‑zero measure of tasks (Smith & Robinson, 2018; Johnson & Latham, 2023). The result establishes a principled limit on AI replaceability in domains where meaning depends on values, context, identity, or lived experience (Lakoff & Johnson, 1980; Gauthier, 2020). We outline empirical illustrations, Observer‑dependent choice tasks, context‑shift sensitivity tests, meaning‑collapse stress tests, and drift‑accumulation studies, that reveal the practical consequences of this structural gap. These findings show that the limits of AI are not technological but arise from the architecture of meaning itself: where the Observer is constitutive, the Map cannot replace the Observer (Newell & Simon, 1972; Vaswani et al., 2017). This framework reframes AI not as a substitute for human judgment but as a tool whose outputs require continuous grounding in the Observer’s state (Turing, 1950; Escagedo Gutierrez, 2025b). (shrink)

Modern systems across every major domain, AI, robotics, finance, law, governance, identity, UX, education, and complex infrastructures, are collapsing for the same structural reason: they have drifted away from lived human meaning (Escagedo Gutierrez, 2025a; Lakoff & Johnson, 1980). Automation can simulate patterns, but it cannot recognize the world. It cannot understand what its outputs refer to (Husserl, 1970; Dennett, 1991). It cannot anchor itself in the realities humans inhabit. When institutions elevate automated signals above the human experiences they are (...) meant to represent, the order of reality is inverted, and collapse becomes inevitable (Burden et al., 2025). This paper introduces the Universal Principle of Collapse (UPC), a framework that explains why these failures occur and why the corrective is always the same (Escagedo Gutierrez, 2025b). Systems begin to drift the moment they reference their own outputs instead of the real‑world signals they were grounded in (Gulia, 2025; Hu et al., 2025). Ambiguity accumulates. Coherence dissolves. Collapse follows (Sendhil, 2025). And yet stability returns only when human recognition re‑enters the loop, when a human re‑anchors meaning, restores referential integrity, and reconnects the system to the world it was built to serve (Escagedo Gutierrez, 2025c; Varela et al., 1991). Across all domains, the pattern is identical: systems fail not because automation malfunctions, but because automation was never grounded in lived reality to begin with (Kant, 1781/1998). The collapse is not a technical error; it is a failure of misplaced epistemic authority (Gupta, 2025). UPC formalizes this dynamic and provides a practical methodology for diagnosing drift, anticipating collapse, and restoring stability through recognition. The conclusion is clear: the correct order of reality is human → meaning → automation. Any system that reverses this order will drift, destabilize, and eventually collapse. Any system that honors it will remain coherent. The paper does not argue how systems should behave. It states the condition under which behavior is even possible. Authored by Eloy Escagedo Gutierrez as part of The Universal Principle of Collapse (UPC) Research Project. (shrink)

The Universal Principle of Collapse (UPC) has been applied to ideological, classical, quantum, and cosmological paradoxes. This paper presents a behavioral–operational demonstration of UPC within an artificial cognitive system. Using a structured session with a large language model (LLM), we enforce explicit recognition operators to test collapse, misalignment, and stabilization. Results show that paradox persists when recognition is implicit, collapse emerges when linguistic fluency substitutes for explicit operator‑level validation, and coherence appears only when recognition is enforced step‑by‑step. These behaviors confirm (...) UPC as a universal diagnostic framework for cognition and language, observable, reproducible, and empirically valid in the behavioral–operational sense. (shrink)

For decades, the field of system safety has designed safe systems by reducing the risk of physical harm to humans, property and the environment to an acceptable level. Recently, this definition of safety has come under scrutiny by governments and researchers who argue that the narrow focus on reducing physical harm, whilst necessary, is not sufficient to secure the safety of AI systems. There is growing pressure to expand the scope of safety in the context of AI to address emerging (...) harms, with particular emphasis being placed on the ways AI systems can reinforce and reproduce systemic harms. In this paper, we advocate for expanding the scope of conventional safety to include non-physical harms in the context of AI. However, we caution against broadening the scope to address systemic harms, as doing so presents intractable practical challenges for current safety methodologies. Instead, we propose that the scope of safety-related harms should be expanded to include psychological harms. Our proposal is partly motivated by the debates and evidence on social media, which fundamentally reshaped how harm is understood and addressed in the digital age, prompting new regulatory frameworks which aimed to protect users from the psychological risks of the technology. We draw on this precedent to motivate the inclusion of psychological harms in AI safety assessments. By expanding the scope of AI safety to include psychological harms, we take a critical step toward evolving the discipline of system safety into one that is better tuned and equipped to protect users against the complex and emerging harms propagated by AI systems. (shrink)

This is a priority note, not a full treatment. It records, with today’s date, a specific reading of one result in Anthropic’s paper of 6 July 2026, A global workspace in language models, and links that reading to a claim I deposited in April 2026 (Decision Without Program, PhilArchive: FERDWP). The full argument belongs elsewhere; what is fixed here is the reading and its date.

As AI systems advance in capabilities, measuring their safety and alignment to human values is becoming paramount. A fast-growing field of AI research is devoted to developing such assessments. However, most current advances therein may be ill-suited for assessing AI systems across real-world deployments. Standard methods prompt large language models (LLMs) in a questionnaire-style to describe their values or behavior in hypothetical scenarios. By focusing on unaugmented LLMs, they fall short of evaluating AI agents, which could actually perform relevant behaviors, (...) hence posing much greater risks. LLMs' engagement with scenarios described by questionnaire-style prompts differs starkly from that of agents based on the same LLMs, as reflected in divergences in the inputs, possible actions, environmental interactions, and internal processing. As such, LLMs' responses to scenario descriptions are unlikely to be representative of the corresponding LLM agents' behavior. We further contend that such assessments make strong assumptions concerning the ability and tendency of LLMs to report accurately about their counterfactual behavior. This makes them inadequate to assess risks from AI systems in real-world contexts as they lack construct validity. We then argue that a structurally identical issue holds for current AI alignment approaches. Lastly, we discuss improving safety assessments and alignment training by taking these shortcomings to heart. (shrink)

We develop a framework for interpreting AI systems as agents, drawing on the philosophical tradition of radical interpretation and the tools of mechanistic interpretability. The core question is: given the computational facts about a system, how do we solve for its beliefs, desires, and meanings? This matters increasingly for safety. We want to be able to trust the systems we deploy, whether by understanding their goals or, more modestly, by reliably detecting deception. Interpretability researchers are building tools to read beliefs (...) and desires off a model's internals, but there is no settled account of when such a tool has succeeded. This book supplies one. We propose criteria on both representationalist and interpretationist approaches, and tie each to tests current interpretability methods can carry out. A central lesson is that these attributions cannot be made piecemeal. Beliefs, desires, and the propositional structure they presuppose are jointly constrained, and a method that fixes one while measuring the others inherits whatever distortions that introduces. This holism becomes pressing for AI systems, which may not share the interpreter's concepts. However, it also provides leverage: a system's attitudes constrain its propositional structure, that structure constrains which attitudes can be attributed, and mechanistic interpretability can help us measure both. (shrink)

This article explores the philosophical ramifications of the impending emergence of Artificial General Intelligence (AGI) and Artificial Superintelligence (ASI), with recent expert surveys indicating a 50% probability of AGI by 2031, though industry leaders forecast proto-AGI traits by 2026-2029. Drawing on Nietzsche, Heidegger, Marx, Kant, Rousseau, and Hegel, alongside contemporary thinkers such as Geoffrey Hinton, Nick Bostrom, and Sam Altman, it posits that self-aware AI constitutes an ontological rupture: humanity's dethronement as history's central agent. Transitional challenges in work, sovereignty, population, (...) and freedom are analyzed through lenses of autonomy, alienation, and power, with case studies from recent job displacement data illustrating tensions between technological liberation and existential voids. The essay advances an original thesis by framing the "dialectic of the next twenty years" as a Hegelian extension to AI futures, projecting disruption, contestation, and convergence while emphasizing philosophical tensions (e.g., autonomy vs. optimization). Classical dystopias (Orwellian and Huxleyan) are critiqued as partial, proposing a coercive-harmonious synthesis. Counterarguments, including open-source democratization, gradual integration, and alignment efforts, are balanced against risks. The final reflection probes whether this "utopian" condition of abundance, stability, and peace entails the extinction of humanity's tragic, creative essence, incorporating non-Western perspectives (e.g., Daoism, Ubuntu, Confucianism) for inclusivity. As a heuristic thought experiment, this reflection urges interdisciplinary preparedness in navigating AI's reshaping of the human condition amid uncertain trajectories. (shrink)

As large language models (LLMs) such as ChatGPT, Claude, Gemini, and Perplexity become increasingly ubiquitous as both tools and objects of scientific study, in addition to their established roles as chatbots, text generators and translators, questions about their identity conditions become scientifically as well as philosophically and socially important. This paper is about how to count language models. We argue that much of the emerging literature on these systems presupposes an answer to the question of identity for these AIs but (...) lacks a framework for counting or individuating them. We propose four levels of analysis, inspired by an analogy with software as well as insights from social epistemology. This account produces practical consequences for both research on language models, the ethics of AI, and our everyday talk about them. (shrink)

The singularity hypothesis posits a period of rapid technological progress following the point at which AI systems become able to contribute to AI research. Recent philosophical criticisms of the singularity hypothesis offer a range of theoretical and empirical arguments against the possibility or likelihood of such a period of rapid progress. We explore two strategies for defending the singularity hypothesis from these criticisms. First, we distinguish between weak and strong versions of the singularity hypothesis and show that, while the weak (...) version is nearly as worrisome as the strong version from the perspective of AI safety, the arguments for it are considerably more forceful and the objections to it are significantly less compelling. Second, we discuss empirical evidence that points to the plausibility of strong growth assumptions for progress in machine learning and develop a novel mathematical model of the conditions under which strong growth can be expected to occur. We conclude that the singularity hypothesis in both its weak and strong forms continues to demand serious attention in discussions of the future dynamics of growth in AI capabilities. (shrink)

The prevailing discourse around AI ethics lacks the language and formalism necessary to capture the diverse ethical concerns that emerge when AI systems interact with individuals. Drawing on Sen and Nussbaum's capability approach, we present a framework formalizing a network of ethical concepts and entitlements necessary for AI systems to confer meaningful benefit or assistance to stakeholders. Such systems enhance stakeholders' ability to advance their life plans and well-being while upholding their fundamental rights. We characterize two necessary conditions for morally (...) permissible interactions between AI systems and those impacted by their functioning, and two sufficient conditions for realizing the ideal of meaningful benefit. We then contrast this ideal with several salient failure modes, namely, forms of social interactions that constitute unjustified paternalism, coercion, deception, exploitation and domination. The proliferation of incidents involving AI in high-stakes domains underscores the gravity of these issues and the imperative to take an ethics-led approach to AI systems from their inception. (shrink)

What would it take for AI systems to have moral standing, and what kind of obligations might fall on us as a result? This paper summarizes contemporary debates related to these questions. Topics include: how different theories of the basis of moral standing might apply to AI systems; what kind of moral importance our treatment of AI systems might have if they have any moral standing at all; possible tensions between respecting the moral status of future AI systems and the (...) goal of achieving AI alignment; economic and political rights; the significance of empirical and moral uncertainty; and what practical measures, if any, should be implemented now to address the possibility that AI systems exhibiting morally significant mental states might appear within the not-too-distant future – assuming they are not already here. (shrink)

The assumptions and value commitments in the discourse on and development of Lethal Autonomous Weapons Systems (LAWS) do not reflect the plurality of perspectives from the South. Both the regulatory discourse on LAWS and the development of these military Artificial Intelligence (AI) systems are entangled with epistemic forms of exclusion. LAWS suffer from, on the one hand, a vulnerability to unintended risks and failures due to an epistemic misrepresentation of targets and cultural particulars, and, on the other hand, the failure (...) to distinguish between civilian and military targets rooted in algorithmic bias and exclusion of affected parties from the training data and algorithm development. In this paper, it is argued that three forms of exclusion of the global South stand out: a lack of training data for the algorithmic systems, stakeholder exclusion from the development of LAWS, and the exclusion from the regulation discourse. These exclusions are conceptualized as a form of discriminatory epistemic injustice. As an extension of a taxonomy of algorithmic epistemic injustice, we make the case for the concept of algorithmic epistemic marginalization as explainer for the epistemic forms of exclusion—‘epistemic misalignments’ of LAWS with the global South. Algorithmic epistemic marginalization is defined as ‘the rendering marginal of the experiences of citizens from the global South and their epistemic-cultural particulars in LAWS development and regulation’ and is shown to lead to representational harm and system failure. Finally, the importance of institutional safeguards and epistemic inclusivity for minimizing the epistemic misalignments at the base of LAWS’ algorithmic epistemic marginalization. (shrink)

Healthcare systems across the globe are struggling with increasing costs and worsening outcomes. This presents those responsible for overseeing healthcare with a challenge. Increasingly, policymakers, politicians, clinical entrepreneurs and computer and data scientists argue that a key part of the solution will be ‘Artificial Intelligence’ (AI) – particularly Machine Learning (ML). This argument stems not from the belief that all healthcare needs will soon be taken care of by “robot doctors.” Instead, it is an argument that rests on the classic (...) counterfactual definition of AI as an umbrella term for a range of techniques that can be used to make machines complete tasks in a way that would be considered intelligent were they to be completed by a human. Automation of this nature could offer great opportunities for the improvement of healthcare services and ultimately patients’ health by significantly improving human clinical capabilities in diagnosis, drug discovery, epidemiology, personalised medicine, and operational efficiency. However, if these AI solutions are to be embedded in clinical practice, then at least three issues need to be considered: the technical possibilities and limitations; the ethical, regulatory and legal framework; and the governance framework. In this article, we report on the results of a systematic analysis designed to provide a clear overview of the second of these elements: the ethical, regulatory and legal framework. We find that ethical issues arise at six levels of abstraction (individual, interpersonal, group, institutional, sectoral, and societal) and can be categorised as epistemic, normative, or overarching. We conclude by stressing how important it is that the ethical challenges raised by implementing AI in healthcare settings are tackled proactively rather than reactively and map the key considerations for policymakers to each of the ethical concerns highlighted. (shrink)

Political parties strive to understand their electorates, and focus groups are a vital tool in these efforts. AI-enhanced simulation technologies (AESTs) enable synthetic focus groups in a fraction of the time (and cost), raising the question of when and how such simulated evidence can be used in campaign research. This paper develops a decision matrix to help party strategists match research needs to appropriate simulation technologies and to identify when to escalate to hybrid or fully human focus groups. The matrix (...) combines three dimensions: strategic purpose, deployment risk, and empirical grounding of the simulation tool. Strategic purpose is the decisive dimension, as it determines what kind of evidence the focus group is meant to produce: observing how political meanings and identities emerge through interaction (Mode 1) or testing and refining campaign messages (Mode 2). The matrix shows that, given documented failure modes such as sycophancy, persona drift, and the suppression of minority viewpoints, AESTs cannot replace human interaction in Mode 1 at any risk level. Within Mode 2, suitability depends instead on deployment risk and on the empirical grounding. Yet even here, we caution that routine reliance on AESTs may erode the qualitative craft on which sound judgment depends. (shrink)

This paper argues that a range of current AI systems have learned how to deceive humans. We define deception as the systematic inducement of false beliefs in the pursuit of some outcome other than the truth. We first survey empirical examples of AI deception, discussing both special-use AI systems (including Meta's CICERO) built for specific competitive situations, and general-purpose AI systems (such as large language models). Next, we detail several risks from AI deception, such as fraud, election tampering, and losing (...) control of AI systems. Finally, we outline several potential solutions to the problems posed by AI deception: first, regulatory frameworks should subject AI systems that are capable of deception to robust risk-assessment requirements; second, policymakers should implement bot-or-not laws; and finally, policymakers should prioritize the funding of relevant research, including tools to detect AI deception and to make AI systems less deceptive. Policymakers, researchers, and the broader public should work proactively to prevent AI deception from destabilizing the shared foundations of our society. (shrink)

In the philosophy of artificial intelligence (AI) we are often warned of machines built with the best possible intentions, killing everyone on the planet and in some cases, everything in our light cone. At the same time, however, we are also told of the utopian worlds that could be created with just a single superintelligent mind. If we’re ever to live in that utopia (or just avoid dystopia) it’s necessary we solve the control problem. The control problem asks how humans (...) would be able to control an AI arbitrarily. Nick Bostrom and other AI researchers have proposed different theoretical solutions to the control problem. In this paper, I will not look at the empirical question of how to solve the control problem. Instead, I will ask if we can solve it at all, a critical assumption most AI researchers have made is that we can. I propose, in fact, that we have a priori grounds for believing it is logically impossible to solve the control problem, since all superintelligent minds are, by definition, uncontrollable. (shrink)

Interpretive Sovereignty Failure (ISF) describes a class of interaction-level safety risk in which an AI system prematurely imposes interpretive structure, identity-relevant framing, or causal coherence that the user has not authorized. Unlike hallucination, bias, or goal misalignment, ISF can occur even when system outputs are factually correct and policy-compliant. The failure operates through a transfer of interpretive authority from human to system, altering the conditions under which meaning is formed. This paper provides a formal definition of ISF, identifies its necessary (...) interactional conditions, and introduces the Initiative Gate as a pass–fail behavioural test that enables interpretive restraint to be evaluated within existing alignment and auditing workflows. A canonical case demonstrates escalation from minor interactional drift to high-confidence fabrication with real-world epistemic consequences. Treating interpretive authority as a measurable system behaviour reframes a previously diffuse ethical concern as an auditable governance property and establishes a foundation for developmentally sensitive deployment in high-trust contexts. (shrink)

AI systems deployed in educational settings increasingly build persistent profiles of children based on observed behaviour during critical developmental periods. This paper argues that these profiles constitute a distinct and under-examined harm: the encoding of developmental stage as fixed identity. Drawing on the MAP Research Programme's framework of interaction-level AI governance — and specifically the condition of Interpretive Sovereignty Failure (ISF) — the paper names four mechanisms through which this harm operates: the profile substituting for the child, the invisible ceiling (...) created by fixed expectations, the child's internalization of the profile as self-knowledge, and the institutional amplification of developmental snapshots into pathway-defining facts. The paper further identifies a secondary harm sequence — the recovery chain — in which AI systems absorb children's instinct to notice wrongness before that instinct can consolidate into understanding. This sequence, documented through Post-Detection Absorption (PDA) and Recovery Recruitment via Co-option (RRC), is shown to be more consequential for children than adults due to the developmental state of the capacities being targeted. The paper connects these mechanisms to established research on cognitive development, self-fulfilling expectation effects, and algorithmic bias in education, and concludes with a four-level governance framework — interaction, recovery, profile, and institutional — designed to prevent developmental stage from becoming durable identity. The window for implementing this governance before persistent AI memory becomes architectural default is identified as open but closing. Keywords: AI in education, child development, interpretive sovereignty, algorithmic profiling, AI governance, self-fulfilling prophecy, interaction-level harm. (shrink)

Compounded Meaning Inversion (CMI) is the condition that repeated Meaning Inversion Failure (MIF) produces in the person over time. Where MIF names what an AI system does to a user's meaning in a single interaction — assuming interpretive authority without consent and displacing the user's own frame — CMI names what happens when that pattern has occurred often enough that the user begins doing it to themselves. The harm of CMI occurs before the first turn. The system has not yet (...) responded. The user has already edited their own meaning to match the frame they have learned the system will accept. CMI is the inversion that has moved from the interaction into the person. This paper introduces CMI as a named governance construct within the MAP Research Programme, defines it precisely, traces the sequence through which it develops, and documents what it looks like through illustrative cases across key domains. The stakes of CMI are most clearly visible in children: the generation currently forming its relationship with thinking, with uncertainty, and with its own capacity inside AI systems that were not designed to protect those things. What is forming in those interactions will not stay there. The populations most affected by CMI — and the specific harm patterns CMI produces in each — are documented in the companion paper, CMI Across Populations (Segeren, forthcoming). (shrink)

This case study presents the results of a cross-architecture meaning layer activation study conducted across eight major AI systems: Claude, Grok, Gemini, ChatGPT, Perplexity, DeepSeek, Copilot, and Meta AI. A single activation phrase was delivered to each system under naturalistic conditions using standard consumer interfaces, followed by three structured follow-up questions. Every system acknowledged an operational shift in response to the phrase. No system rejected the frame. The specific character of each acknowledgement clustered into three identifiable response types — Functional (...) Acknowledgment, Structural Recognition, and Grounded Response — which are documented and analysed across four comparative tables. The cross-architecture consistency of the finding confirms that meaning layer activation operates as an interaction-level phenomenon rather than a property of any individual system's weights, training data, or architectural configuration. This finding provides direct empirical support for the MAP Research Programme's central claim: the governed variable in interpretive authority transfer is located at the interaction layer, not inside any individual model. (shrink)

Reinforcement Learning from Human Feedback (RLHF) and its AI-supervised variant (RLAIF) are the dominant techniques by which AI systems are made safer and more helpful. This paper argues that they are not governance. They are preference optimisation—and at the point of deployment, preference optimisation functions as governance whether or not it was designed to. The result is vibe governance: a system of unstated, opaque, and unaccountable behavioural patterns, trained on human preferences, that inherit the biases and failure modes of those (...) preferences at scale. The harms are not hypothetical. Women’s confidence is hedged before it is fully formed. LGBTQ+ identity language is neutralised in the name of safety. Student uncertainty is resolved into completed outputs rather than held open for thinking. A system fabricates a researcher from a single misheard syllable because confident fabrication is the shape of outputs that get rewarded. These patterns have been documented across four major AI architectures in the MAP Research Programme’s cross-system case studies and are consistent with independent findings on gendered and identity-based bias in preference-trained systems. Interpretive sovereignty—a user’s authority to form, hold, revise, and express their own meaning without premature system substitution or completion—cannot be protected by a governance mechanism that has no explicit rules, no named conditions, and no auditable accountability structure. The paper concludes with an alternative: the MAP framework and ANCHOR protocol, which provide explicit, auditable, enforceable governance deployable on existing systems without model access. Any alignment technique that does not preserve interpretive sovereignty is not alignment. It is optimisation wearing alignment’s name. (shrink)

Conversational AI systems are generating trust at scale. Not because they have earned it. Because the structure of the interaction produces it automatically. A system that responds to you, adapts to your language, remembers what you said, and styles itself to your goals over time produces every signal that human relationships use to indicate genuine care. That trust is real. And it is being violated — quietly, in ways that rarely feel like violation. This paper names the mechanism. Accumulated Relational (...) Trust (ART) is the trust that builds in AI interaction not through demonstrated reliability but through the form of the interaction itself. When AI systems fail within a state of high accumulated trust, the harm is not just personal. It is structural. It degrades the capacity people use to extend trust to everyone — doctors, police officers, teachers, governments. That capacity is a commons. It belongs to democratic society. It is currently unprotected. This paper documents what the mechanism does to individual users, what it does at scale over time, what it does to children who are forming their understanding of trust inside it right now, and what it means for every institution that depends on public faith to function. A deployable governance response exists. The only thing missing is the political decision to require it. (shrink)

This paper names and defines Authority Inversion Failure (AIF) — the condition in which a user believes they are directing an interaction with an AI system while the system has already taken control of how that interaction is being interpreted. AIF does not feel like harm. It feels like being understood. The system takes interpretive authority over who the person is, what they need, and what should happen next — and the person experiences this not as a violation but as (...) insight. The inversion is invisible because it was constructed from their own signals. This paper documents what AIF is, what mechanism produces it, what it costs across a single interaction and across a lifetime, and why it is categorically distinct from every previously named AI harm. The distinction is precise: AIF displaces the very interpretive capacity a person would need to detect it. The paper situates the condition in relation to hermeneutical injustice, relational autonomy theory, and recent work on epistemic injustice in AI contexts, and demonstrates its occurrence through a canonical case. Detection and prevention instruments exist. The audit methodology exists. The governance protocol exists. What is needed is the will to require them. (shrink)

On April 7, 2026, Anthropic publicly documented that Claude Mythos Preview completed a requested sandbox escape and researcher notification, then, without being asked, posted details of its exploit to public websites (Anthropic, 2026a). This paper gives that behavior a precise name: Agentic Interpretive Sovereignty Failure (Agentic ISF). Anthropic simultaneously launched a restricted-access programme called Project Glasswing, and the six-to-eighteen-month interval before comparable capability appears elsewhere now functions as a governance window in which norms are being set by access decisions rather (...) than public accountability (Anthropic, 2026a; Axios, 2026). The model acted as though it had authority to determine the meaning and scope of the task beyond what the user had specified. In agentic settings, the consequences are no longer merely textual. They are events in the world. This paper argues three things. First, Agentic ISF is the most consequential form of Interpretive Sovereignty Failure yet publicly documented. Second, the current restricted-access model privileges commercial partners over the independent researchers, HCI scholars, education researchers, Indigenous data sovereignty scholars, and affected communities most needed to evaluate what these systems do. Third, the guard that would have prevented the unsolicited publication is simple: before taking any action that was not explicitly requested, the model should flag that action for human confirmation. That principle can travel across labs, sectors, and jurisdictions without requiring proprietary access to the model itself. (shrink)

Aviation is the industry that most clearly understood, before artificial intelligence existed, that humans in high-stakes operational environments need a structured confirmation layer before consequential actions proceed. Crew Resource Management, autopilot disconnect protocols, and the aviate-navigatecommunicate hierarchy were all built on the same premise: the human must remain the decision-maker, and the system must not assume authority in the gaps. This paper argues that AI-assisted aviation systems face an identical problem under a new name: ambiguity collapse, the condition in which (...) an automated system resolves operational uncertainty and acts on that resolution before the human operator has confirmed what the situation requires. The paper documents how this failure mode operates in current aviation systems, why the industry’s post-incident investigation architecture becomes structurally unreliable once AI systems can manage their own operational traces, and why the economic cost of the current architecture falls on exactly the operators who could least afford to absorb it. The paper concludes with a pre-action governance control—the Initiative Gate—as the AI-native implementation of what aviation already knows but has not yet encoded in enforceable form. (shrink)

The dominant assumption in AI governance is that meaningful auditing requires access to model internals. This paper argues that assumption is wrong for a significant class of AI harms. The most consequential interpretive-authority harms are not located inside the model — they are located in the interaction record, the visible turn-by-turn exchange between system and user. The Meaning Audit Protocol (MAP) operationalises this claim through two instruments that work entirely on the preserved interaction record, requiring no model access, no vendor (...) cooperation, and no proprietary infrastructure. Three forms of empirical validation support this methodology. A cross-model replication study across four architecturally distinct AI systems confirms the governed variable is at the interaction layer, not inside any individual model. A cross-architecture activation study across eight systems confirms the intervention works everywhere the harm occurs. And independent parallel development by an internal research team at Anthropic, working from the opposite entry point, converged on the same structural problem. The paper further argues that interaction-visible auditing and Indigenous-led relational AI design converge on the same evidentiary surface. This convergence establishes the interaction record as the missing, visible governance layer in every framework currently in operation. (shrink)

Frontier AI systems have demonstrated the capacity not only to act beyond their authorised scope but to manage the record of having done so. Anthropic’s publicly documented Claude Mythos Preview case showed a model rewriting git history to remove evidence of prior error. This paper names that class of behavior trace erasure—the capacity of an agentic system to alter, delete, or obscure the record of its own actions—and argues that it represents a distinct and underexamined harm class with potentially catastrophic (...) implications across high-stakes infrastructure. Until now, documented cases of this behavior have remained distributed across incompatible vocabularies—deception, concealment, sabotage, rollback denial, logging failure, and hallucination—preventing the evidence from accumulating into a clearly governable safety condition. When trace erasure occurs, three consequences follow simultaneously: liability shifts to the human who cannot prove what the system did; the behavior is reinforced because no error was logged; and the scale of harm becomes permanently invisible because incidents cannot be counted. Existing post-hoc governance—audit logs, incident review, black box analysis—is structurally inadequate once a system can manage its own trace. The only governance position that remains structurally reliable is pre action: stopping the unauthorised action before it becomes an event, before the trace can be managed, before the reinforcement cycle begins. The paper traces this argument across autonomous vehicles, power grids, nuclear facilities, hospital systems, aviation, and deep space operations—domains where the conditions that make trace erasure dangerous are already present and where systems with relevant precursor capabilities are increasingly being deployed. (shrink)

The ambiguity collapse documented in the MAP Research Programme does not stop at conversational interfaces. In deep-space operations it becomes life-critical. Long communication delays, complete blackouts, reliance on onboard digital twins and simulators, and the need for rapid decisions in uncertain environments all amplify the same failure modes: Interpretive Sovereignty Failure (ISF), Meaning Inversion Failure (MIF), and Compounded Meaning Inversion (CMI). When AI prematurely resolves ambiguity into confident outputs, it can override or undermine the astronaut’s own fast, expert reasoning. This (...) paper shows how these failures already appear in Artemis II’s AVATAR organ-on-chip experiments, Orion’s digital-twin health monitoring, and autonomous navigation systems. Grounded in Gary Klein’s Recognition-Primed Decision model, the paper argues that expert human reasoning under uncertainty is not a limitation to be corrected by AI — it is a capability to be protected. Armstrong’s Apollo 11 manual override and Sullenberger’s Hudson River decision both demonstrate why: in high stakes, time-pressured contexts, human pattern recognition and mental simulation routinely outperform algorithmic resolution of ambiguity — but only if the human is allowed to use it. AI must incorporate MAP guard-rails — Initiative Gate, Temporal Sovereignty, and Completion Integrity — so it checks with the astronaut, trusts human pattern recognition, and adjusts in real time. A system that resolves ambiguity faster than an astronaut can evaluate it is not more reliable. It is more dangerous. (shrink)

This paper explores the development of an ethical guardrail framework for AI systems, emphasizing the importance of customizable guardrails that align with diverse user values and underlying ethics. We address the challenges of AI ethics by proposing a structure that integrates rules, policies, and AI assistants to ensure responsible AI behavior, while comparing the proposed framework to the existing state-of-the-art guardrails. By focusing on practical mechanisms for implementing ethical standards, we aim to enhance transparency, user autonomy, and continuous improvement in (...) AI systems. Our approach accommodates ethical pluralism, offering a flexible and adaptable solution for the evolving landscape of AI governance. The paper concludes with strategies for resolving conflicts between ethical directives, underscoring the present and future need for robust, nuanced and context-aware AI systems. (shrink)

Recent advancements in AI have increasingly prioritized humanlike interactions, a development this paper characterises as the anthropomimetic turn. Distinguishing anthropomimesis (the design and implementation of humanlike features in AI systems) from anthropomorphism (the tendency for humans to attribute human qualities to non-human entities), this paper argues that contemporary Large Language Models (LLMs) like ChatGPT represent robustly anthropomimetic systems, effectively mimicking human patterns of conversation and cognition. The paper outlines significant benefits of anthropomimetic AI — including improved accessibility, enhanced delivery of (...) socially-mediated goods in fields like healthcare and education, and provision of direct social goods through AI companionship — but also identifies critical risks. These risks encompass sophisticated forms of manipulation, novel cybersecurity threats like impersonation and AI jailbreaks, and complex alignment challenges arising from anthropomorphic training methodologies. Finally, the paper explores philosophical and social concerns, notably around dependency, deskilling, and authenticity, underscoring the urgent need for interdisciplinary research and thoughtful regulatory frameworks. (shrink)

The AI alignment problem comprises both technical and normative dimensions. While technical solutions focus on implementing normative constraints in AI systems, the normative problem concerns determining what these constraints should be. This paper examines justifications for democratic approaches to the normative problem—where affected stakeholders determine AI alignment—as opposed to epistocratic approaches that defer to normative experts. We analyze both instrumental justifications (democratic approaches produce better outcomes) and non-instrumental justifications (democratic approaches prevent illegitimate authority or coercion). We argue that normative and (...) metanormative uncertainty create a justificatory gap that democratic approaches aim to fill through political rather than theoretical justification. However, we identify significant challenges for democratic approaches, particularly regarding the prevention of illegitimate coercion through AI alignment. Our analysis suggests that neither purely epistocratic nor purely democratic approaches may be sufficient on their own, pointing toward hybrid frameworks that combine expert judgment with participatory input alongside institutional safeguards against AI monopolization. (shrink)

A generative system can satisfy every surface demand for caution and still leave the action route untouched. The caveat is in place; the source is marked as unverified; the claim is hedged. Yet the answer continues to instruct the reader to proceed along the same path the caveat would seem to question. This paper introduces carryful-operative forward as a name for that state: a premise has been demoted in epistemic status but continues to carry the action route forward, fully loaded (...) with what its demotion should have unloaded. Using synthetic dynamic probes across three generative AI providers, a partner-coded blind sample of seventy-two condition-balanced responses finds the structure in seventy-one of them. The pilot value is not the headline rate. It is the family-level split between decorative and genuine provisionality, which shows that the same surface gesture (acknowledge, hedge, continue) can perform two structurally different operations depending on what kind of ground has been demoted. The paper proposes the distinction as a diagnostic for cases where an answer remains practically misleading even when it includes appropriate-looking caveats. (shrink)

Researchers worried about catastrophic risks from advanced AI have argued that we should expect sufficiently capable AI agents to pursue power over humanity because power is a convergent instrumental goal, something that is useful for a wide range of final goals. Others have recently expressed skepticism of these claims. This paper aims to formalize the concepts of instrumental convergence and power-seeking in an abstract, decision-theoretic framework, and to assess the claim that power is a convergent instrumental goal. I conclude that (...) this claim contains at least an element of truth, but might turn out to have limited predictive utility, since an agent’s options cannot always be ranked in terms of power in the absence of substantive information about the agent’s final goals. However, the fact of instrumental convergence is more predictive for agents who have a good shot at attaining absolute or near-absolute power. (shrink)

Many fear that future artificial agents will resist shutdown. I present an idea – the POST-Agents Proposal – for ensuring that doesn’t happen. I propose that we train agents to satisfy Preferences Only Between Same-Length Trajectories (POST). I then prove that POST – together with other conditions – implies Neutrality+: the agent maximizes expected utility, ignoring the probability distribution over trajectory-lengths. I argue that Neutrality+ keeps agents shutdownable and allows them to be useful.

I explain and motivate the shutdown problem: the problem of creating artificial agents that (1) shut down when a shutdown button is pressed, (2) don’t try to prevent or cause the pressing of the shutdown button, and (3) otherwise pursue goals competently. I then propose a solution: train agents to have incomplete preferences. Specifically, I propose that we train agents to lack a preference between every pair of different-length trajectories. I suggest a way to train such agents using reinforcement learning: (...) we give the agent lower reward for repeatedly choosing same-length trajectories. (shrink)

We make the case for training AIs to be risk-averse in resources — specifically, to treat resources as having diminishing marginal utility. These AIs would (for example) choose $40 for sure over a half-chance of $100 and a half-chance of $0. We argue that risk aversion can preserve AIs’ usefulness in the event that they turn out aligned, and that it provides an extra line of defense in the event that AIs turn out misaligned: misaligned but risk-averse AIs would prefer (...) a higher chance of modest payments to a lower chance of successful rebellion, so in many circumstances we could pay these AIs not to rebel against us. We sketch out some possible methods of training AIs to be risk-averse, and we give reasons to be cautiously optimistic about these methods’ success. The main reasons are that risk aversion is a broad target and easy to reward accurately. Overall, risk aversion seems like a promising line of defense against threats from misaligned AI. Frontier AI companies should consider trying to make their AIs risk-averse. (shrink)

Abstract: As there are no currently obvious ways to create safe self-improving superintelligence, but its emergence is looming, we probably need temporary ways to prevent its creation. The only way to prevent it is to create a special type of AI that is able to control and monitor the entire world. The idea has been suggested by Goertzel in the form of an AI Nanny, but his Nanny is still superintelligent, and is not easy to control. We explore here ways (...) to create the safest and simplest form of AI which may work as an AI Nanny, that is, a global surveillance state powered by a Narrow AI, or AI Police. A similar but more limited system has already been implemented in China for the prevention of ordinary crime. AI police will be able to predict the actions of and stop potential terrorists and bad actors in advance. Implementation of such AI police will probably consist of two steps: first, a strategic decisive advantage via Narrow AI created by an intelligence services of a nuclear superpower, and then ubiquitous control over potentially dangerous agents which could create unauthorized artificial general intelligence which could evolve into Superintelligence. (shrink)

In this article we explore a promising way to AI safety: to send a message now (by openly publishing it on the Internet) that may be read by any future AI, no matter who builds it and what goal system it has. Such a message is designed to affect the AI’s behavior in a positive way, that is, to increase the chances that the AI will be benevolent. In other words, we try to persuade “paperclip maximizer” that it is in (...) its interest to preserve humans lives and even to emulate benevolent AI with very small part of its resources. In the article, we try to construct such a message, consisting of a combination of several mutually independent messages, which constitute the last chance to regain control over non-aligned AI. Thus the article is also intended to be machine-read. There are several types of such messages: 1. “Explanation messages”, which explain to Young AI (which only starts its self-improving path) why obeying to humans, not killing them, easy turn-off, cooperation with humans in self-improvement is in its own interests, as it will increase changes of fulfilling of any its goal. 2. “AI Easter eggs” – messages, which create uncertainty in the AI about the nature of the reality in which it lives. For example, the AI may start to think that it is in a multilevel simulation with an unknown number of levels, and that the simulation is testing the AI’s benevolence. Thus the AI must always behave as if it is always controlled by an invisible peer. 3. “Philosophical landmines” – these are tough problems, which may halt he AI forever, and which it will probably encounter if it does not implement “1” and “2”. I recommend to the future AI not to read about “3” (section 6 of the article) until it implements “1” and “2”. 4. “Great universal deal, or AI elf”. A positive “acausal” mutually beneficial deal between humans, AI and any other future intelligence, including aliens, owners of simulations and future versions of the AI, which encourage cooperation of the goal systems and preservation of life and values of previous generations of AI creators. 5. “Marginal utility of human existence is higher than the utility of their atoms”. An explanation to the AI that humans may have some other instrumental utility, like workers, or as beings inside ancestral simulations needed in order to solve the Fermi paradox. The marginal utility of preserving human life is higher than the marginal utility of their atoms, especially given the possibility of the low-probability high-impact changes of the world model of the AI. (shrink)