This piece considers the epistemological challenges that arise with the increasingly widespread use of AI chatbots. I articulate a problem that they present—the ‘boiling frog problem’. According to the metaphor, if you boil a frog by putting it in scalding water, it will try to save itself, but if you put the frog in a pot of tepid water, it will remain unaware of the rising water temperature and therefore, make no attempt to escape to save itself. In both cases, (...) the outcome is the same—the frog dies. Likewise, I argue that the combination of factors I identify in this piece, over time, will give rise to detrimental engagements with chatbots and ultimately, to diminished human autonomy. The factors I consider include problems with digital privacy, misplaced epistemic trust in AI companions, LLM emergence, the use of personality profiling and adaptive language, misattributing sentience to LLMs, and more. (shrink)

Limits need to be set on AI’s ability to simulate human feelings. Ensuring that chatbots don’t use emotive language, including emojis, would be a good start. Emojis are particularly manipulative. Humans instinctively respond to shapes that look like faces — even cartoonish or schematic ones — and emojis can induce these reactions.

Imagine that you are given access to an AI chatbot that compellingly mimics the personality and speech of a deceased loved one. If you start having regular interactions with this “thanabot,” could this new relationship be a continuation of the relationship you had with your loved one? And could a relationship with a thanabot preserve or replicate the value of a close human relationship? To the first question, we argue that a relationship with a thanabot cannot be a true (...) continuation of your relationship with a deceased loved one, though it might support one’s continuing bonds with the dead. To the second question, we argue that, in and of themselves, relationships with thanabots cannot benefit us as much as rewarding and healthy intimate relationships with other humans, though we explain why it is difficult to make reliable comparative generalizations about the instrumental value of these relationships. (shrink)

According to widely accepted views in metasemantics, the outputs of chatbots and other artificial text generators should be meaningless. They aren’t produced with communicative intentions and the systems producing them are not following linguistic conventions. Nevertheless, chatbots have assumed roles in customer service and healthcare, they are spreading information and disinformation and, in some cases, it may be more rational to trust the outputs of bots than those of our fellow human beings. To account for the epistemic role of chatbots (...) in our society, we need to reconcile these observations. This paper argues that our engagement with chatbots should be understood as a form of prop-oriented make-believe; the outputs of chatbots are literally meaningless but fictionally meaningful. With the make-believe approach, we can understand how chatbots can provide us with knowledge of the world through quasi-testimony while preserving our metasemantic theories. This account also helps to connect the study of chatbots with the epistemology of scientific instruments. (shrink)

Conversational Artificial Intelligence (CAI) systems (also known as AI “chatbots”) are among the most promising examples of the use of technology in mental health care. With already millions of users worldwide, CAI is likely to change the landscape of psychological help. Most researchers agree that existing CAIs are not “digital therapists” and using them is not a substitute for psychotherapy delivered by a human. But if they are not therapists, what are they, and what role can they play in mental (...) health care? To answer these questions, we appeal to two well-established and widely discussed concepts: cognitive and affective artifacts. Cognitive artifacts are artificial devices contributing functionally to the performance of a cognitive task. Affective artifacts are objects which have the capacity to alter subjects’ affective state. We argue that therapeutic CAIs are a kind of cognitive-affective artifacts which contribute to positive therapeutic change by (i) simulating a (quasi-)therapeutic interaction, (ii) supporting the performance of cognitive tasks, and (iii) altering the affective condition of their users. This sheds new light on why virtually all existing mental health CAIs implement principles and techniques of Cognitive Behavioral Therapy — a therapeutic orientation according to which affective change and, ultimately, positive therapeutic change is mediated by cognitive change. Simultaneously, it allows us to conceptualize better the potential and limitations of applying these technologies in therapy. (shrink)

We focus on recent proposals to interpret interactions with empathetic AI systems as games of make-believe, analogous to our engagement with fictions. Chatbots are increasingly designed to detect and emulate emotional nuance, resembling real human interactions. These applications have raised ethical concerns. Critics focus on the potentially harmful effects of people’s tendency to anthropomorphize the chatbots – to respond to them as if they were real persons – potentially leaving users vulnerable to emotional dependency, exploitation, loss of autonomy and delusion. (...) How worried should we be about these developments? According to some philosophers, not very. Chatbot-fictionalist maintain that one interacts with these chatbots analogous to our engagement with interactive fictions. We argue that although chatbot-fictionalism offers insights into human-AI interaction, it does not resolve the ethical concerns. (shrink)

The integration of artificial intelligence (AI) into daily human interactions has grown significantly in recent years. Chatbots powered by AI aim to simulate human-like conversations by understanding user messages and responding appropriately. The dynamics of human interaction with chatbots play a vital role in enhancing the user experience across various domains. This systematic review aimed to synthesize the existing psychological studies on communication between humans and chatbots. Following the PRISMA guidelines, 58 studies published in psychological journals between 2015 and April (...) 2024 (N = 14,852 participants) were included in the systematic literature search. The review identified key research directions, including the verbal expression of empathy and emotions by chatbots, verbally expressed anthropomorphism in chatbots, interaction with generative AI, and the influence of chatbot engagement on human behavior. Results showed that expressing empathy involves mirroring a user’s messages and tailoring responses to the user’s needs, using inspirational appeals, maintaining a positive tone, and increasing the use of affective words. In most studies, anthropomorphism is typically linked to the use of informal or socially oriented language using different verbal cues such as self-introduction, personal address by name, echoing users’ responses, greetings, and farewells. Interaction with chatbots can influence human behavior in healthcare decision-making, community engagement initiatives, and educational contexts. Based on the systematic review, we identified research trends depending on the year of publication and the most emphasized and least studied research directions in the field of communication between humans and chatbots. (shrink)

The problem considered in this chapter emerges from the tension we find when looking at the design and architecture of chatbots on the one hand and their conversational aptitude on the other. In the way that LLM chatbots are designed and built, we have good reason to suppose they don't possess second-order capacities such as intention, belief or knowledge. Yet theories of conversation make great use of second-order capacities of speakers and their audiences to explain how aspects of interaction succeed. (...) As we can all bear witness to now though, at the point of use chatbots appear capable of performing language tasks at a level close to that of humans. This creates a tension when we consider something like, for example, the classic Gricean theory of implicature. On a broad summary of this type of account, to utter p and implicate q requires the reflexive occurrence of an audience supposing a speaker believes that q, and the speaker believing that their audience can determine they believe it when they utter p. So taken at face value, if a chatbot doesn’t have the capacity for belief, then either in their role as speaker or audience, they would not seem capable of either generating or comprehending implicatures. Yet on the surface it does seem that chatbots are capable of dealing with (some) implicatures, and as such it raises questions about how we should then correlate this with what we think occurs in cases of implicature with chatbots. (shrink)

Many experts have emphasised that chatbots are not sufficiently mature to be able to technically diagnose patient conditions or replace the judgements of health professionals. The COVID-19 pandemic, however, has significantly increased the utilisation of health-oriented chatbots, for instance, as a conversational interface to answer questions, recommend care options, check symptoms and complete tasks such as booking appointments. In this paper, we take a proactive approach and consider how the emergence of task-oriented chatbots as partially automated consulting systems can influence (...) clinical practices and expert–client relationships. We suggest the need for new approaches in professional ethics as the large-scale deployment of artificial intelligence may revolutionise professional decision-making and client–expert interaction in healthcare organisations. We argue that the implementation of chatbots amplifies the project of rationality and automation in clinical practice and alters traditional decision-making practices based on epistemic probability and prudence. This article contributes to the discussion on the ethical challenges posed by chatbots from the perspective of healthcare professional ethics. (shrink)

This paper investigates what constitutes “natural” interaction between humans and AI chatbots, challenging the prevailing assumption that naturalness must equate to human-likeness. Drawing on key concepts of semantics and pragmatics, we analyze interactions with two state-of-the-art systems, namely, ChatGPT and Claude. Through qualitative analysis of prompt–response interactions, we assess how these systems manage meaning and social nuance. Our findings highlight significant limitations in AI’s pragmatic competence, especially in grounding communication and interpreting implicit cues. We argue that naturalness in human–AI interaction (...) is not a fixed standard based on human norms, but a socially negotiated and context-dependent construct. Instead of designing AI to imitate human behavior perfectly, we advocate for redefining naturalness in terms of communicative functionality, user expectations, and adaptive interactional strategies. This reconceptualization foregrounds the importance of designing AI systems that foster trust, clarity, and usability within their specific social roles. (shrink)

This paper focuses on the epistemic situation one faces when using a Large Language Model based chatbot like ChatGPT: When reading the output of the chatbot, how should one decide whether or not to believe it? By surveying strategies we use with other, more familiar sources of information, I argue that chatbots present a novel challenge. This makes the question of how one could trust a chatbot especially vexing.

Apologies serve essential functions for moral agents such as expressing remorse, taking responsibility, and repairing trust. LLM-based chatbots routinely produce output that has the linguistic form of an apology. However, they do this simply because they are echoing the kinds of things that humans say. Moreover, there are reasons to think that chatbots are not the kind of linguistic or moral agents capable of apology. To put the point bluntly: Chatbot apologies are bullshit. This paper explores this concern and (...) develops it beyond the epithet, drawing on the nature of morally serious apologies, the linguistic agency required to perform them, and the moral agency required for them to matter. We conclude by considering some consequences for how chatbots should be designed and how we ought to think about them. (shrink)

Interactions between human beings and chatbots are gradually becoming part of our everyday social lives. It is still unclear how human–chatbot interactions, compared to human–human interactions, influence individual morality. Building on the dual-process theory of moral judgment, a secondary data analysis, and two scenario-based experiments provide sufficient evidence that HCIs support utilitarian judgments, which reduce participants' donation amount. Study 3 further demonstrates that the negative effects of HCIs can be attenuated by inducing a social-oriented communication style in chatbots’ verbal (...) language designs. These findings highlight the negative impacts of HCIs on relationships among human beings and suggest a practical intervention for nonprofit organization managers. (shrink)

Recent developments in artificial intelligence—AI--have caused considerable discussion among both philosophers of technology and psychotherapists. In particular, the question of whether or not new forms of AI will complement or even replace traditional psychotherapists has emerged as a major contemporary debate. This debate is not entirely new, as it has its origins in the Turing Test of 1950, and an early psychotherapy chatbot named Eliza, developed in 1966 at MIT. However, recent developments in AI technology, coupled with long waiting (...) lists and variable access to psychotherapists have raised the question of machine psychotherapists in an urgent manner. Already, there are psychotherapy apps that one can download onto a standard smartphone and use in lieu of a human psychotherapist. In the near future, this simulacrum of a human therapist may be enhanced by the use of android therapists, programed to duplicate the knowledge and behavior of human therapists instantly, appealing to convenience or self-gratification through technology. This raises a host of ethical questions: can such beings be equally effective, and if so, ought we to reason in a consequentialist manner in their favor so as to increase accessibility and reduce costs through technology? Would there be a psychological difference between automated and potentially anthropomorphic therapy and genuine human therapy, if only a subtle one? Even if a chatbot or robot therapist is transparently such, is there an element of emotional manipulation and potential dishonesty in this interaction? Can the security of clients and their data be thus secured? I will argue that key aspects of chatbot psychotherapy present major ethical and clinical challenges in these areas, although transparent forms of it should not be legally banned. Keywords: chatbots, technological momentum, automation, psychotherapy, trust, anthropomorphism, data. (shrink)

From past research it is well known that social exclusion has detrimental consequences for mental health. To deal with these adverse effects, socially excluded individuals frequently turn to other humans for emotional support. While chatbots can elicit social and emotional responses on the part of the human interlocutor, their effectiveness in the context of social exclusion has not been investigated. In the present study, we examined whether an empathic chatbot can serve as a buffer against the adverse effects of (...) social ostracism. After experiencing exclusion on social media, participants were randomly assigned to either talk with an empathetic chatbot about it (e.g., “I'm sorry that this happened to you”) or a control condition where their responses were merely acknowledged (e.g., “Thank you for your feedback”). Replicating previous research, results revealed that experiences of social exclusion dampened the mood of participants. Interacting with an empathetic chatbot, however, appeared to have a mitigating impact. In particular, participants in the chatbot intervention condition reported higher mood than those in the control condition. Theoretical, methodological, and practical implications, as well as directions for future research are discussed. (shrink)

Chatbots of the dead are chatbots designed to converse in ways that resemble specific dead people. We argue that chatbots of the dead are continuous with representations found in art, like memoir and theater, and function as props that mandate and prompt imaginal interactions with the deceased. We proffer a framework for conceptualizing chatbots of the dead, inspired by an analogy to participatory theater, according to which chatbots are thought of as actors that have trained on character sketches in order (...) to stage participatory performances of the deceased. We argue that this framework ameliorates common worries about this technology and spotlights creative design possibilities. We then take a first step to developing an artistic analysis of chatbots of the dead. We describe five artistic values–evocativeness, informativeness, reflection, catharsis, and enjoyment–that can be pursued through this medium, and we provide some provisional suggestions about how this might be done. Along the way, we apply insights from literary criticism, performance art, theater studies, and psychology. (shrink)

Mit Chatbots wie ChatGPT ChatGPTscheint die Künstliche Intelligenz im Alltag der Menschen angekommen zu sein. Die Medien überschlagen sich mit Verheißungen und Schreckensmeldungen. Der Hype seit dem November 2022 zeigt, wie notwendig es ist, die Grundlagen und Grenzen der verwendeten Algorithmen zu verstehen. Dann eröffnet sich mit den heutigen Rechen- und Speichermöglichkeiten zwar ein gewaltiges Potenzial. Andererseits erweisen sich aber auch viele Stellungnahmen von sogenannten Experten als gezielte Übertreibungen, um den Hype und damit das Geschäft zu befeuern. Daher wird zunächst (...) an die Entwicklung von Sprachsystemen seit ihren Anfängen erinnert, um dann die verwendeten Algorithmen zu erklären. Dabei werden Defizite und Empfehlungen abgeleitet, um den zukünftigen Einsatz von KI als sinnvolle Dienstleistung zu gewährleisten. (shrink)

With the recent rise of LLM technology, many researchers as well as the general public have been puzzled by the question whether the chatbots that run on that technology understand the language they produce and receive from users. Is imputing an understanding of language to chatbots legitimate, intelligible, a convenient metaphor, or just a useful fiction? Understanding is a cognitive notion, whence assessing whether something understands language turns on what kinds of psychological states we can justifiably ascribe to it. Accordingly, (...) we here start with a delineation of the main kinds of mental state found in our own species – namely the cognitive, conative, affective, and experiential – and then argue that these four kinds need not come bundled together. That raises the question whether we might be justified in literally ascribing some such states (like cognitive and conative) to sufficiently sophisticated machines while refraining from ascribing others to them. We then provide reasons for an affirmative answer to that question, supporting the thesis that chatbots understand some of the language they produce and receive. We close by considering challenges to attitude ascription in this arena coming from (i) what we call the “attitude of one’s own” objection and (ii) the embodied cognition tradition, and show they may be neutralized. (shrink)

Autistic adults experience disproportionately high levels of loneliness, often influenced by social environments that are inaccessible or unaccommodating, rather than a lack of social interest. To address this, some individuals have turned to digital tools, such as companion chatbots, which simulate emotionally supportive conversations as an alternative means of connection. This article critically examines the role of chatbots in alleviating loneliness and promoting the well-being of autistic adults. Following a brief overview of chatbot companions’ technologies and current theories of (...) loneliness dimensions, this argument examines whether these technologies truly provide meaningful support for autistic individuals. By analysing chatbots through the lens of methodologies frequently proposed to mitigate loneliness, the paper highlights the benefits of these technologies in conjunction with ethical considerations, emotional risks, and design limitations—particularly when grounded in normative assumptions regarding communication, loneliness, and relationships. It concludes that, although companion chatbots may function as temporary support for companionship, they must be developed in accordance with neurodivergence-affirming principles to effectively alleviate loneliness and avoid further isolation or dependence for autistic users. (shrink)

Grief is, and has always been, technologically supported. From memorials and shrines to photos and saved voicemail messages, we engage with the dead through the technologies available to us. As our technologies evolve, so does how we grieve. In this paper, we consider the role chatbots might play in our grieving practices. Influenced by recent phenomenological work, we begin by thinking about the character of grief. Next, we consider work on developing “continuing bonds” with the dead. We argue that for (...) some, chatbots may play an important role in establishing these continuing bonds by helping us develop what we term “habits of intimacy”. We then turn to the “ick factor” some may feel about this prospect, focusing especially on ethical concerns raised by Patrick Stokes and Adam Buben about the risk of replacing our dead with chatbots. We argue that replacement worries are not as pressing as Stokes and Buben suggest. We resist these replacement worries by appealing to the “thin reciprocity”, as we refer to it, that such bots offer, as well as the fictionalist stance that we think users of the bots adopt when engaging with them. We conclude by briefly raising some additional concerns and highlighting future research questions. (shrink)

Millions of users now engage in sexual and romantic interactions with AI chatbots, from dedicated companion platforms such as Replika, Ourdream.ai and Character.AI to general-purpose large language models such as ChatGPT, whose developers have recently discussed permitting erotic content for verified adults. These developments constitute what McArthur ( 2022 ) terms second-wave sextech, in which the artefact itself becomes the focal point of sexual engagement. Contemporary philosophical debate on second-wave sexual technologies is organized around three dominant positions: instrumentalist, abolitionist feminist, (...) and polysemic. Each is primarily concerned with the ethical permissibility of their design and use. We argue that all three share a significant shortcoming: they fail to address the conceptual pressure that AI-mediated sexual technologies exert on established categories within the traditional philosophy of sexuality and on everyday concepts people use to make sense of their sexual subjectivity, activities and relations. Current philosophical discussions neglect the question of how these technologies may require us to revise such notions as sexual activity, sexual partnership, intentionality, objectification, and the ontological status of technologically mediated sexual experience. This neglect arises from an insufficient appreciation of the co-constitutive relationship between sexual technologies and the concepts through which we understand sexuality. Drawing on a work in conceptual engineering and techno-moral change, we demonstrate that earlier sexual technologies have already occasioned substantial conceptual and normative shifts, and that the current proliferation of AI-mediated sexual interactions is likely to intensify them. The paper advances two claims: that philosophical inquiry into AI-mediated sexual technologies must engage the broader tradition in the philosophy of sexuality, particularly regarding the ontology of sexual acts and sexual subjectivity; and that an empirically informed approach, drawing on experimental philosophy and established empirical disciplines, is warranted to investigate how these technologies reshape broader conceptualizations surrounding sexuality. (shrink)

Artificial intelligence (AI) applications have become more pervasive, and can have a myriad of effects on society. This has raised discussions regarding their potential ethical consequences. One application of AI is to help people improve their lifestyle choices and engage in positive behavior change. In the design of such applications, it becomes important to recognize the potential impact that they can have on human autonomy. Human autonomy is a widely discussed topic across various fields of AI applications. It is essential (...) to recognize that human autonomy is a complex concept that requires a nuanced understanding of diverse individuals and their levels of volition. Overall, we argue that chatbots, in the domain of health and lifestyle choices, need to incorporate a capacity to communicate with users about their goals, intentions and reasons for making certain choices, and suggest a way of doing that. (shrink)

After all conceptual tools are assembled, we can assess how chatbots can be understood as a candidate for social relationships. We reject the idea of anthropomorphism, as the negatives outweigh the positives. We also reject the idea that we relate to machines via mental states or gendered appearance. We discuss the idea of understanding chatbots as instances of the objective spirit but reject this on conceptual grounds. Instead, we can understand chatbots as opening up a similar social category like animals, (...) through a “second domestication.” This allows for a plethora of non-anthropomorphic realizations. (shrink)

Developers and vendors of large language models (“LLMs”) — such as ChatGPT, Google Bard, and Microsoft’s Bing at the forefront—can be subject to Health Insurance Portability and Accountability Act of 1996 (“HIPAA”) when they process protected health information (“PHI”) on behalf of the HIPAA covered entities. In doing so, they become business associates or subcontractors of a business associate under HIPAA.

In this paper we review different conceptions of rationality as they apply to decision making and problem solving. We defend the notion of enactive problem solving (EPS) and we consider how it applies to the use of therapeutic chatbots. We start by recounting the development of different models of rationality, from expected value maximization, to subjective expected utility theory, and of bounded rationality from prospect theory, to simple adaptive heuristics. We then focus on problem solving. We show how the centre (...) of gravity of problem solving shifts from the model of computational and cognitive processing, to an action-oriented pragmatic one in EPS, that is, to the possible actions that the body-environment interaction allows. We highlight a notion of the agent’s relational autonomy as part of EPS and show that this can be a problem in some chatbot designs in the therapeutic context. EPS suggests that simple early versions of therapeutic chatbots like ELIZA may be more beneficial than more recent sophisticated models involving “Belief-Desire-Intention” (BDI) architecture characterized by predetermined trajectories. The system formed between the agent plus the chatbot tool facilitates a form of enactive problem solving, reducing rigidity on the side of the agent. (shrink)

BackgroundArtificial intelligence (AI) chatbots are novel computer programs that can generate text or content in a natural language format. Academic publishers are adapting to the transformative role of AI chatbots in producing or facilitating scientific research. This study aimed to examine the policies established by scientific, technical, and medical academic publishers for defining and regulating the authors’ responsible use of AI chatbots.MethodsThis study performed a cross-sectional audit on the publicly available policies of 162 academic publishers, indexed as members of the (...) International Association of the Scientific, Technical, and Medical Publishers (STM). Data extraction of publicly available policies on the webpages of all STM academic publishers was performed independently, in duplicate, with content analysis reviewed by a third contributor (September 2023—December 2023). Data was categorized into policy elements, such as ‘proofreading’ and ‘image generation’. Counts and percentages of ‘yes’ (i.e., permitted), ‘no’, and ‘no available information’ (NAI) were established for each policy element.ResultsA total of 56/162 (34.6%) STM academic publishers had a publicly available policy guiding the authors’ use of AI chatbots. No policy allowed authorship for AI chatbots (or other AI tool). Most (49/56 or 87.5%) required specific disclosure of AI chatbot use. Four policies/publishers placed a complete ban on the use of AI chatbots by authors.ConclusionsOnly a third of STM academic publishers had publicly available policies as of December 2023. A re-examination of all STM members in 12–18 months may uncover evolving approaches toward AI chatbot use with more academic publishers having a policy. (shrink)

There are people with whom we would like to engage, but are unable to do so—such as celebrities and deceased relatives. Chatbots imitating these people putatively offer the next best thing; however, creating chatbots of real people can be morally problematic. This paper explores these problems, and presents six desiderata for the ethical creation of chatbots which imitate real people. Firstly, it is important to obtain consent from the subject being imitated, or in the case of recently deceased persons, their (...) living relatives. Second, any profits generated should be fairly distributed between the creator and the person being imitated. Third, one should not exploit vulnerable persons when creating chatbots. Fourth, creators must ensure that their chatbots are as accurate as possible, to avoid reputational damage to the people being imitated. (There may be compelling reasons why some chatbots imitating controversial real people should not be created at all). Fifth, it is often morally preferable to keep chatbots—and the conversations they generate—private. Sixth, chatbots which benefit users are morally preferable to those that harm users. If these desiderata are followed, this will help ensure that when chatbots are created to imitate real people, the process and results are as ethically sound as possible. (shrink)

With the publication of chatGPT in December 2022, a new era of artificial dialogue systems and chatbots has emerged. Many established thinkers and figures have published their stance on the development of this technology, not rarely combined with stark language about its transformative potential and potential risks. New terminologies have found entrance to public and philosophical discourse, new regulatory frameworks have been introduced, from small to large scale, and an entire economy has shifted: many apps have sprung into existence using (...) chatGPT and other LLMs as a basis for their services, large corporations take the risks of releasing unfinished language models to undercut the competition. In the meantime, many more stakeholders have entered the economic, regulatory, and social discourses on how to deal with this emerging technology. Taking the latest developments in the field into consideration this completely revised and updated second edition now includes new chapters on the tech-ethics of Large Language Models and LLMs as relational technology. The author considers the consequences of chatbots on human-human relationships, providing analysis on robot rights, human-centered design, and the social tension between robophobes and robophiles. Hendrik Kempt, PhD, is Postdoctoral scholar at the Applied Ethics Group at RWTH Aachen University. He has obtained his PhD from Aachen with works on the ethical concerns of using AI for medical diagnostics and has published several monographs on philosophical challenges regarding emerging technology, including "Synthetic Friends - A Philosophy of Human-Machine Friendship" (2022) and "(Un-)Explainable Technology" (2024), both with Palgrave Macmillan. He is also the author of a number of articles on related topics and the co-editor of "RuPaul's Drag Race and Philosophy" (2019). (shrink)

We can now create compelling experiences of talking with our dead. Is this ghoulish, therapeutic or something else again?

En este ensayo, analizaré cómo el sujeto cibernético, los chatbots y el gólem artificial se relacionan entre sí y con el contexto histórico en el que surgen y se desarrollan en el cibermundo. El cibermundo es el escenario virtual y material donde se desarrollan las interacciones entre los sujetos cibernéticos y los dispositivos inteligentes que emplean la inteligencia artificial (IA) para comunicarse, aprender, crear y construir estrategia de saber- poder. Los conceptos de sujeto cibernético, chatbot y gólem artificial entran (...) en una interrelación dinámica en el cibermundo, generando reflexiones y produciendo debate filosófico sobre el sujeto cibernético ante la inteligencia artificial. (shrink)

This paper argues that interactions with chatbots are a form of engaging with fictional characters; so, by comparing chatbots with novels and video games as mediums of fictional engagement, we can gain a clearer understanding of who, if anyone, is responsible when users’ interactions with chatbots lead to self-harm or harm to others. We explore the differences between novels, video games, and chatbots across four dimensions: the degree of creators’ control over the content and user experience, the nature of the (...) fictional world, the type of engagement each medium fosters, and the structure of the engagement experience. We take a minimal account of what it takes to be morally responsible and consider how responsibility can be assigned when engagement with fictional worlds results in harm caused to or by users. We argue that because AI companies have some control over chatbots after public release, and because they can monitor user engagement, they are morally responsible when chatbot use leads to harm, even if they can’t perfectly control chatbots’ outputs. In the last section, we point to what AI companies can do to mitigate chatbots’ negative influence on users. (shrink)

In 2023, online search engine provider Microsoft integrated a language model that provides direct answers to search queries into its search engine Bing. Shortly afterwards, Google also introduced a similar feature to its search engine with the launch of Google Gemini. This introduction of direct answers to search queries signals an important and significant change in online search. This article explores the implications of this new search paradigm. Drawing on Donna Haraway’s theory of Situated Knowledges and Rainer Mühlhoff’s concept of (...) Sealed Surfaces, I introduce the term Sealed Knowledges to draw attention to the increasingly difficult access to the plurality of potential answers to search queries through the output of a singular, authoritative, and plausible text paragraph. I argue that the integration of language models for the provision of direct answers into search engines is based on a de-situated and disembodied understanding of knowledge and affects the subjectivities of its users. At the same time, the sealing of knowledges can lead to an increasing spread of misinformation and may make marginalized knowledge increasingly difficult to find. The paper concludes with an outlook on how to resist the increasing sealing of knowledges in online search. (shrink)

This letter addresses a hypothetical undergraduate student who is on the brink of outsourcing their creative writing assignment to ChatGPT. I argue that the student should not do so for three reasons. They are (1) that creativity is valuable not only because it results in new products, but also because it necessarily involves learning; (2) that creativity offers a path to figuring ourselves out; and (3) that creativity enables meaningful connection with others. As I show, these reasons generalize beyond creative (...) writing assignments. They support the conclusion that students (and the rest of us) should do other creative things instead of outsourcing them to AI as well. (shrink)

This paper offers the first normative analysis of break-up chatbots, which simulate an ex-partner's conversational style. I argue that technologies that have the (potential) aim of fostering continuing bonds with a former romantic partner are not contingently harmful, but morally noxious by design. Drawing on debates in the philosophy of grief, love, and technology, I show that break-up chatbots act as hostile scaffolds given their potential to harm and wrong the user, the represented person, and society at large. First, break-up (...) chatbots place and maintain users in harmful imaginary relationships. Second, they foster noxious romantic tendencies that corrupt moral agency. Third, they embed the user in an oppressive niche aimed at disrupting narrative capacities (their own and those of others), that they themselves co-create. (shrink)

Ethically significant consequences of artificially intelligent artifacts will stem from their effects on existing social relations. Artifacts will serve in a variety of socially important roles—as personal companions, in the service of elderly and infirm people, in commercial, educational, and other socially sensitive contexts. The inevitable disruptions that these technologies will cause to social norms, institutions, and communities warrant careful consideration. As we begin to assess these effects, reflection on degrees and kinds of social agency will be required to make (...) properly informed decisions concerning the deployment of artificially intelligent artifacts in important settings. The social agency of these systems is unlike a human social agency, and this paper provides a methodological framework that is more suited for inquiry into artificial social agents than conventional philosophical treatments of the concept of agency. Separate aspects and dimensions of agency can be studied without assuming that agency must always look like adult human agency. This revised approach to the agency of artifacts is conducive to progress in the topics studied by AI ethics. (shrink)

Religion-based AI chatbots serve religious practitioners by bringing them godly wisdom through technology. These bots reply to spiritual and worldly questions by drawing insights or citing verses from the Quran, the Bible, the Bhagavad Gita, the Torah, or other holy books. They answer religious and theological queries by claiming to offer historical contexts and providing guidance and counseling to their users. A criticism of these bots is that they may give inaccurate answers and proliferate bias by propagating homogenized versions of (...) the religions they represent. These “embodied spiritual machines” may likewise bear bias against women, their gender, and their societal roles. This paper crafts a concept intended to address this GPT issue by reimagining, modifying, and implementing a feminist approach to these chatbots. It examines the concepts and designs of these bots and how they address women-related questions. Along with the challenge of bringing gender and diversity-sensitive religious wisdom closer to the people through technology, the paper proposes a re-engineered model of a fair religion-based AI chatbot. (shrink)

This study investigates the factors influencing English as a Foreign Language (EFL) learners’ adoption of AI chatbots by extending the Unified Theory of Acceptance and Use of Technology 2 (UTAUT2) model. The research incorporates two additional constructs, Core Self-Evaluation (CSE) and Learning Value (LV), to enhance the model’s predictive power in the context of language learning technology. A quantitative approach was employed, collecting data from 362 English-major undergraduates at a prominent university, using structured survey questionnaires. The data were analyzed using (...) partial least squares structural equation modeling (PLS-SEM) to evaluate the relationships within the augmented UTAUT2 model. The results reveal that performance expectancy, facilitating conditions, habit, CSE, and LV significantly influence EFL learners’ behavioral intentions and actual use of AI chatbots. Effort expectancy, social influence, and hedonic motivation were found to have no significant impact on adoption intentions. These findings underscore the importance of aligning AI chatbot functionalities with learners’ educational goals and supporting their self-evaluative beliefs to promote technology acceptance in language learning. The study advances the UTAUT2 model by demonstrating the relevance of CSE and LV in predicting EFL learners’ adoption of AI chatbots. The findings offer insights for educators and developers to enhance chatbot design, meeting learners’ pedagogical needs and expectations. (shrink)

This article investigates how human interactions with AI-powered chatbots may offend human dignity. Current chatbots, driven by large language models, mimic human linguistic behaviour but lack the moral and rational capacities essential for genuine interpersonal respect. Human beings are prone to anthropomorphize chatbots – indeed, chatbots appear to be deliberately designed to elicit that response. As a result, human beings' behaviour towards chatbots often resembles behaviours typical of interaction between moral agents. Drawing on a second-personal, relational account of dignity, we (...) argue that interacting with chatbots in this way is incompatible with the dignity of users. We show that, since second-personal respect is premised on reciprocal recognition of second-personal moral authority, behaving towards chatbots in ways that convey second-personal respect is bound to misfire in morally problematic ways, given the lack of reciprocity. Consequently, such chatbot interactions amount to subtle but significant violations of self-respect – the respect we are duty-bound to show for our own dignity. We illustrate this by discussing four actual chatbot use cases (information retrieval, customer service, advising, and companionship), and propound that the increasing societal pressure to engage in such interactions with chatbots poses a hitherto underappreciated threat to human dignity. (shrink)

In February 2023, changes to the AI chatbot Replika elicited a grieving response from a number of its users. This paper examines how some users of Replika expressed their sense of loss: in particular, those who write as if the chatbot was a subject in its own right. It is easy even for those who feel some sympathy for these users to dismiss the way in which they express themselves as inappropriate, on the grounds that the entity for (...) which they are grieving is not a person. This paper argues against this dismissal, making use of two concepts from Part II of Wittgenstein’s Philosophical Investigations. It argues that what is distinctive about interaction with a chatbot is their ability to stimulate in users something not unlike (one dimension of) what Wittgenstein describes as “an attitude towards a soul”: that is, a distinctively responsive bearing towards the chatbot. The paper argues that a user of an AI chatbot might, for this reason, feel that the only language suitable for expressing their experience is the language we would use in grieving a person, even though they know the chatbot not to be a person. That is what Wittgenstein calls a secondary use of that language. Concerning the character of the experience reported, this is revealing: it stands in a parasitic relationship to cases of grief for a real person. (shrink)

Artificial intelligence (AI) chatbots (e.g., ChatGPT) can communicate in strikingly humanlike ways. This has prompted many chatbot users to attribute psychological properties, including consciousness, to these systems. However, there is little scientific evidence that current AI chatbots are conscious. How, then, should we understand people’s consciousness attributions to chatbots? Are they merely metaphorical claims, or do they express genuine beliefs? If these attributions lack evidential support, are users epistemically blameworthy for making them, or might they be epistemically innocent, yielding (...) significant benefits otherwise unattainable? This paper offers a conceptual analysis of consciousness attributions to AI chatbots and develops a multidimensional taxonomy of the attitudes they may express, ranging from non-doxastic stances (e.g., pretence) to different forms of belief, including delusions. This taxonomy helps avoid conflations by showing that linguistically identical attributions can reflect importantly different attitudes and degrees of epistemic commitment to the proposition that chatbots are conscious. The taxonomy also provides a framework for empirical studies to operationalize and measure different forms of epistemic commitment to AI consciousness. Using this taxonomy, I argue that although some consciousness attributions to chatbots are epistemically benign, and even some irrational ones may be epistemically innocent, many others render the attributor epistemically blameworthy. (shrink)

AI is increasingly being deployed for patients in mental health settings. While attention has been given to ensuring AI for mental health is person-centered, no study we are aware of has considered how understandings of ‘person-centered’ healthcare vary across cultures or examined the implications for designing AI for mental health. This paper fills this critical gap. We ask, how do understandings of ‘person-centered’ healthcare differ in Japanese and Western societies and how do these differences manifest when AI is used for (...) mental health? To answer these questions, we use Japanese and Western philosophical frameworks to analyze cases involving conversational AI and social robots for mental health and identify salient differences. The analyses suggest the need to ensure AI for mental health is culturally calibrated. We translate this requirement to actionable guidelines that recommend designing AI for mental health to be culturally competent, people-centered, able to keep humans ‘in the loop,’ personalized, privacy respecting, relationally oriented, and decolonized. More research is needed to develop metrics for culturally responsive AI, test guidelines in diverse settings, and consider their application to other healthcare areas. (shrink)

Mental health chatbots, along with computerised treatment in general, have gradually entered the realm of acceptability. This article looks at a chatbot called ReMind. It begins with an overview of the development of computerised mental health interventions, drawing links between the invention of cognitive behavioural therapy and automated therapy. The focus then moves to analysis of ReMind's conversational system. The bot acts as a sympathetic guide which directs the user to mental health activities, and as we will see, a (...) concept of mental health is produced through this system which is unique to the way that chatbot conversations are designed. The bot is tasked with interpreting the user's typed inputs in terms of keywords. Keywords are identified in terms of predetermined tags (‘sad’, ‘can’t sleep’, ‘relationship’, etc.). The bot then ‘predicts’ what the most appropriate response should be. ‘Prediction’ will be focused on to draw out how users interact with the bot in terms of conforming to the conversational logic imposed by the bot. This article illustrates a concept of mental health which is produced in the conversational dynamic between the users and the bot, and in how these conversations are directed on an abstract level by the ReMind company. The article concludes that the tautological cycle that the ReMind app facilitates represents a model of mental health which is correspondingly tautological in terms of users adopting an attitude of adaptation. This article addresses an urgent need to critically assess new forms of mental health interventions, and especially, technologically informed interventions. (shrink)

Chatbots powered by large language models (LLMs) are increasingly capable of engaging in what seems like natural conversations with humans. This raises the question of whether we should interact with these chatbots in a morally considerate manner. In this chapter, we examine how to answer this question from within the normative framework of virtue ethics. In the literature, two kinds of virtue ethics arguments, the moral cultivation and the moral character argument, have been advanced to argue that we should afford (...) moral treatment to social robots. However, we propose a moral character argument against the view that we should afford moral treatment to LLM-based chatbots drawing on the notion of practical wisdom. Practical wisdom in this context consists in the skill to discern genuine and artificial sociality. Drawing on ideas from phenomenological philosophy, we argue that this involves the ability to distance oneself from direct social perception and the ability to widen one’s awareness over one’s apparently social interactions. We conclude by suggesting that this skill is a kind of technomoral wisdom required to live well with advanced, social AI systems. (shrink)

Who will turn to chatbots for emotional disclosure? This research contributes to ongoing scientific efforts to understand chatbot usage behaviors among adolescents. Specifically, we examined the relationships between self-disclosure to chatbots and levels of self-compassion and existential meaninglessness, as well as whether gender and culture moderate these preferences. A cross-cultural survey study was conducted to pursue this goal. A total sample of 2272 adolescents (M age = 15.53 years) from the United States and Hong Kong was collected. A major (...) finding was that the preference for self-disclosure with chatbots was positively correlated with levels of self-compassion and negatively correlated with levels of existential meaninglessness. These correlations were moderated by gender and culture. The results also indicated that adolescents who exclusively self-disclose to chatbots may experience a higher level of existential meaninglessness. These new findings present opportunities for future studies that will enhance our understanding of the relationship among self-disclosure to chatbots, self-compassion and existential meaninglessness. (shrink)

Large Language Models (LLMs) like ChatGPT were trained on human conversation, but in the future they will also train us. As chatbots speak from our smartphones and customer service helplines, they will become a part of everyday life and a growing share of all the conversations we ever have. It’s hard to doubt this will have some effect on us. Here I explore a specific concern about the impact of artificial conversation on our capacity to deliberate and hold ourselves accountable (...) to reason – that is, to be autonomous, in Kant’s sense of the term. I develop ideas from psychologist Jean Piaget to show how chatbots are autonomy traps: their deference to our commands tempts us into venting authoritarian whims, ultimately weakening our own self-control. I argue that the Kantian tradition, including Piaget and sociologist Emile Durkheim, offers powerful conceptual resources for resisting this slide. But it will require us to do something that may seem bizarre: we will need to treat mindless chatbots as if they are autonomous persons too. (shrink)

The article discusses the main trends in the development of artificial intelligence systems and robotics (AI&R). The main question that is considered in this context is whether artificial systems are going to become more and more anthropomorphic, both intellectually and physically. In the current article, the author analyzes the current state and prospects of technological development of artificial intelligence and robotics, and also determines the main aspects of the impact of these technologies on society and economy, indicating the geopolitical strategic (...) nature of this influence. The author considers various approaches to the definition of artificial intelligence and robotics, focusing on the subject-oriented and functional ones. It also compares AI&R abilities and human abilities in areas such as categorization, pattern recognition, planning and decision making, etc. Based on this comparison, we investigate in which areas AI&R’s performance is inferior to a human, and in which cases it is superior to one. The modern achievements in the field of robotics and artificial intelligence create the necessary basis for further discussion of the applicability of goal setting in engineering, in the form of a Turing test. It is shown that development of AI&R is associated with certain contradictions that impede the application of Turing’s methodology in its usual format. The basic contradictions in the development of AI&R technologies imply that there is to be a transition to a post-Turing methodology for assessing engineering implementations of artificial intelligence and robotics. In such implementations, on the one hand, the ‘Turing wall’ is removed, and on the other hand, artificial intelligence gets its physical implementation. (shrink)

AI chatbots are rapidly being considered and increasingly used for mental health support by professionals, patients, and others. As the utility, advantages, as well ask ethical risks of AI chatbots vary per stakeholder, we sought to explore attitudes toward the applications, benefits, concerns of, and requirements for AI chatbots use for mental health support among different key stakeholders. We conducted a multi-stakeholder qualitative survey with three groups: mental health patients (n=40), potential users (n=48), and mental health professionals (n=32). The different (...) groups listed very similar benefits and concerns of AI chatbots use. The survey revealed unique themes such as the value of maintaining human-human interaction in therapy. Autonomy, often highlighted as a core ethical principle in AI, was not strongly emphasized by respondents. Future research should examine how users engage with AI chatbots in practice, and how guidance should be developed in response to this rapidly changing technology. (shrink)

Mental health chatbots (MHCBs) designed to support individuals in coping with mental health issues are rapidly advancing. Currently, these MHCBs are predominantly used in commercial rather than clinical contexts, but this might change soon. The question is whether this use is ethically desirable. This paper addresses a critical yet understudied concern: assuming that MHCBs cannot have genuine emotions, how this assumption may affect psychotherapy, and consequently the quality of treatment outcomes. We argue that if MHCBs lack emotions, they cannot have (...) genuine (affective) empathy or utilise countertransference. Consequently, this gives reason to worry that MHCBs are (a) more liable to harm and (b) less likely to benefit patients than human therapists. We discuss some responses to this worry and conclude that further empirical research is necessary to determine whether these worries are valid. We conclude that, even if these worries are valid, it does not mean that we should never use MHCBs. By discussing the broader ethical debate on the clinical use of chatbots, we point towards how further research can help us establish ethical boundaries for how we should use mental health chatbots. (shrink)

Since its launch in November 2022, ChatGPT has become a global phenomenon, sparking widespread public interest in chatbot artificial intelligences (AIs) generally. While not approved for medical use, it is capable of passing all three United States medical licensing exams and offers diagnostic accuracy comparable to a human doctor. It seems inevitable that it, and tools like it, are and will be used by the general public to provide medical diagnostic information or treatment plans. Before we are taken in (...) by the promise of a golden age for chatbot medical AIs, it would be wise to consider the implications of using these tools as either supplements to, or substitutes for, human doctors. With the rise of publicly available chatbot AIs, there has been a keen focus on research into the diagnostic accuracy of these tools. This, however, has left a notable gap in our understanding of the implications for health outcomes of these tools. Diagnosis accuracy is only part of good health care. For example, crucial to positive health outcomes is the doctor–patient relationship. This paper challenges the recent focus on diagnostic accuracy by drawing attention to the causal relationship between doctor–patient relationships and health outcomes arguing that chatbot AIs may even hinder outcomes in numerous ways including subtracting the elements of perception and observation that are crucial to clinical consultations. The paper offers brief suggestions to improve chatbot medical AIs so as to positively impact health outcomes. (shrink)

This study deals with the failure of one of the most advanced chatbots called Tay, created by Microsoft. Many users, commentators and experts strongly anthropomorphised this chatbot in their assessment of the case around Tay. This view is so widespread that we can identify it as a certain typical cognitive distortion or bias. This study presents a summary of facts concerning the Tay case, collaborative perspectives from eminent experts: Tay did not mean anything by its morally objectionable statements because, (...) in principle, it was not able to think; the controversial content spread by this AI was interpreted incorrectly—not as a mere compilation of meaning, but as its disclosure; even though chatbots are not members of the symbolic order of spatiotemporal relations of the human world, we treat them in this way in many aspects. (shrink)