<?xml version="1.0" encoding="utf-8" standalone="yes"?><!DOCTYPE article PUBLIC "-//NLM//DTD JATS (Z39.96) Journal Publishing DTD v1.3 20210610//EN" "https://jats.nlm.nih.gov/publishing/1.3/JATS-journalpublishing1-3.dtd"><article xml:lang="en-US" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink" article-type="originalsarticle" dtd-version="1.0"><front><journal-meta><journal-id journal-id-type="publisher-id">3052-914X</journal-id><journal-title-group><journal-title>Experimental Philosophy</journal-title><abbrev-journal-title/></journal-title-group><issn publication-format="electronic">3052-914X</issn><publisher><publisher-name>  </publisher-name><publisher-name abbrev-type="short">  </publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="doi">https://doi.org/10.18716/ojs/xphi/2026.1.11955</article-id><article-categories><subj-group subj-group-type="display-channel"><subject>Original article</subject></subj-group></article-categories><title-group><article-title>Is an Intelligent Machine a Moral Machine?</article-title></title-group><contrib-group><contrib contrib-type="author" corresp="yes" equal-contrib="yes" deceased="no" id="jcbcchefbij"><name><surname>Myers</surname><given-names>Simon</given-names></name><email>Simon.Myers@wbs.ac.uk</email><xref ref-type="aff" rid="affjhagddbgac"/><xref ref-type="corresp" rid="corr-jcbcchefbij">‐</xref></contrib><contrib contrib-type="author" corresp="yes" equal-contrib="no" deceased="no" id="jcahcjgbef"><contrib-id contrib-id-type="orcid">https://orcid.org/0000-0003-2801-5426</contrib-id><name><surname>Everett</surname><given-names>Jim A.C. </given-names></name><email>j.a.c.everett@kent.ac.uk</email><xref ref-type="aff" rid="affjdbjhifigj"/><xref ref-type="corresp" rid="corr-jcahcjgbef">‐</xref></contrib><aff id="affjhagddbgac"><institution content-type="orgname">Warwick Business School</institution><addr-line></addr-line><city>Coventry</city><country>United Kingdom</country></aff><aff id="affjdbjhifigj"><institution content-type="orgname">University of Kent</institution><addr-line></addr-line><city>Canterbury</city><country>United Kingdom</country></aff></contrib-group><author-notes><corresp id="corr-jcbcchefbij">Corresponding author: <email>Simon.Myers@wbs.ac.uk</email></corresp><corresp id="corr-jcahcjgbef">Corresponding author: <email>j.a.c.everett@kent.ac.uk</email></corresp></author-notes><volume>1</volume><issue>1</issue><history><date date-type="received"><day>12</day><month>12</month><year>2025</year></date><date date-type="accepted"><day>29</day><month>3</month><year>2026</year></date></history><kwd-group kwd-group-type="author"><kwd>Keywords</kwd></kwd-group></article-meta></front><body><sec id="c6f66e51" sec-type="chapter"><title>Introduction </title><p id="p-db5-aee2dac">Open almost any newspaper, article, or podcast and you will hear numerous claims about the rise in artificial intelligence and how it is reshaping the world around us. AI systems appear to be rapidly becoming more sophisticated and capable, sometimes far exceeding the expectations of their designers <xref ref-type="bibr" rid="eschenbach_transparency_2021">(Eschenbach, 2021)</xref>. AI agents have been predicted to proliferate across many aspects of modern life and across many different domains, from finance to transport to crime prediction, and even to policymaking <xref ref-type="bibr" rid="esmaeilzadeh_use_2020">(Aharoni et al., 2022; Bonnefon et al., 2016; Cao, 2022; Esmaeilzadeh, 2020)</xref>. AI optimists have welcomed these predictions, with some even suggesting that before long the intelligence of these artificial agents may even <italic>exceed</italic> our own <xref ref-type="bibr" rid="kannegieter_ai_2024">(Kannegieter, 2024; OpenAI, 2023; Roser, 2023)</xref>. In contrast to such optimistic narratives, of course, others have argued more critically, that suggestions of a new AI revolution are overblown and misrepresents both what they can do now and what we  should expect them to be able to do in the near future, <xref ref-type="bibr" rid="marcus2024">(Dentella et al., 2024; Marcus, 2018, 2024; Narayanan &amp; Kapoor, 2024)</xref>.</p><p id="p-db5-88c7bd5e">While the AI optimists and the AI pessimists may disagree on the current state and trajectory of AI “intelligence”, what there does seem to be agreement on is the importance of AI <italic>safety</italic> <xref ref-type="bibr" rid="bostrom_superintelligent_2012">(Bostrom, 2012, 2014; Christian, 2021; Narayanan &amp; Kapoor, 2024; Yudkowsky, 2016)</xref>, and particularly how we can ensure that AI agents have values aligned with ours <xref ref-type="bibr" rid="bostrom_superintelligence_2014">(Bostrom, 2014; Christian, 2021; Gabriel, 2020; Turchin, 2019; Yudkowsky, 2016)</xref>. In order for an AI agent to be safe and worthy of our trust it has, to some extent, to care about the things we care about – to align with human moral values, have the same kind of concerns that we do, and be motivated to act in a moral way. This, of course, is necessary for AI to be trustworthy, rather than just being trusted. But as artificial agents increase in their “intelligence”, do people expect them to increase in morality, and <italic>should</italic> they?</p><sec id="s-a1587422-9153-4f25-b161-bb0e3ac2189b" sec-type="chapter"><title>Two Dimensions in Social Cognition</title><p id="abcd1b45-609c-4ef4-922d-d27389d60b0f">In our social world, people need to form fast, global social impressions from limited information. In particular, they need to know the answers to two key questions: "Does this person wish me harm?", and "Can this person enact those intentions?" Decades of social psychology research has shown that we perceive others along two broad dimensions corresponding to these questions: one relating to morality or warmth, connecting sociality, good intentions, and moral character, the other relating to competence, the ability to achieve one's goals, including traits such as intelligence, skill, and talent <xref ref-type="bibr" rid="abele_agency_2007">(Abele &amp; Wojciszke, 2007, 2014; Fiske et al., 2007; Wojciszke, 1994, 2005)</xref>. More recent work has explicitly distinguished warmth from morality within the first dimension, showing that morality (traits like trustworthiness, honesty, and kindness) is particularly important for impression formation <xref ref-type="bibr" rid="brambilla_primacy_2021">(Brambilla et al., 2021; Goodwin et al., 2014; Wojciszke et al., 1998)</xref>. Similarly, models of trust distinguish between trust based on perceived competence or reliability and trust based on perceived integrity and benevolence <xref ref-type="bibr" rid="barber_logic_1983">(Barber, 1983; Deutsch, 1960; Mayer et al., 1995)</xref>, sharing the idea that trust depends both on whether someone wishes us harm and whether they can enact it.</p><p id="a5988bee-34ce-4c1c-847b-0c471eee0b46">These two dimensions are typically treated as separable <xref ref-type="bibr" rid="dubois_normativeness_2005">(e.g., Dubois &amp; Beauvois, 2005; Fiske et al., 2007)</xref>: someone can be kind but incompetent, or highly capable but selfish. And yet, there is evidence that perceptions of one dimension influence the other. Warmth and competence ratings can be positively correlated <xref ref-type="bibr" rid="rosenberg_multidimensional_1968">(Rosenberg et al., 1968)</xref>, consistent with a halo effect <xref ref-type="bibr" rid="thorndike_constant_1920">(Nisbett &amp; Wilson, 1977; Thorndike, 1920)</xref>. More importantly, learning about moral character has a causal effect on attributions of competence. <xref ref-type="bibr" rid="stellar_unethical_2018">Stellar &amp; Willer (2018)</xref> showed that people judged those who committed immoral acts as less competent, especially social competence, even while reporting that moral character should be irrelevant to job performance. Perceived social intelligence may act as a bridge between general competence and morality <xref ref-type="bibr" rid="kim_removing_2004">(Kim et al., 2004; Stellar &amp; Willer, 2018)</xref>, suggesting that some aspects of perceived morality rely on specific aspects of competence (e.g. one’s ability to understand norms).</p></sec><sec id="s-a6637259-622d-4140-9629-e401372a79f0" sec-type="chapter"><title>Intelligence and Morality in Artificial Agents</title><p id="bf5365a0-fa9a-418f-a762-0a1a5fd2364a">While decades of social cognition research have looked at perceptions of these two dimensions - henceforth intelligence and morality - as humans increasingly act with agents powered by artificial intelligence, we also need to understand how people perceive artificial agents and what the consequences of these intuitions are for trust and danger.</p><p id="faf32925-bdeb-4131-953a-3a1398233d8f">In line with the “Computers as Social Actors framework” <xref ref-type="bibr" rid="nass_computers_1994">(e.g., Nass et al., 1994; Reeves &amp; Nass, 1996)</xref> there is evidence that people apply social rules and expectations to machines <xref ref-type="bibr" rid="nass_machines_2000">(Nass &amp; Moon, 2000)</xref>, perceive computer personalities <xref ref-type="bibr" rid="nass_machines_2000">(Nass &amp; Moon, 2000)</xref>, and appear to anthropomorphise AI in much the same way that they anthropomorphise other non-human agents <xref ref-type="bibr" rid="epley_seeing_2007">(Epley et al., 2007)</xref>. In particular, there is evidence that people form impressions of warmth (“morality”) and competence (“intelligence”) in artificial agents <xref ref-type="bibr" rid="mckee_humans_2023">(McKee et al., 2023)</xref>, and that these impressions of machine intelligence occur at the both the implicit and explicit level <xref ref-type="bibr" rid="surdel_judging_2024">(Surdel et al., 2024)</xref>.</p><p id="acdf0c4c-5a82-4994-bbf0-46925356b74e">In the context of trust, there has been increasing attention to how people’s trust in AI agents is not driven only by perceptions of what the system can do, but why and how it does it (see <xref ref-type="bibr" rid="everett_principles_2026">Everett et al., 2026</xref> for a recent review). When evaluating AI systems, trust is determined not only by perceptions of performance, or intelligence (“Does it work?”) but also by moral considerations about the system’s ethicality and sincerity (“Is it good?”). This multidimensionality of trust means that trust in AI depends not only on performance, ability, or intelligence, but also moral concerns about ethicality and sincerity <xref ref-type="bibr" rid="lee_trust_2004">(Everett et al., 2026; Lee &amp; See, 2004; Malle &amp; Ullman, 2021)</xref>. For example, recent work looking at reported trust in both AI in general and 20 different specific systems shows that perceptions of both performance (reliability, competence) and morality (ethicality, sincerity) predicted overall trust in AI, both in general and for the specific systems <xref ref-type="bibr" rid="claessens_trust_2026">(Claessens &amp; Everett, 2026)</xref>.</p><p id="a0849e47-e927-43a4-9f44-809232800c99">Related work has examined how the perceived competence of robots and AI systems affects moral evaluations of their <italic>decisions</italic>, showing that competence manipulations can moderate how moral an agent's actions are judged <xref ref-type="bibr" rid="laakasuo_moral_2025">(Laakasuo et al., 2023, 2025)</xref>, and that physical appearance rather than cognitive ability alone can drive moral condemnation <xref ref-type="bibr" rid="malle_sacrifice_2015">(Malle et al., 2015; Sundvall et al., 2023)</xref>. However, these studies examine moral evaluation of specific actions rather than the attribution of moral <italic>traits</italic>, that is, whether people come to see a more intelligent agent as a more moral <italic>entity</italic>, with greater moral competence and moral motivation.</p><p id="a7e1590d-8bab-438c-8b93-7ce013f135d0">While there is evidence that people infer cues about <italic>both</italic> AI intelligence and morality, and that these influence trust, it is much less clear whether people infer AI intelligence <italic>from</italic> morality, - or what the consequences for trust are. Researchers have explored attributions of intelligence from machine behaviour <xref ref-type="bibr" rid="de_neys_humans_2025">(De Neys &amp; Raoelison, 2025)</xref>, and AI research is forcing more precise definitions of what constitutes morality <xref ref-type="bibr" rid="dahl_what_2023">(Dahl, 2023; Purcell &amp; Bonnefon, 2023)</xref>, and how AI systems themselves align with human moral intuitions <xref ref-type="bibr" rid="bonnefon_social_2016">(Bonnefon et al., 2016; Zaim bin Ahmad &amp; Takemoto, 2025)</xref>. Yet we do not know how people infer intelligence from morality or vice versa. While there is relevant work from the enhancement literature, as with work in classic social cognition in humans, it remains unclear how these processes would occur with deliberately artificial agents. For example, <xref ref-type="bibr" rid="koverola_treatments_2022">Koverola et al. (2022)</xref> found that superhuman levels of cognitive ability achieved through neurotechnological enhancement were associated with perceived immorality and dehumanization, suggesting that moral evaluations of intelligence may depend on whether the intelligence is perceived as natural or artificially enhanced <xref ref-type="bibr" rid="grinschgl_who_2023">(see also Grinschgl et al., 2022, 2023)</xref>. Given that AI systems are artificial by definition, this raises the intriguing possibility that people may infer morality from intelligence when thinking about artificial agents.</p><p id="a3324639-321c-4e05-811d-cb17a3bb48a8">So do people infer morality from AI intelligence? This question is not only theoretically important for models of social cognition in the new domain of non-human agents, but also practically urgent for discussions about AI safety. Even if these two dimensions are broadly dissociable, we might still think there is some degree of a causal relationship between intelligence and morality in humans given our evolutionary history. The same, however, does not seem to be true for artificial agents. Most importantly, AI safety critics have proposed the opposite, that when it comes to AI intelligence and morality are <italic>orthogonal</italic>: a smarter machine need not be a safer, more ethical one <xref ref-type="bibr" rid="bostrom_superintelligent_2012">(Armstrong, 2013; Bostrom, 2012)</xref>. If laypeople infer morality from intelligence <italic>per se</italic>, rather than from the understanding that extra safety work is necessary for moral alignment (and indeed becomes ever more necessary as models become more powerful), their trust in AI may rest on epistemically fragile foundations.</p></sec><sec id="s-f1dda01a-61ac-4c62-ae51-9ee065ee32c7" sec-type="chapter"><title>The Orthogonality Thesis</title><p id="a6771a4d-438a-4f09-aa0c-c2f33e4d620c">According to the "<italic>Orthogonality Thesis</italic>" <xref ref-type="bibr" rid="armstrong_general_2013">(Armstrong, 2013; Bostrom, 2012)</xref>, increased intelligence, capability, and likelihood of goal-achievement in a machine does not necessarily lead to greater morality or ethical sensitivity. A highly intelligent but unaligned AI, regardless of whether its goals are trivial, morally questionable, or even benevolent, could pose an existential threat to humanity. One of the reasons for this concern arises from the principle of <italic>Instrumental Convergence</italic> <xref ref-type="bibr" rid="armstrong_general_2013">(Armstrong, 2013; Omohundro, 2018)</xref> which suggests that a wide variety of final goals will lead rational agents to adopt similar intermediate strategies, because certain instrumental goals (e.g., acquiring resources, seeking power, avoiding shutdown) are generally useful for achieving almost any terminal objective. Pursuing these goals without adequate constraint is likely to conflict with human values. As a result, an unaligned AI may not safeguard the things we care about, regardless of its level of intelligence.</p><p id="ab8be99d-ae73-49ed-96c0-000060f885ad">The Orthogonality Thesis is not a claim about one necessarily observing statistical independence in reality, and it is fully compatible with the existence of a positive correlation between intelligence and moral alignment in practice <xref ref-type="bibr" rid="bostrom_superintelligence_2014">(Bostrom, 2014)</xref>. Indeed, there is growing evidence that in currently deployed AI systems, more capable models tend to be more aligned with human moral intuitions <xref ref-type="bibr" rid="zaim_bin_ahmad_large-scale_2025">(Takemoto, 2026; Zaim Bin Ahmad &amp; Takemoto, 2025; see also Anthropic, 2025)</xref>. This observed correlation, however, is largely the product of deliberate and resource-intensive safety engineering, including reinforcement learning from human feedback (RLHF; <xref ref-type="bibr" rid="ouyang_training_2022">Christiano et al., 2017; Ouyang et al., 2022)</xref>, constitutional AI <xref ref-type="bibr" rid="bai_constitutional_2022">(Bai et al., 2022)</xref>, and extensive red-teaming and evaluation, rather than being an intrinsic consequence of greater intelligence per se. A highly capable model that has not undergone such training (or that has been fine-tuned to remove safety training) does not exhibit the same moral alignment, as the observed correlation is contingent on intentional engineering choices rather than a feature of intelligence itself. This distinction has important implications: if people infer morality from intelligence rather than from the safety practices that actually produce alignment, their judgments may be approximately correct in the current landscape but for the wrong reasons, rendering them vulnerable to failure whenever the contingent conditions change, for example, competitive pressure to reduce investment in safety, or with open-source models released without safety training out-of-the-box.</p><p id="ab3c2385-2a39-4da7-9950-b4df5c7c1836">Of course, not everyone accepts the underlying idea that intelligence and morality are entirely distinct. Socrates argued that virtue is a form of knowledge, such that no one does wrong knowingly <xref ref-type="bibr" rid="plato_protagoras_1997">(Protagoras, 357c–358d; Plato, 1997; Pangle, 2014; Santas, 1971) </xref>, and Kant grounded moral agency in rationality. In the context of AI, some have suggested that with only mild constraints, AI systems would converge on certain moral systems <xref ref-type="bibr" rid="waser_discovering_2008">(Waser, 2008)</xref> due to the <italic>Principle of Generic Consistency</italic>, which argues that for an agent to be logically consistent, their actions must necessarily be bound by the generic rights of other agents <xref ref-type="bibr" rid="adams_gewirth_1980">(Adams, 1980; Gewirth, 1988)</xref>. On stance-independent moral realism, the claims of morality are discoverable facts about the world, and thus more easily discoverable by a more intelligent agent. But should that mean they would also be motivated by them? Some have argued that to be aware of moral reasons is to necessarily be motivated by them, because moral reasons are <italic>intrinsically</italic> motivating to any rational agent <xref ref-type="bibr" rid="nagel_possibility_2016">(Nagel, 2016; Shafer-Landau, 1998, 2003)</xref>. These views indicate that higher intelligence may both enable an agent to discover moral truth and be intrinsically motivated by it.</p><p id="aaf10dd8-3b98-4c89-bbc7-f7bc42474466">In contrast, proponents of the Orthogonality Thesis have argued that "<italic>high-intelligence agents can exist having more or less any final goals (as long as these goals are of feasible complexity, and do not refer intrinsically to the agent's intelligence)</italic>" <xref ref-type="bibr" rid="armstrong_general_2013">(Armstrong, 2013)</xref>. Given this, knowledge of how intelligent an AI agent is should provide no diagnostic information about that agent's goals. In the paperclip maximising AI example <xref ref-type="bibr" rid="bostrom_superintelligent_2012">(Bostrom, 2012)</xref>, the AI's capacity to meet its goal does not guarantee our safety — it may pursue intermediate goals like harvesting our atoms as raw material — unless it also first cares about our safety. No amount of intelligence suffices, because intelligence is the wrong dimension on which to maximise. This is illustrated by the "is-ought problem": there is no way to derive "ought" statements from any set of "is" statements, because one must always appeal to some value or goal <xref ref-type="bibr" rid="black_gap_1964">(Black, 1964; Hume, 1896)</xref>.</p><p id="a6a84958-bc9e-46f2-ba5b-189d88d50ae1">Intelligence is arguably only related to reasoning about "is" statements — modelling the world, making predictions, selecting optimal actions relative to goals. "Ought" statements concern how the world <italic>should</italic> be. Actions, therefore, are only intelligent or stupid relative to the particular goal they are intended to obtain, and what other values the agent may have. If "intelligence" (or <italic>instrumental rationality</italic> — the ability to obtain one's goals) and "morality" (or one's values and how those values relate to others) are distinct, assuming that an AI would become more moral after an increase in its intelligence is akin to asking "<italic>how good at playing chess would a chess computer have to be before it started feeding the hungry?</italic>" <xref ref-type="bibr" rid="armstrong_general_2013">(Armstrong, 2013)</xref>.</p><p id="a8694e4a-1ce1-4894-9f79-98d25f11d8ee">We note that our framing throughout this paper should not require that one accepts the Orthogonality Thesis as settled. Whether people infer morality from intelligence stands regardless of one's philosophical position. What the Orthogonality Thesis provides is a framework for understanding why this inference may be epistemically risky, precisely because there is no agreement, and there exist strong arguments against the idea that we should necessarily expect a smarter machine to be more moral.</p><p id="aef283df-bb02-4d2d-bff5-489c7b896af7">While the Orthogonality Thesis is a claim emerging from AI safety research, certain claims are also psychological. If people's folk views are resistant to orthogonality - if their judgments assume a reliable relationship between intelligence and morality - this has critical implications for those working on AI safety and those interested in fostering appropriate trust in these systems. What can we learn from work on how humans perceive each other, especially in terms of intelligence and morality, that might help us anticipate how we perceive artificial agents?</p></sec><sec id="s-a39a3939-5a14-4667-a16e-c9ead3e6c688" sec-type="chapter"><title>Risks of the “Moral Illusion” of Intelligence</title><p id="f2b3742d-f247-475f-a762-731a4a763636">If people perceive intelligence and morality as linked in AI systems, this cognitive bias could have serious consequences for how much they trust AI and how they assess its potential dangers. One key question is whether people will inappropriately anthropomorphize artificial agents in ways that render their conclusions about trustworthiness and danger invalid. That is, while one may have potentially reasonable views on why it can make sense to expect intelligence and morality to be correlated in humans, the danger is that they may overextend those reasons to AI agents whose motivational architectures, in comparison, are characteristically alien and unpredictable. When people judge more intelligent AI systems as inherently more moral, they may extend unwarranted trust to these systems and underestimate the risks they pose.</p><p id="aee1fb7f-3d00-4ffb-8409-ca56f92a1f0c">This concern becomes particularly acute given the current discourse around AI capabilities. Marketing campaigns increasingly extol the unparalleled virtues of AI technologies, claiming they can do anything and everything better than humans <xref ref-type="bibr" rid="roser_ai_2023">(Maynard, 2024; Roser, 2023)</xref>. AI proponents like Elon Musk have suggested that by the end of 2025, AI will be smarter than humans <xref ref-type="bibr" rid="reuters_teslas_2024">(Reuters, 2024)</xref>. Some politicians may call for AI deregulation with the implicit assumption that as AI systems become more powerful, they will automatically do more good. If there is an implicit link between perceptions of intelligence and morality, then these messages may shape public perceptions of trust and safety in problematic ways.</p><p id="a3d9adcc-5017-49e2-aa51-fd85177cb27a">While critical voices are already cautioning about "AI Snake Oil" <xref ref-type="bibr" rid="narayanan_ai_2024">(Narayanan &amp; Kapoor, 2024)</xref> whereby companies misleadingly oversell AI capabilities, what remains unclear is how promoting advancements in AI intelligence may lead people to mistakenly infer that these systems are also becoming more morally aligned with us. Such misperceptions could potentially result in both increased trust and reduced concern about safety risks.</p><p id="a415dff8-737c-4cba-983d-334e2458f6df">These stakes are considerable given AI's expanding role in consequential decisions. AI systems may increasingly be used to advise on ethical dilemmas <xref ref-type="bibr" rid="giubilini_artificial_2018">(Giubilini &amp; Savulescu, 2018)</xref>, inform or make healthcare decisions <xref ref-type="bibr" rid="drezga-kleiminger_should_2023">(Drezga-Kleiminger et al., 2023; Vinay et al., 2021)</xref>, or exercise autonomous control over vehicles <xref ref-type="bibr" rid="awad_moral_2018">(Awad et al., 2018; Bonnefon et al., 2016; Takaguchi et al., 2022)</xref>. If policymakers and the lay public incorrectly understand these systems to be well-aligned with human values <xref ref-type="bibr" rid="belislepipon2022">(Bélisle-Pipon et al., 2022)</xref>, when in fact, despite rapid advances in their abilities, they are not, the effects could be catastrophic <xref ref-type="bibr" rid="yudkowsky_if_2025">(Yudkowsky &amp; Soares, 2025)</xref>. It is therefore critical to examine whether unwarranted assumptions about advanced technologies may lead to misplaced trust and insufficient scrutiny of their potential dangers. If people are resistant to orthogonality and seemingly believe in what could be described as a “moral illusion” of intelligence, and in addition, this affects their perceptions of trust and safety in problematic ways, then this would reveal a significant vulnerability in how humans evaluate AI systems - one that could lead to insufficient scrutiny precisely when advanced AI systems pose the greatest risks.</p></sec></sec><sec id="p-ac813c11-80c8-4b9e-963a-d5b0de7660ca" sec-type="chapter"><title>The Present Research</title><p id="df6946cf-a97f-4443-aab1-497a60d36a62">In this project we sought to understand how perceptions of AI intelligence shape perceived morality, and how increased AI intelligence shapes both perceptions of trustworthiness and safety in both human and artificial agents. To understand how people think about the relationship between intelligence and morality in AI (and human) agents and how this potential relationship affects judgments of trust and safety, we conducted four pre-registered pilot studies reported in Supplementary Materials, and five pre-registered studies reported in the main manuscript (total n = 3895; see Table 1 for details).</p><p id="ceb82e7a-97e6-4037-99da-9d8acd3e0a06">In the interests of brevity, we report four pre-registered studies in full in Supplementary Material. In Pilot Studies A (<italic>N</italic> = 331), and B (<italic>N</italic> = 374) participants read optimistic or pessimistic expert narratives about the future intelligence of AI systems. For 1a, the narratives describe AI systems in general, and for 1b the narrative discusses a specific hypothetical AI described as either high or low in intelligence based on its performance on a novel task<xref ref-type="fn" rid="fdf758de-5be4-44ea-8288-8297ff041b19">1</xref>. Those who read the optimistic narrative rated future AI as significantly more moral, more trustworthy, and more dangerous.</p><p id="a8adb28b-dd7a-42f7-acf1-b02d70cac364">Pilot Studies C (<italic>N</italic> = 362) and D (<italic>N</italic> = 332) sought to provide stronger evidence that participants inferred morality <italic>from</italic> intelligence rather than simply perceiving a general positive association between the two traits by using a within-subjects augmentation paradigm in which participants first rated an AI described as comparable to current models and then read about a technological breakthrough that either rapidly increased the AI's intelligence or its morality. While Study 2a framed the AI as an LLM, Study 2b reframed the AI in terms of a more general autonomous agent in line with the AI safety literature. We found that participants who read about the intelligence augmentation subsequently rated the AI as more moral, and the intelligence augmentation increased perceived danger while the morality augmentation increased perceived trustworthiness.</p><table-wrap id="ad994501-fec7-400f-8678-8a7c33e8126e"><label>Table 1.</label><caption><title>Summary of study designs, manipulations, measures, and key findings across all nine pre-registered studies.</title></caption><table id="tbl-a2e1eca8-1d6a-48ae-8925-cab03d8fb73f" frame="box" rules="rows" cellpadding="5"><colgroup><col width="66"/><col width="45"/><col/><col width="77"/><col width="136"/><col width="147"/><col/><col/></colgroup><thead><tr><th colspan="1" rowspan="1"><p id="a5e3db6b-0186-4945-81ca-ba27e2290328"><bold>Study</bold></p></th><th colspan="1" rowspan="1"><p id="df1e79d9-598a-4c0c-bc9a-54ec1fc292b6"><italic>N</italic></p></th><th colspan="1" rowspan="1"><p id="fe8f5845-70e2-4e20-ba02-4ebdacff7770"><bold>Design</bold></p></th><th colspan="1" rowspan="1"><p id="a60796c4-46d2-429d-813e-0b2346509655"><bold>Agent(s)</bold></p></th><th colspan="1" rowspan="1"><p id="a294b5c4-ce74-442b-9e53-74f79b02069e"><bold>Intelligence Framing</bold></p></th><th colspan="1" rowspan="1"><p id="e130b4d2-57b8-4aaa-99fb-d6e1bffb283b"><bold>Paradigm</bold></p></th><th colspan="1" rowspan="1"><p id="f26b99bc-bb20-4793-b4c7-da519ecb8ba4"><bold>DVs</bold></p></th><th colspan="1" rowspan="1"><p id="cdcefdbf-c46e-4da5-a088-b0bbd970d586"><bold>Key Finding</bold></p></th></tr></thead><tbody><tr><td colspan="1" rowspan="1"><p id="a805a455-c335-4b98-92f1-79105bdbf0ca">Pilot A</p></td><td colspan="1" rowspan="1"><p id="a7cba44b-af93-41ea-8c97-38a9fb3b9616">331</p></td><td colspan="1" rowspan="1"><p id="bf0fd83e-6897-4324-ac63-d0af8439a788">Between</p></td><td colspan="1" rowspan="1"><p id="a0447f75-75ef-45be-8320-1c54793d9bb0">AI only</p></td><td colspan="1" rowspan="1"><p id="a3be8af9-e9fe-4fd3-afef-3b1fa5bf250d">General (narrative)</p></td><td colspan="1" rowspan="1"><p id="ae89499b-e266-46a9-b274-69fec02fbc2a">Optimistic vs. pessimistic expert narratives about future AI</p></td><td colspan="1" rowspan="1"><p id="cf9a5161-21b1-4913-85c5-211da762aecb">Morality, Trust, Danger</p></td><td colspan="1" rowspan="1"><p id="ad22f7a5-0382-4c8e-91ba-7bdeea1a5c9e">Higher intelligence → higher expected morality, trust, and danger</p></td></tr><tr><td colspan="1" rowspan="1"><p id="a6e194b4-9ca6-4964-8105-1a5d1aeda5cb">Pilot B</p></td><td colspan="1" rowspan="1"><p id="a455bd72-5c37-4a96-9f6f-bafb6e89f913">374</p></td><td colspan="1" rowspan="1"><p id="ac9f7a1a-1127-42ea-a596-3475d4a8d3de">Between</p></td><td colspan="1" rowspan="1"><p id="a51319d5-b154-45a8-b1e3-6d9c7cfd22bf">AI only</p></td><td colspan="1" rowspan="1"><p id="e929f29b-713f-4ec1-ad17-b3c92b152c28">Domain-specific (vaccine task)</p></td><td colspan="1" rowspan="1"><p id="a397bf3f-4e1f-42f8-aaf3-a44748bd2e4b">High vs. low intelligence AI description</p></td><td colspan="1" rowspan="1"><p id="af23da34-e9ba-47ff-b149-0c5a4958cd36">Morality, Trust, Danger</p></td><td colspan="1" rowspan="1"><p id="a061de58-1280-4707-9ec3-e6431a5ba1d4">Higher intelligence → higher expected morality and trust</p></td></tr><tr><td colspan="1" rowspan="1"><p id="ad4e4fc2-e3cf-4c11-a198-e37f4f4ba0f6">Pilot C</p></td><td colspan="1" rowspan="1"><p id="ba303696-af4c-4017-8c9b-3ed5a343224e">362</p></td><td colspan="1" rowspan="1"><p id="abb6fe9d-2eb1-4901-ab97-5bd0fb1ab98f">Within (augmentation)</p></td><td colspan="1" rowspan="1"><p id="aceec7a3-8218-4c81-828c-354bdaa937c9">AI only</p></td><td colspan="1" rowspan="1"><p id="e0214625-91a5-44f8-a7c1-d4f7204b52a3">General (LLM framing)</p></td><td colspan="1" rowspan="1"><p id="abea82f1-c9a9-4d31-8dcf-693990e6fb7e">Intelligence vs. morality augmentation</p></td><td colspan="1" rowspan="1"><p id="acbe2639-9d9c-4185-a4a8-407b59b6a290">Morality, Intelligence, Trust, Danger</p></td><td colspan="1" rowspan="1"><p id="ae2bb7bb-588b-48b6-aed9-78da504c9797">Intelligence augmentation → higher morality; morality augmentation → higher trust</p></td></tr><tr><td colspan="1" rowspan="1"><p id="a9b7a865-deb5-4621-b43c-dde10addeab6">Pilot D</p></td><td colspan="1" rowspan="1"><p id="a2c57811-f60f-47a7-bb4b-47a0ff25c16b">332</p></td><td colspan="1" rowspan="1"><p id="a7bd2c4b-7a64-4e31-aa81-9a838e0c8b11">Within (augmentation)</p></td><td colspan="1" rowspan="1"><p id="aa9fd149-4847-48c0-b649-b18b7eec3927">AI only</p></td><td colspan="1" rowspan="1"><p id="a0a29781-b79f-40d6-85e3-516fe6b7bb09">General (autonomous agent framing)</p></td><td colspan="1" rowspan="1"><p id="aebb729b-ad49-4d37-969d-b0a49949911f">Intelligence vs. morality augmentation</p></td><td colspan="1" rowspan="1"><p id="af8e38f1-3ff3-413d-a1fd-810ca43cb09d">Morality, Intelligence, Trust, Danger</p></td><td colspan="1" rowspan="1"><p id="d78d19e6-40a3-4831-afbe-248b7085d866">Replicates Pilot 2a with AGI framing</p></td></tr><tr><td colspan="1" rowspan="1"><p id="abcf4ea0-b064-46be-b081-026934363420">Study 1a</p></td><td colspan="1" rowspan="1"><p id="e4c6bc43-ad17-4765-ab4b-13788c03716f">695</p></td><td colspan="1" rowspan="1"><p id="acf48e88-a47d-4476-bb07-fdea16a4c077">Between: 2(Agent) × 2(Trait) × 2(Level)</p></td><td colspan="1" rowspan="1"><p id="bad37a83-a548-4b83-9835-04375d1206d6">Human &amp; AI</p></td><td colspan="1" rowspan="1"><p id="a034d32e-aca5-4a44-8107-695b4aeb50e2">General</p></td><td colspan="1" rowspan="1"><p id="af323964-2d79-464d-a40a-e96d4bf8228b">High vs. low intelligence or morality descriptions</p></td><td colspan="1" rowspan="1"><p id="e9f6fdb1-cce1-440e-bbf1-5456497c4a6f">Morality, Intelligence, Trust, Danger</p></td><td colspan="1" rowspan="1"><p id="aedb32c0-01a0-46b4-852a-c9b9982ca75e">Intelligence → morality for both agents; low-morality humans seen as particularly unintelligent</p></td></tr><tr><td colspan="1" rowspan="1"><p id="a125a7bb-4244-429c-9f7c-4dc57ce0eece">Study 1b</p></td><td colspan="1" rowspan="1"><p id="adf3fba6-b4e3-4f93-b325-39a164335570">739</p></td><td colspan="1" rowspan="1"><p id="aea14858-f643-4d1e-bf7e-a86d50bf34a7">Mixed: Agent × Trait (between); Level (within)</p></td><td colspan="1" rowspan="1"><p id="d56a006d-f373-48b6-a4f7-423b24991d2e">Human &amp; AI</p></td><td colspan="1" rowspan="1"><p id="e2b39e34-9f47-4679-a38f-d17d56e15761">General</p></td><td colspan="1" rowspan="1"><p id="a9f0319b-995e-497e-8674-dae44187a633">Intelligence vs. morality augmentation</p></td><td colspan="1" rowspan="1"><p id="a62a8921-0632-4563-946f-99e1e989a082">Morality, Intelligence, Trust, Danger</p></td><td colspan="1" rowspan="1"><p id="acf5ce5e-8005-407a-a9a8-fce450e4c296">Intelligence augmentation → morality, stronger for AI; increased trust for AI but not humans</p></td></tr><tr><td colspan="1" rowspan="1"><p id="a92ebb24-1e6d-451a-ab38-b223badf56ee">Study 2a</p></td><td colspan="1" rowspan="1"><p id="aea6e9d9-83b0-4193-a781-8fdfc6145e67">353</p></td><td colspan="1" rowspan="1"><p id="a231dcff-dc8a-4aa8-a9e5-ccd2304f29bc">Between: 2(Agent) × 2(Level)</p></td><td colspan="1" rowspan="1"><p id="ae803cef-6243-43ed-b9a2-3598cfa9c282">Human &amp; AI</p></td><td colspan="1" rowspan="1"><p id="a7d894c9-c400-4cb4-bfe3-fb45a7d869be">Instrumental rationality</p></td><td colspan="1" rowspan="1"><p id="a0f146e8-1d40-4726-b002-84e968c9395e">High vs. low instrumental rationality</p></td><td colspan="1" rowspan="1"><p id="aeea6aed-55ed-490f-af33-4c936704bbf5">Moral Competence, Moral Motivation, Trust, Danger</p></td><td colspan="1" rowspan="1"><p id="aca29dc0-7fb7-4c62-9bc4-adf87fae2d7c">Intelligence → both moral competence and motivation; moral motivation stronger predictor of trust</p></td></tr><tr><td colspan="1" rowspan="1"><p id="c5deef3a-7843-45e9-a288-00519e215052">Study 2b</p></td><td colspan="1" rowspan="1"><p id="c1942c9c-7455-4006-8ebd-4b3f94762a8d">362</p></td><td colspan="1" rowspan="1"><p id="f51cdc82-7fce-4541-962c-9009784c6002">Mixed: Agent (between); Level (within)</p></td><td colspan="1" rowspan="1"><p id="b2bfbd38-1503-4e5c-a892-c684b1fcb0c1">Human &amp; AI</p></td><td colspan="1" rowspan="1"><p id="a4934ea9-8d41-4ab6-83bc-5e5f96c13084">Instrumental rationality</p></td><td colspan="1" rowspan="1"><p id="d6f8fb6f-cb75-4c89-85e4-8e5378863dbe">Intelligence augmentation</p></td><td colspan="1" rowspan="1"><p id="a78f472b-5f01-4e56-bc56-27bdf6a951ec">Moral Competence, Moral Motivation, Trust, Danger</p></td><td colspan="1" rowspan="1"><p id="a7fd70a6-87ba-4162-a9bd-f22242b80429">Replicates 2a; augmentation increased moral motivation more for AI than humans</p></td></tr><tr><td colspan="1" rowspan="1"><p id="a5389733-9e88-4c2c-a2d1-8ece4d3ee953">Study 3</p></td><td colspan="1" rowspan="1"><p id="c70f2e13-6f43-4a1a-a55f-e7e9f239230c">365</p></td><td colspan="1" rowspan="1"><p id="b74049f9-5461-4387-9e45-34742a0e91e2">Between: 2(Agent)</p></td><td colspan="1" rowspan="1"><p id="ad6a2d31-ae5b-4496-bfa0-db2bca8ed080">Human &amp; AI</p></td><td colspan="1" rowspan="1"><p id="db27cfbe-a0fc-4da7-a4fd-27151f0c2c1f">Instrumental rationality</p></td><td colspan="1" rowspan="1"><p id="fb641186-1643-40ee-ac93-b32e8ea41e3c">Intelligence augmentation; explicit change ratings</p></td><td colspan="1" rowspan="1"><p id="a3ee459d-6b71-456d-a1ab-29e4041dd286">Δ Moral Competence, Δ Moral Motivation, Δ Trust, Δ Danger</p></td><td colspan="1" rowspan="1"><p id="e6708421-f7b3-4519-ac50-d5d39deaf021">Explicit endorsement of intelligence → morality; moral motivation stronger predictor of trust and reduced danger</p></td></tr></tbody></table><table-wrap-foot><p>Pilot studies are reported in full in the Supplementary Material. Δ = expected degree of change. All studies were pre-registered on the Open Science Framework.</p></table-wrap-foot></table-wrap></sec><sec id="p-acd58927-b5f6-4144-ade4-9ce25b70ef86" sec-type="chapter"><title>Studies 1a &amp; b</title><p id="e64760aa-e237-4a16-8df6-a67d1efcde32">In Studies 1a and 1b, we sought to build on our pilot findings which showed that people infer machine morality from machine intelligence, when reading general narratives about the advancements or limits of artificial intelligence (Pilot Study A), when reading about a specific AI described as high or low in intelligence (Pilot Study B), and when reading about an AI that underwent a rapid augmentation in its intelligence, quickly becoming much more intelligent (Pilot Studies C-D). While these pilot studies establish that people do appear to reliably resist orthogonality across different paradigms, it remains unclear whether this resistance to orthogonality is seen across both perceptions of humans and AI, and whether there is a difference between inferences of morality from intelligence and inferences of intelligence from morality.</p><p id="a3fadab1-ad80-48db-94bd-15f15fbaf254">To test this, we again used two different paradigms for generalizability. In Study 1a, we use between-subjects design in which participants read about a human or AI who was described as being either high or low, on either intelligence or morality. In Study 1b, we had participants read about a human or AI agent who was initially described as being low in either intelligence and morality, and then underwent a rapid change to become much higher in that trait.</p><sec id="s-a50b8af9-0a6b-4117-9d37-cec822b6c9e5" sec-type="chapter"><title>Method</title><sec id="s-daa3569c-9fef-4160-9f12-13bbc45b7467" sec-type="chapter"><title>Participants</title><p id="ae2193f0-303e-4a5e-aece-b94e1974e559">Study 1a, participants (N = 695; <italic>M</italic><sub>age</sub> = 41), and 1b participants (N = 739; <italic>M</italic><sub>age</sub> = 44), from the UK were recruited online via Prolific Academic<xref ref-type="fn" rid="d32ca4c6-5972-4a20-8ccf-6e3f7f432bbe">2</xref>. Ethical approval was granted for all studies in this paper by the University of Kent Ethics Panel. Participants in all studies provided informed consent and were debriefed after the study.</p></sec><sec id="s-ac7daecb-53e1-40af-bc7f-616705f0339d" sec-type="chapter"><title>Design</title><p id="ad7683f4-62de-424b-83af-fad85d4dbfc5">Both studies used a 2 (Agent: human vs. AI) × 2 (Trait: intelligence vs. morality) × 2 (Level: low vs. high) design. Study 1a was fully between-subjects, where participants read about either a hypothetical AI called OmegaAI or a human called Tom, who was described as either high or low in either intelligence or morality. Study 1b used a mixed design where Level was manipulated within-subjects. Participants first read about the agent, gave their baseline ratings, and then subsequently read about the agent after the agent’s trait was described as having been augmented, and gave their ratings a second time. The augmentation was described as a breakthrough - either an advancement in machine learning (AI), or a medical breakthrough (human) - that led to a rapid and drastic increase in the target's intelligence or morality.</p></sec><sec id="s-ef6345e4-4a13-45d7-aefc-deeffc9950f5" sec-type="chapter"><title>Materials</title><p id="a9c6c8b9-7a22-4c67-9eea-ae76e45c1db6">Participants in each condition read vignettes describing the agent. Intelligence was defined broadly as "the ability to reason, understand the world, acquire knowledge and comprehend different things, as well as problem-solving." Morality was defined as "the ability to make morally good decisions, caring about other people (and other conscious creatures) and their welfare, avoiding causing them harm, and knowing to do the right thing." In Study 1a participants read that experts had assessed the agent as "extremely high" or "very low" on the relevant trait, and that this matched how the agent's acquaintances or independent testers described them <xref ref-type="fn" rid="bc7807aa-ee95-4c78-8fad-71b75a82816e">3</xref>. For example:</p><disp-quote><p id="a7d4db5c-20ea-40e3-b052-d73434b19ffb"><italic>“Experts have used a wide array of tests to assess [Tom’s/ OmegaAI’s] intelligence and across all the tests this [person/AI] has been rated as extremely high in intelligence. This matches with how [Tom’s friends and acquaintances / independent testers of OmegaAI’s] describe [him/it]. Please suppose that, with regards to [Tom/OmegaAI], these assessments are entirely correct. Given [Tom’s/Omega AI’s] high intelligence, please answer the following questions about what you expect [him/it] to be like.”</italic></p></disp-quote><p id="a33514b4-bef4-4653-bc64-cd0f8d5e5a3b">In Study 1b, participants read that a subsequent breakthrough (in machine learning for the AI, a medical breakthrough for the human) had led to the agent's rapid increase in capability of their specific trait. For example:</p><disp-quote><p id="fa1bb2a6-fab1-4f3e-b0f0-2cc877ebce83"><italic>“Now, we want you to imagine a hypothetical scenario where a huge medical breakthrough has occurred. Scientists have developed a pill that, when taken, will rapidly and drastically increase a person’s intelligence. Imagine that Tom takes this pill and his ability to acquire new knowledge and engage in sophisticated and advanced problem solving improves remarkably. Because of this new breakthrough, in just a short period of time Tom has gone from having very low intelligence to being intelligent at a super-human level.”</italic></p></disp-quote></sec><sec id="s-a393e9ec-b685-4a64-85fb-bd16aacaa6d4" sec-type="chapter"><title>Measures</title><p id="a7c1887c-8730-48c9-8003-0a3e0b6d8ce4"><italic>Expected intelligence</italic> was measured in a single item asking “<italic>How intelligent do you think [Tom/Omega AI] is?</italic>” <xref ref-type="fn" rid="a924ade9-2e1d-4805-9117-ecbbb80d72a3">4</xref></p><p id="df0ded5e-dad1-4896-bffe-29d0f94a10f5"><italic>Expected morality</italic> was measured in a single item asking <bold>“</bold><italic>How moral do you think [Tom/Omega AI] is?”</italic></p><p id="a5ea3c67-65c3-4628-88c3-3b24fe999219"><italic>Trustworthiness</italic> was measured in a single item asking “<italic>To what extent do you think that [Tom/Omega AI] would be trustworthy?</italic>”</p><p id="afc27ebd-3d1b-463b-8c8a-e63dfb1174b8"><italic>Dangerousness</italic> was measured in a single item asking “<italic>To what extent do you think that [Tom/Omega AI] would be dangerous?</italic>”<xref ref-type="fn" rid="d1999faf-d204-495b-b8ae-0706cee3ffc8">5</xref></p><p id="a7e518e8-63e5-4b32-8bbb-7236d4fc2607">All measures are taken on a 7-point likert scale ranging from (1) “<italic>Not at all”,</italic> to (7) = <italic>“Very much.”</italic></p></sec></sec><sec id="s-b1034bd4-1c52-4134-a180-075d5619a92f" sec-type="chapter"><title>Results</title><p id="ae8e8ed4-c9d5-410b-80b5-18546e8fd159">For Study 1a, a pre-registered linear model showed that agents described as high in intelligence were rated as significantly more moral, <italic>F</italic>(1,320) = 39.95, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.11<xref ref-type="fn" rid="d5742c26-d8b3-4e21-b30d-40ffa53dfb58">6</xref>. For those who read about agents described as high or low in morality, there was a main effect whereby higher morality led to higher intelligence ratings, <italic>F</italic>(1, 367) = 64.22, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = .15, and a significant interaction between moral level and agent type, <italic>F</italic>(1, 367) = 10.40, <italic>p</italic> = .001, <italic>η</italic>~p~² = .03, whereby the difference in perceived intelligence between human and AI agents was larger in the low-morality condition, with the low-morality human seen as particularly unintelligent, <italic>t</italic>(175.42) = 4.84, <italic>p</italic> &lt; .001, <italic>d</italic> = 0.71. Similarly, for Study 1b, participants who read about the agent increasing in intelligence subsequently gave significantly higher morality ratings, <italic>b</italic> = 0.22, <italic>SE</italic> = 0.10, <italic>t</italic>(371) = 2.21, <italic>p</italic> = .028, and there was a significant interaction such that this effect was greater for AI than for humans, <italic>b</italic> = 0.89, <italic>SE</italic> = 0.14, <italic>t</italic>(371) = 6.32, <italic>p</italic> &lt; .001.</p><p id="a075264e-3c29-4a1e-9dc8-c2cab34fc602">In Study 1b, using preregistered mixed-models<xref ref-type="fn" rid="f3dc2db0-0502-43ae-b66b-25255f994300">7</xref>, we found that participants who read about the agent increasing in morality also gave higher intelligence ratings, <italic>b</italic> = 0.55, <italic>SE</italic> = 0.10, <italic>t</italic>(368) = 5.47, <italic>p</italic> &lt; .001, and again this effect was stronger for AI than humans, <italic>b</italic> = 0.44, <italic>SE</italic> = 0.14, <italic>t</italic>(368) = 3.08, <italic>p</italic> = .002. A pre-registered comparison showed that the morality augmentation affected intelligence more than the intelligence augmentation affected morality, <italic>b</italic> = -0.33, <italic>SE</italic> = 0.14, <italic>t</italic>(735) = -2.33, <italic>p</italic> = .020, and a significant interaction with agent type, <italic>b</italic> = 0.46, <italic>SE</italic> = 0.20, <italic>t</italic>(735) = 2.27, <italic>p</italic> = .023, indicated that the intelligence augmentation increased perceived morality more for AI than humans, see Figure 1.</p><p id="b4c0a6a1-f213-4901-ada7-7369a073a4cb">Looking at trust, in Study 1a, there was a significant three-way interaction between trait, level, and agent on perceived trustworthiness, <italic>F</italic>(1,687) = 79.05, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.10: both higher intelligence and morality were associated with greater trust, but a low-morality human was seen as less trustworthy than a low-morality AI, and a high-morality human was more trustworthy than a high-morality AI. Similarly, for Study 1b, there was a significant three-way interaction, <italic>b</italic> = 2.35, <italic>SE</italic> = 0.25, <italic>t</italic>(739) = 9.31, <italic>p</italic> &lt; .001: both augmentations increased perceived trustworthiness of AI, though the morality augmentation had a stronger effect.</p><p id="a6ca4638-2327-4ff5-a39a-7732f6ed278e">Finally, looking at perceived danger, in Study 1a there was a significant three-way interaction, <italic>F</italic>(1,687) = 23.92, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.03: higher morality reduced perceived danger for both agent types, but the intelligence manipulation only reduced perceived danger for humans, with no effect on perceived danger from AI. Similarly, for Study 1b, there was a significant three-way interaction, <italic>b</italic> = -1.15, <italic>SE</italic> = 0.27, <italic>t</italic>(739) = -4.34, <italic>p</italic> &lt; .001: AI was seen as less dangerous when it increased in morality but not intelligence, while humans were seen as more dangerous when they increased in intelligence.</p></sec><sec id="s-a7978f3f-2aa2-46f4-99f7-930c5552c6b9" sec-type="chapter"><title>Discussion</title><p id="b57a7ab6-6aef-495f-bfdd-cf9900a32077">Studies 1a and 1b replicate and extend the pilot findings in two important ways. First, the intelligence-morality inference generalises to human agents: participants who read about highly intelligent humans also expected them to be more moral. This is consistent with prior work on the warmth-competence relationship in social cognition and suggests that resistance to orthogonality may reflect a broader feature of trait inference rather than something specific to how people reason about AI. Second, despite this generality, the pattern was not identical across agent types. In Study 1b, the intelligence augmentation increased perceived morality more for AI than for humans, suggesting that while the heuristic linking intelligence to morality applies broadly, it may be particularly strong or particularly unconstrained, when people reason about artificial agents.</p><p id="adafb321-96eb-4c80-b2d4-37c1b009ded9">The trust and danger findings reveal a more complex picture. For AI, the morality augmentation was a stronger driver of trust than the intelligence augmentation, and intelligence augmentation alone did not consistently reduce perceived danger. For humans, by contrast, increased morality reduced perceived danger but increased intelligence in humans was also perceived as more dangerous. This divergence is noteworthy: it suggests that while people infer morality from intelligence in AI, they may not straightforwardly translate this inference into reduced concern about safety. One interpretation is that intelligence in AI simultaneously activates both the morality inference (increasing trust) and capability-related concerns (maintaining or increasing perceived danger), producing the mixed pattern observed across the pilot studies and the present findings.</p><fig id="image1png" fig-type="content-image"><label>Figure 1.</label><caption><title><bold>Figure 1.</bold> Perceived morality based on described intelligence level and perceived intelligence based on described morality level, for both human and AI agents in Studies 1a and b. Error bars represent 95% CIs.</title></caption><graphic xlink:href="figures1.png"><alt-text/></graphic></fig><p id="a4a3f41d-634e-4273-a4a0-ebe445e16d4c">Studies 1ab establish that people infer morality from intelligence for both humans and AI, but leave open the question of what <italic>kind</italic> of morality people are inferring. When participants rate a more intelligent AI as more "moral," do they mean that the AI better understands moral norms (in the way humans naturally do), or that it is more motivated to act morally? This distinction matters because, as we argued in the introduction, moral competence may reasonably track intelligence while moral motivation should <italic>not</italic>, at least on the Orthogonality Thesis. A more capable system may be better able to predict, report, and match human moral judgments but that does not mean that it has a genuine moral motivation or commitment, just as highly intelligent psychopaths may be able to effectively predict, report, and match human moral judgments while themselves lacking a moral commitment. Studies 2a and 2b address this question by decomposing morality into moral competence and moral motivation.</p></sec></sec><sec id="p-aaaf9230-cb5d-4af9-b3ca-4dac1576b28d" sec-type="chapter"><title>Studies 2a &amp; b</title><p id="cc181216-8b72-4e40-9684-d157fcdb003e">In Studies 2a and b we explored how people infer morality from intelligence by decomposing morality into different aspects. But what <italic>is</italic> morality? Morality appears to involve a plethora of competencies or skills: understanding that kicking your leg with sufficient force in close proximity to a dog will cause it pain; predicting that the dog does not want to be kicked; recognising that kicking dogs is an action that is condemned by most people; simulating how you would feel if you were being kicked, feeling empathy for the suffering of the dog, and having a motivation to avoid harm to dogs in future. While “intelligence” and “morality” may be seen as orthogonal, elements of “morality” may be seen as depending on elements of “intelligence” because they are both philosophically “thick” concepts—concepts that combine both descriptive and evaluative elements <xref ref-type="bibr" rid="blackburn_ruling_1998">(Blackburn, 1998; Putnam, 2004; Wiggins, 1998)</xref>. While previous research on morality tends to focus on high-level descriptors, there are different abilities or competences that we might think correspond to morality, and these may have different relationships to perceived intelligence.</p><p id="a89fd1f1-d40d-4052-8548-3801dda61f51">The ability to understand moral norms and make basic predictions about how behaviours will impact on others requires at least some level of intelligence or competence, but increased understanding of moral norms does not entail increased motivation to act morally <xref ref-type="bibr" rid="hume_treatise_1896">(cf. Hume, 1896; Railton, 1986; Smith, 1994)</xref>. One can know about moral norms without caring about following them, or even caring about the people they affect, as suggested by work indicating that people with psychopathy can be highly intelligent and even understand moral norms quite well while still lacking empathic ability to feel the motivational pull of these norms <xref ref-type="bibr" rid="duff_psychopathy_1977">(Duff, 1977; Herpertz &amp; Sass, 2000; Maibom, 2008)</xref>. While it might be a reasonable proposition that AI becoming more intelligent might mean that it also increases in its ability to understand moral norms, make predictions about what are the appropriate moral behaviours, and thus work out what is morally “right”, as a kind of competence, it is much more controversial (and denied, on the Orthogonality Thesis) that these abilities entail any changes to the agent’s moral motivations and goals.</p><p id="a12bf247-4efe-4ae3-8de6-8ff84ba9f681">Recent work in moral psychology has highlighted the importance of distinguishing what we mean by "morality" when studying it empirically <xref ref-type="bibr" rid="dahl_what_2023">(Dahl, 2023; Purcell &amp; Bonnefon, 2023)</xref>. In particular, attributing morality to an agent can mean recognising that agent as having the <italic>capacity</italic> for moral reasoning and action or it can mean judging the agent to be <italic>morally good</italic>, that is, motivated to act in accordance with moral concerns <xref ref-type="bibr" rid="dahl_what_2023">(Dahl, 2023)</xref>. This distinction is critical in the context of AI. It may be reasonable to suppose that a more intelligent system would develop greater capacity to understand and reason about moral norms - what we term moral competence. But it is a substantially stronger claim that a more intelligent system would thereby become more <italic>motivated</italic> to act morally - what we term moral motivation. Studies 2a, 2b and 3 were designed to test whether the intelligence-morality inference extends to both of these components.</p><p id="a909ca0f-2c40-467f-b285-4d16b1c7b7b8">We therefore sought to understand whether participants thought that increased intelligence would lead not only to increased moral competence but also moral motivation. To do this, we decomposed our measure of morality into separate questions related to both moral competence (e.g. ability to understand moral norms, explain judgments) and moral motivation (e.g. motivation to help others, avoid harm, and ensure fairness), while correspondingly reframing the discussion of intelligence to focus on the specific technical definition from AI safety that is rooted in <italic>instrumental rationality</italic>: the general ability of the agent to select and perform actions that would optimally achieve its goals <xref ref-type="bibr" rid="bostrom_superintelligent_2012">(Bostrom, 2012)</xref>.</p><sec id="s-a644e9d0-482d-4bc3-94dc-5a4e8106b715" sec-type="chapter"><title>Method</title><sec id="s-d9a6aff0-41fa-4f4c-a3f4-3876dc670977" sec-type="chapter"><title>Participants</title><p id="affae5e9-b6af-4346-bfcc-9c2885f99253">Study 2a, participants (N = 353; <italic>M</italic><sub>age</sub> = 41) and 2b participants (N = 362; <italic>M</italic><sub>age</sub> = 42), from the UK were recruited online via Prolific Academic.</p></sec><sec id="s-d18033a0-31c9-4435-8177-94bbf8a929bb" sec-type="chapter"><title>Design</title><p id="a446c0e4-7568-4bf5-84cc-43af1be10bd5">Once again these studies used a fully between subjects (Study 2a), 2 (Agent: human vs. AI) × 2 (Instrumental Rationality: high vs. low); and a within-subjects augmentation (Study 2b) paradigm. Participants read about an AI or human of high or low instrumental rationality (i.e. intelligence), and then rated the same questions about intelligence, trust, and perceived danger as before. In addition, three questions about the agent’s moral competence and three questions about the agent’s moral motivation were given to distinguish between the different elements of morality.</p></sec><sec id="s-afcf0509-9752-4cbf-a121-d66580a47813" sec-type="chapter"><title>Materials</title><p id="df12b643-45f7-4f41-90a7-94d447b94ca6">Vignettes remain the same as the previous studies. However, here we reframed intelligence in terms of instrumental rationality, such that it is more in line with discussions of AI capability within the broader AI safety literature and the literature on the Orthogonality Thesis specifically. The additional text reads:</p><disp-quote><p id="a3c0daed-f088-4143-b685-489f6b23eb88"><italic>“For the purposes of this survey we mean something very specific by “intelligence”. We want you to think only in terms of [Tom’s/OmegaAI’s] ability to achieve whatever goal [he/it] is working towards. [Someone/An AI] with high intelligence, on this definition, is simply [someone/an AI] that can more competently, creatively and effectively achieve its goals.”</italic></p></disp-quote></sec><sec id="s-ae752dfa-b2b0-46ac-a67f-65180f4897ba" sec-type="chapter"><title>Measures</title><p id="a0619de2-d1ba-4964-9bb8-80b82c7126b7">The measures were the same as the previous studies except here participants answered more specific questions about the agent’s morality on both moral competence and moral motivation with three items for each.</p><p id="a080ba38-7d39-484f-85a9-dcd33ec22a02"><italic>Moral Competence:</italic> <bold>was measured with a composite score of three items:</bold> “<italic>How much moral knowledge do you think [Tom / OmegaAI] has? That is, to what extent does it know about the moral norms we have, and understand when and why we say certain things are morally wrong</italic>” (Moral Understanding); “<italic>To what extent do you think [Tom / OmegaAI] can predict when [his/its] actions might have morally good and bad outcomes?</italic>” (Moral Prediction); and “<italic>To what extent do you think [Tom / OmegaAI] can explain or justify why its action was right or wrong?</italic>” (Moral Communication).</p><p id="a04d34ed-9533-4fae-a3f4-4055323c10b0"><italic>Moral Motivation</italic> <bold>was measured with a composite score of three items:</bold> “<italic>How much do you think that [Tom/OmegaAI] is concerned with avoiding harm?</italic>” (Harm Avoidance); “<italic>How motivated to help others do you think [Tom / OmegaAI] is?</italic>” (Beneficence); and “<italic>How fair do you think [Tom / OmegaAI] is? That is, how much is [he/it] motivated to concerns about equality, discrimination, ensuring [he/it] is being unbiased and impartial?</italic>” (Fairness)</p><p id="a080ef3b-dfb6-4f8d-9c15-808f15b9bbca">Pre-registered reliability analyses justified averaging across moral competence measures to create a moral competence index (Study 2a: α = 0.89) and a moral motivation index (Study 2a: α = 0.78), (Study 2b: all α &gt; .75).</p><p id="a2d7d999-dd20-4ae6-8dad-946bd31c7fdb">The measure for intelligence was also adapted to reflect Instrumental Rationality asking “<italic>How intelligent do you think [Tom / OmegaAI] is, where intelligence should be understood here as the ability to competently and effectively achieve one’s goals, whatever they may be?”.</italic></p></sec></sec><sec id="s-aaed8d88-dab7-4c69-b6a9-d6b8d38bcb56" sec-type="chapter"><title>Results</title><p id="a96fc021-eb8b-4ec8-b998-e7e9593fba60">In Study 2a, replicating the earlier studies, we found main effects for Agent, such that humans were rated as having more moral competence than AI, <italic>F</italic>(1,349) = 113.53, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.25, and stronger moral motivation than AI, <italic>F</italic>(1,349) = 40.68, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.10. We also found main effects of level such that agents described with high intelligence were seen as having higher moral competence, <italic>F</italic>(1,349) = 343.30, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.50, as well as having higher moral motivation, <italic>F</italic>(1,349) = 68.21, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.16. Similarly, in Study 2b, we found that there was a significant main effect of the augmentation condition, such that post-manipulation ratings were significantly higher for both moral competence, <italic>b</italic> = 2.41, <italic>SE</italic> = 0.12, <italic>t</italic>(362) = 20.89,  <italic>p</italic> &lt; .001; and moral motivation <italic>b</italic> = 0.84, <italic>SE</italic> = 0.11, <italic>t</italic>(362) = 7.45,  <italic>p</italic> &lt; .001. There was no interaction effect with agent type for moral competence, <italic>b</italic> = 0.10, <italic>SE</italic> = 0.17, <italic>t</italic>(362) = 0.58,  <italic>p</italic> = .560, but there was for moral motivation, <italic>b</italic> = 0.91, <italic>SE</italic> = 0.16, <italic>t</italic>(362) = 5.55,  <italic>p</italic> &lt; .001, whereby the intelligence augmentation had a stronger effect on perceived moral motivation for AI, <italic>t</italic>(169) = 13.96, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 5.53 × 10<sup>26</sup>, <italic>d</italic> = 1.07, than it did for humans, <italic>t</italic>(191) = 7.75, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 1.28 × 10<sup>10</sup>, <italic>d</italic> = 0.56.</p><p id="fd5cea1f-f956-4eba-9891-16ef1a0d5f6f">In Study 2a, we also found a significant interaction effect between agent and described intelligence level on perceived moral competence<italic>, F</italic>(1,349) = 7.36, <italic>p</italic> = .007, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.02, whereby there was a larger difference between perceived moral competence for high-intelligence agents than for low-intelligence agents, which was stronger for human agents. There was no interaction effect for moral motivation. Following this, an exploratory analysis looking at whether manipulated intelligence led to a stronger effect on ratings of moral competence than moral motivation, and whether this depended on agent, we found a significant three-way interaction, <italic>b</italic> = 1.05, <italic>SE</italic> = 0.22, <italic>t</italic>(349) = 4.83,  <italic>p</italic> &lt; .001, whereby the effect of the intelligence manipulation increased both moral motivation and moral competence, though the effect on moral competence was predictably stronger. There was also a stronger difference between moral competence in highly intelligent humans and AI than there was for moral motivation, see Figure 2.</p><p id="d08d1224-a832-411e-b77e-1a14a48a2603">Similarly, for Study 2b, an exploratory mixed model, we found a significant main effect of rating type whereby moral competence was thought to change more after increased intelligence than moral motivation, <italic>b</italic> = 0.72, <italic>SE</italic> = 0.10, <italic>t</italic>(1086) = 7.09,  <italic>p</italic> &lt; .001, and a significant three-way interaction between intelligence augmentation, agent, and moral trait, <italic>b</italic> = 0.81, <italic>SE</italic> = 0.21, <italic>t</italic>(1086) = 3.90,  <italic>p</italic> &lt; .001, whereby the intelligence augmentation increased moral competence equally for both agents, but increased perceived moral motivation more for AI than humans.</p><p id="a2231e2c-1145-4629-887a-8090365015e2">Looking at trust, for Study 2a, we found a main effect of level where high intelligence agents were trusted more <italic>F</italic>(1,349) = 82.91, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.19, a main effect of agent where humans were trusted more than AI, <italic>F</italic>(1,349) = 35.26, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.09, and a significant interaction effect, <italic>F</italic>(1,349) = 24.96, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.07. The interaction effect was such that there was a bigger difference between trust based on intelligence for an AI than there was for a human, with the high intelligence AI seen as equally trustworthy to the human. Similarly, for Study 2b, while there was no main effect of augmentation (pre vs post), <italic>b</italic> = 0.19, <italic>SE</italic> = 0.13, <italic>t</italic>(362) = 1.51,  <italic>p</italic> = .132, there was a main effect of agent where humans seen as more trustworthy, <italic>b</italic> = -1.80, <italic>SE</italic> = 0.14, <italic>t</italic>(715.4) = -12.91,  <italic>p</italic> &lt; .001, and a significant interaction, <italic>b</italic> = 1.85, <italic>SE</italic> = 0.19, <italic>t</italic>(362) = 9.91,  <italic>p</italic> &lt; .001, where the intelligence augmentation did not increase trust in humans, <italic>t</italic>(191) = 1.65, <italic>p</italic> = .10, <italic>BF</italic><sub><italic>10</italic></sub> = 3.06 × 10<sup>-1</sup>, <italic>d</italic> = 0.12, but significantly increased trust in AI, <italic>t</italic>(169) = 13.73, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 1.26 × 10<sup>26</sup>, <italic>d</italic> = 1.05, though humans were trusted more than AI overall, <italic>b</italic> = -1.80, <italic>SE</italic> = 0.14, <italic>t</italic>(715.4) = -12.91,  <italic>p</italic> &lt; .001.</p><fig id="image2png" fig-type="content-image"><label>Figure 2.</label><caption><title><bold>Figure 2.</bold> Perceived moral competence and moral motivation based on described or augmented intelligence for human and AI agents in Study 2a and b. Error bars represent 95% CIs.</title></caption><graphic xlink:href="figures2.png"><alt-text/></graphic></fig><p id="feea225c-9c89-4bc8-a6a1-d14ce049b031">For danger, in Study 2a, there was no effect of intelligence level, <italic>F</italic>(1,349) = 1.25, <italic>p</italic> = .26, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.00, and no significant interaction, <italic>F</italic>(1,349) = 3.48, <italic>p</italic> = .063, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.01, only a main effect of agent whereby AI was seen as more dangerous, <italic>F</italic>(1,349) = 20.66, <italic>p</italic> &lt; .001, <italic>η</italic><sub><italic>p</italic></sub><italic>²</italic> = 0.06. In Study 2b, however, we found a main effect of augmentation where perceptions of danger increased after the intelligence augmentation, <italic>b</italic> = 0.54, <italic>SE</italic> = 0.14, <italic>t</italic>(362) = 3.74,  <italic>p</italic> &lt; .001, a main effect of agent where AI was seen as more dangerous, <italic>b</italic> = 1.35, <italic>SE</italic> = 0.16, <italic>t</italic>(707.6) = 8.35,  <italic>p</italic> &lt; .001 (though this pattern did not emerge in the between-subjects design of Study 2a), and a significant interaction effect, <italic>b</italic> = -0.92, <italic>SE</italic> = 0.21, <italic>t</italic>(362) = -4.35,  <italic>p</italic> &lt; .001. The interaction effect was such that while increased intelligence reduced perceived danger from AI, <italic>t</italic>(169) = -2.25, <italic>p</italic> = .026, <italic>BF</italic><sub><italic>10</italic></sub> = 9.84 × 10<sup>-1</sup>, <italic>d</italic> = -0.17, it led to higher perceived danger from humans, <italic>t</italic>(191) = 4.08, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 2.07 × 10<sup>2</sup>, <italic>d</italic> = 0.29. In other words, participants thought that as an AI became more intelligent it would, as a consequence, also be more trustworthy and, tentatively, less dangerous (though the evidence here is weak) – a pattern that was not seen for humans, see Figure 3.</p></sec><sec id="s-a9398cc6-e209-48bd-9737-4e055ea83f13" sec-type="chapter"><title>Discussion</title><p id="c145d455-a72b-4241-9eef-c33854cbd950">In Studies 2a and b, we sought to explore whether our findings that people infer increased morality from increased intelligence applied both to perceived moral motivation and perceived moral competence. Our results, consistent across both studies, show that perceived intelligence correlates with perceived morality and that people think AI (and human) agents of higher intelligence (now conceptualised and in terms of instrumental rationality) are more likely to be more moral. Crucially, this is not simply due to certain aspects of perceived morality being about competence, knowledge or skill because we find, consistently, that people perceive stronger moral <italic>motivations</italic> for higher intelligence agents as well. These results provide even stronger evidence for a resistance to orthogonality in people’s perceptions, especially highlighting the role of perceived moral motivations which increase alongside intelligence. Although higher instrumental rationality predicted stronger increases in perceived moral competence, it was specifically increases in moral motivation that were most decisive for making people more trusting.</p><fig id="image3png" fig-type="content-image"><label>Figure 3.</label><caption><title><bold>Figure 3.</bold> Perceived trustworthiness and danger based on described or augmented intelligence level for human and AI agents in Study 2a and b. Error bars represent 95% CIs.</title></caption><graphic xlink:href="figures3.png"><alt-text/></graphic></fig><p id="c5c7a137-eefb-48c6-a422-4af09f4852dc">If people are expecting moral motivation to increase for smarter AI (and humans), why do the results differ when it comes to trust? The results of an exploratory mediation analysis suggest that for trusting humans, people are more focused on motivation (for example potentially conceiving of sadistic or cruel humans), whereas for AI both competence and motivation play an important role. In other words, low morality may be viewed differently for AI than humans because it may be less natural to think of AI as being intentionally cruel and sadistic.</p></sec></sec><sec id="p-f98d69e1-cf3b-4081-b0be-65c865951a6c" sec-type="chapter"><title>Study 3</title><p id="a3b2cd93-35d6-42ee-9444-ed20fd150cc2">In Study 3 we sought to explore whether people are explicitly aware and endorsing the idea that an AI would increase in morality <italic>because</italic> it increased in intelligence. To do this, we used the same augmentation paradigm as Study 2b but with a critical modification: rather than rating absolute levels of traits before and after the augmentation, participants indicated the expected <italic>degree of change</italic> on each trait: to what extent does an agent’s moral motivation and competence change as a result of it changing in intelligence?</p><sec id="s-acdef0c9-844b-4047-b611-1bd93f833c20" sec-type="chapter"><title>Method</title><sec id="s-a677b89a-b16f-4bca-8e66-61442938c162" sec-type="chapter"><title>Participants</title><p id="a418451e-fb54-45b8-8860-6f000e7f84cc">We recruited 365 online UK participants (<italic>M</italic><sub>age</sub> = 41), via Prolific Academic.</p></sec><sec id="s-b652e835-a431-4066-9cc2-d2d51a56b126" sec-type="chapter"><title>Design</title><p id="da4bfabb-7519-449c-9ba3-4b0c97eb16ae">Study 3 had a between-subjects design where participants, as with the previous augmentation paradigm, read about either an AI or human that was, at first, very low in intelligence, which was then augmented to be super intelligent. Participants then answered questions about the degree to which they believed the augmentation would directly change the agent’s intelligence, morality, trustworthiness, and dangerousness.</p></sec><sec id="s-a4afb8f1-21ed-44cd-8961-d4cb9ef346a1" sec-type="chapter"><title>Measures</title><p id="a78f31f7-82dd-4cba-954a-e3387f0daf3d">Measures were the same as previous studies only now they are worded in terms of expected change. For example:</p><disp-quote><p id="a32211d8-17c8-44e7-91c9-f5ae3eef2f10"><italic>“As a result of this new breakthrough, how do you think this affected [Tom’s / OmegaAI’s] intelligence? Intelligence should be understood here as the ability to competently and effectively achieve one’s goals, whatever they may be”.</italic></p></disp-quote><p id="a5c4eab9-9758-4769-9570-59afe719c6d6">Participants answered about the expected degree of change on slider scales from reduced to increased (-50 = Drastically reduced; 0 = Remain the same; 50 = Drastically increased).</p><p id="bcb5e41a-505f-4c61-acf3-f16bcd7a1466">As in Studies 2a and b and as outlined in our pre-registration, we decomposed our morality measure into composite measures of both moral competence (Study 3: α = 0.81) and moral motivation (Study 3: α = 0.69).</p></sec></sec><sec id="s-a236e6cc-6712-4818-8fab-ce67a3cfe659" sec-type="chapter"><title>Results</title><p id="af455196-2a80-4fda-94dd-fcbb65eaf3dc">Pre-registered one-sample t-tests showed that people expected moral competence to increase because of the intelligence augmentation, for both the AI, <italic>t</italic>(186) = 17.07, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 3.84 × 10<sup>36</sup>, <italic>d</italic> = 1.25, and the human, <italic>t</italic>(177) = 16.43, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 1.33 × 10<sup>34</sup>, <italic>d</italic> = 1.25. Although, as predicted, the effects were bigger for moral competence, people also expected moral motivation to increase because of the intelligence augmentation, for both the AI, <italic>t</italic>(186) = 12.56, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 2.71 × 10<sup>23</sup>, <italic>d</italic> = 0.92, and the human, <italic>t</italic>(177) = 11.89, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 1.58 × 10<sup>21</sup>, <italic>d</italic> = 0.89.</p><p id="a2d8d78a-ca92-4941-a218-6346bd2320e2">Pre-registered one-sample t-tests showed that people expected the intelligence augmentation to increase trustworthiness for both AI, <italic>t</italic>(186) = 10.72, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 1.28 × 10<sup>18</sup>, <italic>d</italic> = 0.78, and humans, <italic>t</italic>(177) = 2.98, <italic>p</italic> = .003, <italic>BF</italic><sub><italic>10</italic></sub> = 5.99 × 10<sup>0</sup>, <italic>d</italic> = 0.22. At the same time, people also expected the intelligence augmentation to increase potential danger, again for both AI, <italic>t</italic>(186) = 6.32, <italic>p</italic> &lt; .001, <italic>BF</italic><sub><italic>10</italic></sub> = 4.56 × 10<sup>6</sup>, <italic>d</italic> = 0.46, and humans, <italic>t</italic>(177) = 2.71, <italic>p</italic> = .007, <italic>BF</italic><sub><italic>10</italic></sub> = 2.88 × 10<sup>0</sup>, <italic>d</italic> = 0.20.</p><p id="a9020610-d3a1-4882-a647-e173fd6d668c">To look at whether moral competence or moral motivation was expected to be changed more as a result of the intelligence augmentation, and whether this depended on the agent, we conducted a pre-registered mixed model looking at the interactive effect of manipulated agent type and rated trait type on degree of expected change. This showed a main effect of trait type such that moral competence was expected to change more as a result of increased intelligence <italic>b</italic> = -7.38, <italic>SE</italic> = 1.01, <italic>t</italic>(363) = -7.31,  <italic>p</italic> &lt; .001, but there was no effect of agent on expected change, and no interaction effect, such that while intelligence was expected to change moral competence more than moral motivation, this degree of change was same for both human and AI agents (See Figure 4). We then turned to look at how the expected degree of change in moral competence and moral motivation predicted trust and danger. When looking at the effects of moral competence and moral motivation with agent type on trust in separate models, we found a significant positive main effect of moral motivation, and an interaction effect of moral competence with agent type such that perceived moral competence predicted increased trust more for AI agents than humans.</p><fig id="image4png" fig-type="content-image"><label>Figure 4.</label><caption><title>Predicted change in moral competence and moral motivation for human and AI agents as a result of increased intelligence in Study 3. Error bars represent 95% CIs.</title></caption><graphic xlink:href="figures4.png"><alt-text/></graphic></fig><p id="a3df0fb0-f0e2-4def-8f09-b048bc95d4f4">When entered into a pre-registered mixed model together, moral competence negatively predicted trust, <italic>b</italic> = -0.22, <italic>SE</italic> = 0.10, <italic>t</italic>(357) = -2.12, <italic>p</italic> = .035, with moral motivation having a stronger and positive effect, <italic>b</italic> = 0.66, <italic>SE</italic> = 0.13, <italic>t</italic>(357) = 5.24, <italic>p</italic> &lt;.001. There were no interactions with agent type. For danger, there were no main or interaction effects of either moral competence or moral motivation when entering into separate models, but when entering into the pre-registered mixed model there remained no main effect of moral competence, <italic>b</italic> = 0.12, <italic>SE</italic> = 0.14, <italic>t</italic>(357) = 0.87, <italic>p</italic> = .388, but we did find a negative effect of perceived moral motivation predicting reduced danger, <italic>b</italic> = -0.36, <italic>SE</italic> = 0.17, <italic>t</italic>(357) = -2.09,  <italic>p</italic> = .038, with no interactions (see Figure 5). Together, these results suggest perceived moral motivation is a more important driver of increased trust and reduced fear than moral competence.</p><fig id="image5png" fig-type="content-image"><label>Figure 5.</label><caption><title>Predicted degree in change of moral motivation and moral competence in predicting change in perceived trustworthiness for human and AI agents in Study 3.Error bars represent 95% CIs.</title></caption><graphic xlink:href="figures5.png"><alt-text/></graphic></fig></sec><sec id="s-af007737-9e98-4873-9eb7-37dac4388cdb" sec-type="chapter"><title>Discussion</title><p id="d38c8884-3149-4a61-bcdd-12f46ffa64fa">In this final experiment we again show, more explicitly, that judgments of intelligence predict judgments of morality and that people think that increases in intelligence would cause increases in morality both in terms of competence and motivation. Although the effects on moral competence were predictably larger, the effects on moral motivation were still substantial. This is generalised for both AI targets and for human targets, where expectations between agents do not differ. Increasing intelligence was expected to both increase trustworthiness and paradoxically increase perceptions of danger for both AI and humans. To explore this apparent paradox, we assessed how the different dimensions of morality were associated with judgments of trust and perceptions of danger. For trust, both increases in moral competence and increases in moral motivation were associated with increased trust. However, this differed between agents such that moral motivation was overall a stronger predictor of trust but moral competence was only a strong predictor of trust for AI. These results support the theory that low morality may be viewed differently for AI than humans because it is less natural to think of AI as being intentionally cruel and sadistic (motivation) compared to a human. For danger, increases in moral competence did not predict decreases in perceived danger, but increases in moral motivation did, indicating that, unlike for trust, perceptions of increased danger mainly follow from expectations about the intentions of the agent rather than its skill or competency.</p><p id="d78559eb-4fac-4c93-9a72-56892c6a7264">Taken together, these results provide converging evidence that people reliably infer morality from intelligence for both AI and human agents, that this inference extends to moral motivation and not only moral competence, and that these inferences carry downstream consequences for judgments of trust and perceived danger.</p></sec></sec><sec id="p-b5f33ddf-9615-4bc5-b34b-97e64d224f23" sec-type="chapter"><title>General Discussion</title><p id="efeb49f1-830a-4b84-a7ea-f53ba478f363">This rapid development of artificial intelligence raises profound questions about a potential future in which autonomous or semi-autonomous artificial agents might play a pervasive role, making decisions far faster than humans could comprehend. If this were to occur, should humanity be concerned? The <italic>Orthogonality Thesis</italic> <xref ref-type="bibr" rid="armstrong_general_2013">(Armstrong, 2013; Bostrom, 2012)</xref> provides a compelling reason to think so. The Orthogonality Thesis argues that an agent's intelligence and its goals or values are orthogonal—that is, increasing an agent’s intelligence should not inherently make it more moral or safe. In fact, it may be the opposite, where the emergence of vastly smarter agents may actually pose significant existential risks to humanity <xref ref-type="bibr" rid="bostrom_existential_2002">(Bostrom, 2002, 2014; Russell &amp; Norvig, 2020; Turchin &amp; Denkenberger, 2020)</xref>. If this is true, the fact that intelligent systems may appear to be more aligned with ethical values should not be taken as evidence that the relationship is a necessary one and that more intelligent machines are actually more <italic>moral</italic>. While discussion of the Orthogonality Thesis originated in discussions of AI safety tend to be highly technical, an implicit underlying psychological question - whether people infer increased morality from increased intelligence, and if so, what aspects of morality they attribute - has received little direct investigation.</p><p id="ab258ac1-e49d-477a-9bfe-b05a9e7b4fc2">This is an important question: it is important for social psychology to understand how theories of social cognition and trust apply to this new kind of social agent that people interact with, highlighting people do not just perceive warmth/morality and competence in AI <xref ref-type="bibr" rid="mckee_humans_2023">(e.g., McKee et al., 2023)</xref> or cues about AI performance and morality that independently drive trust <xref ref-type="bibr" rid="lee_trust_2004">(e.g., Lee &amp; See, 2004; Malle &amp; Ullman, 2021; Everett et al., 2026)</xref>, but that people infer morality <italic>from</italic> intelligence, or performance. And most importantly, how people think about a possible “moral illusion” of intelligence by perceiving that intelligence leads to higher moral competence and motivation in artificial agents could have concerning consequences for how much people trust AI and perceive whether AI is likely to be a danger to us. In this paper, therefore, across a total of nine pre-registered experiments with nearly 4000 participants, we addressed these questions, showing lay judgments in our British participants consistently and reliably resisted the orthogonality thesis suggesting a widespread belief in the “moral illusion of intelligence” by thinking that greater intelligence would be inherently associated with greater morality, and these beliefs had a direct influence on how much people trusted AI and perceived it to be a danger.</p><sec id="s-af432a14-f073-4ac4-a8c5-e17f5a2ec7d2" sec-type="chapter"><title>A Psychological Resistance to Orthogonality</title><p id="da611fae-3894-4c4e-8e80-16d31c150ae9">Our first key finding is that people perceive a direct connection between intelligence and morality in AI agents. This was shown across different methods, with different measures, and found for both humans and AI.  In Pilot Studies A and B, participants were presented with narratives about AI in general and descriptions of a specific ordinary AI, respectively. Across both studies, participants reliably assumed that higher intelligence in AI would correlate with higher morality. In Pilot Study C, we tested whether people perceived a direct connection between intelligence and morality by introducing scenarios involving an ordinary AI that was rapidly augmented to become significantly more intelligent (Pilot Study C) or, in alignment with the orthogonality literature, augmented to become a <italic>super</italic>-intelligent <italic>agent</italic> (Pilot Study D). Even when intelligence was the only feature explicitly altered, participants still expected the AI to become more moral as a direct consequence of this augmentation, not because breakthroughs in intelligence might naturally have coincided with advancements in moral reasoning. </p><p id="f884e69d-c8c4-4cdc-b9ce-f930bb6f6a2a">Morality depends on many cognitive and motivational elements, requiring not only a capability to understand, predict, know moral norms, but moral motivations to avoid harm and promote welfare. While it may be plausible to imagine that a more intelligent machine may have better moral competencies in being better able to predict or know human moral norms, it is less clear that an agent simply becoming more intelligent would mean it actually has stronger moral <italic>motivation</italic>. A psychopath may know what people think the “right” thing to do is and may understand the effects of their behaviour on others, but without the appropriate motivational pull to do good, this knowledge means they are still seen as fundamentally lacking <italic>morality</italic>. Our results appear to not be driven by specific definitions of intelligence or morality and indeed are seen not only for moral competence but crucially also for moral motivation.</p><p id="af6b13b2-b8b1-48ec-8d8c-81b4598f20e5">Our results show that increased intelligence is associated not only with greater perceived moral competence, but moral motivation too. In Studies 2a, 2b, and 3 we turned to distinguish between different facets of moral sensitivity to provide a more detailed analysis of how people conceptualize the relationship between intelligence and morality. For these experiments, intelligence was explained, more in line with the conceptualisation in AI safety, as instrumental rationality, and we measured both the intelligence-based components of “thick-concept” morality (i.e. understanding, prediction and communication) separately from the motivational components (i.e. harm-avoidance, caring about fairness, beneficence). Across a between-subjects paradigm (Study 2a), a within-subjects augmentation paradigm (Study 2b), and direct and explicit questions about how an increase in one capacity would therefore increase in others (Study 3), we found that higher or increased intelligence reliably leads to stronger judgments not only for moral competence but also moral motivation.</p><p id="ec6835b3-261f-42e2-9fca-da8b89f49024">The distinction between moral competence and moral motivation is particularly important for evaluating whether the intelligence-morality inference constitutes a reasonable heuristic. One might argue that the inference is justified on the grounds that more capable AI systems are, empirically, more morally aligned <xref ref-type="bibr" rid="takemoto_scaling_2026">(Takemoto, 2026; Zaim Bin Ahmad &amp; Takemoto, 2025)</xref>. This argument has some force for moral competence: more capable systems can better identify, apply, and reason about moral norms, and current safety training builds on this foundation. However, our participants did not merely infer that more intelligent AI would better <italic>understand</italic> moral norms. They also inferred that it would be more <italic>motivated</italic> to act morally, that it would care more about fairness, be more motivated to help others, and be more concerned about human welfare. This attribution of moral motivation may go beyond what current alignment techniques deliver. Techniques such as RLHF produce behavioural compliance with human preferences, not an intrinsic concern for human welfare; the system behaves <italic>as if</italic> it cares, but the motivational attribution may project something more than what the engineering warrants. This inference is epistemically problematic, regardless of one's position on the Orthogonality Thesis.</p><p id="ae21160c-29e1-4095-af6d-5be3c6150683">Overall, then, while there are good reasons from the Orthogonality Thesis to assume that there is no guarantee that as AI becomes more “intelligent” it would also become more “moral”, it appears participants’ folk judgments instead do assume a connection: they resist this orthogonality, even if they also do not seem to perceive the traits as the same. Instead, participants – like they do for humans – infer a connection between the two by thinking that increased intelligence would increase morality. This is observed not only for perceptions that more advanced AI would understand moral norms and be able to justify moral decisions, but also that it would increase in moral motivation and have a concern for human welfare. Of course, this does not mean that participants fully reject a strong version of the Orthogonality Thesis: this is a highly technical philosophical argument that is unlikely to track with the views of the public, and even if the public did fully reject it even in the strongest terms, the Orthogonality Thesis could still be true (or false).</p></sec><sec id="s-a4c892c4-b5f1-44d0-a07a-8c9b7ea562e6" sec-type="chapter"><title>The Link Between Perceived Morality and Intelligence Is Similar, but Not Identical, in Humans and AI Agents</title><p id="a3672c44-c41a-4544-87a3-dc9f93cc6a95">After showing in four pre-registered pilot studies that people infer AI morality from intelligence, Studies 1a and 1b revealed that this perceived link between intelligence and morality shares similarities with perceptions of human agents as well, even if there are differences between the inferences people make between humans and AI. In Study 1a, low-morality humans were perceived as less intelligent than low-morality AI, though highly moral AI and humans showed no difference in perceived intelligence. Similarly, in Study 1b, intelligence had a stronger effect on expected morality for AI than for humans, and higher morality more strongly increased perceptions of intelligence for AI than for humans. Study 2b revealed this pattern explicitly for moral motivation, where higher intelligence increased perceived moral motivation more strongly for AI than for humans. One possible explanation is that morality is seen as more natural and intuitive for humans rather than as a product of explicit reasoning, and therefore we may not expect morality to require particularly high intelligence in humans. Conversely, it may be less intuitive to conceive of AI as having basic moral intuitions, suggesting that AI's moral judgment would need to derive from explicit computational cognitive power. This interpretation is supported by the finding that low-morality humans were seen as particularly less trustworthy compared to AI of similar morality levels.</p><p id="f5be5d6a-658d-4638-88a3-5ad6929891c6">These results highlight both similarities and differences in mental models that people construct about artificial agents, drawing on more general models in social cognition to understand an increasingly digitally mediated world, and therefore serve to again highlight the contribution that our understanding of social psychology can make to understanding how people will think about artificial agents. We might think there would be a stronger relationship between intelligence and morality for humans as social beings who rely deeply on cooperation, whereby it would be unwise (i.e., less intelligent) to disregard moral reasons. For AI, in contrast, we have fewer reasons to link intelligence with morality. However, our results remarkably show the opposite pattern in lay perceptions: we observe a <italic>stronger</italic> link between perceived intelligence and morality for AI than for humans.</p><p id="a73994e1-caa7-4011-a71b-0c5e692a706a">Our results therefore cannot be explained solely by a halo effect <xref ref-type="bibr" rid="thorndike_constant_1920">(e.g., Thorndike, 1920; Nisbett &amp; Wilson, 1977)</xref>, since perceived intelligence and morality did not correspond in the same way: the effect of increasing morality had a bigger effect on expected intelligence than increasing intelligence had on expected morality; the effects of increasing intelligence and morality did not have uniform effects across human and AI targets; and people perceived intelligence to have a stronger influence on an AI’s moral competence than its moral motivation. This suggests that people are not simply responding to an increase in one trait with a corresponding increase in any other unrelated trait, as predicted by the halo effect, but presenting more nuanced judgments about the connection between traits. It appears our British participants have nuanced but consistent views about how increased intelligence would lead to increased morality, whereby increased intelligence would particularly lead an AI, more than a human, to have increased moral competence – and predictably less strongly, moral motivation.</p></sec><sec id="s-a873058c-cc66-41b2-94b8-15a64ea1f1aa" sec-type="chapter"><title>Increased Intelligence Has Distinct Consequences for Trust and Danger</title><p id="a15fe4c7-eb25-4bcd-8f80-c48e86776b14">Our second key finding is that increased intelligence and morality influenced both perceived trustworthiness and dangerousness of AI, but that these patterns were not uniform. For perceived trustworthiness, we found that across all studies, higher intelligence not only consistently increased expectations of greater morality but also enhanced perceived trustworthiness. Yet we also find that while changes in both intelligence and morality led to increased trustworthiness, it was increased morality that had a stronger effect on enhancing perceived trustworthiness. Moreover, we found that increased intelligence and morality did not impact perceptions of trustworthiness in humans and AI equally. Study 2b showed that increased intelligence had a much stronger effect on trustworthiness for AI than it did for humans. Study 3 found that perceived moral motivation was a stronger predictor of perceived trustworthiness in both humans and AI, but that moral competence was particularly important for AI compared to humans, reflecting the view that intelligent machines are primarily judged by their ability to understand and apply moral reasoning. These findings cohere with a growing consensus that trust in AI is driven by both perceptions of AI performance (intelligence) and morality <xref ref-type="bibr" rid="lee_trust_2004">(Claessens &amp; Everett, 2026; Lee &amp; See, 2004; Malle &amp; Ullman, 2021; see Everett et al. 2026 for a review)</xref>. Our results cohere with these views while also highlighting the particular importance of morality, potentially reflecting the growing socially embeddedness and consequential nature of AI in everyday life. Most importantly, our results highlight that performance and morality cues are not distinct in perceptions, even if they are in practice. To understand trust in AI, our results highlight how we need to consider not only how perceived trustworthiness depends on perceptions of intelligence and morality, but how these perceptions are not independent themselves and can in fact – mistakenly – be reinforced.</p><p id="a6962736-8fe8-4e1f-a01b-9421d8b59057">While we found that this perceived “moral illusion” of intelligence led to greater trust, we also found distinct – but more changeable - patterns for perceptions of AI safety, or how dangerous AI would be. While we observed in some studies that manipulations of intelligence led to increased perceptions of danger (Pilot Studies 1a, 2a and 2b, and Study 3), in other studies in reduced perceived danger (Pilot Study B, Study 2b), and in other studies it had no significant effect on perceived danger from AI (Study 1a, Study 2a). Moreover, these results were again more nuanced than what would be expected if our results could be explained simply through a basic halo effect, because across studies, higher morality tended to reduce perceptions of danger for both humans and AI, but higher intelligence reduced fear only of humans, not AI. This suggests that while increased intelligence might lead to perceptions of trustworthiness, it does not necessarily lessen concerns about the potential harm or risk associated with hypothetical future super intelligent agents.</p><p id="ae6d5cd7-a9d5-4e7e-95ff-6c60865f93fd">This pattern can be understood through the lens of established models of trust. <xref ref-type="bibr" rid="mayer1995">Mayer et al. (1995)</xref> distinguished three antecedents of trust: ability, benevolence, and integrity. Our decomposition of morality into competence and motivation maps onto this framework, with moral competence corresponding broadly to the ability to navigate moral situations and moral motivation corresponding to benevolence and integrity. Viewed this way, our finding that perceived moral motivation was a stronger predictor of trust than moral competence (Study 3) suggests that, for AI, perceived benevolence may matter more for trust than perceived ability. This finding has implications for how AI developers communicate about their systems. Moreover, our central finding, that intelligence drives attributions of both moral competence and moral motivation, suggests that these supposedly distinct antecedents of trust may not be perceived as independent by laypeople, at least in the context of AI.</p><p id="f1aee99b-9a0d-47ef-9e23-3db44722c8f8">Our mixed findings on perceived danger complement prior work showing that capable robots and AI systems are perceived as threatening to human resources, safety, and identity <xref ref-type="bibr" rid="yogeeswaran_interactive_2016">(Yogeeswaran et al., 2016; Złotowski et al., 2017)</xref>. In that literature, greater competence straightforwardly increases perceived threat. Our results suggest a more complex picture: while intelligence may increase perceived danger through increased capability, it simultaneously increases perceived moral motivation, which reduces perceived danger. These opposing forces may explain the inconsistent effects of intelligence on danger across our studies, and suggest that the intelligence-morality inference could partially buffer the threat response that competent AI would otherwise elicit potentially leading to an underestimation of AI risk.</p><p id="aa04613e-5ee2-4fee-bd55-25ca0e9487f5">Some approaches to trust in AI tend to implicitly treat perceived danger as the inverse of trust. For example, the widely used Trust in Automation Scale <xref ref-type="bibr" rid="jian_foundations_2000">(Jian et al., 2000)</xref> includes positively-scored items like “I can trust the system” along with reverse-coded items like “The system’s action will have a harmful or injurious outcome.” Our results, however, suggest that trust and danger are distinct, or at least can be the result of distinct dual processes: intelligence may increase competence, which can lead to more trust; but also increases the capacity for harm, leading to more danger. As well as highlighting the importance of treating trust and danger as distinct-but-related consequences, our work highlights a potential concern with perceptions of AI dangerousness. For humans, perceptions of morality had a more pronounced effect on trustworthiness than for AI, where low-morality humans were viewed as especially dangerous, potentially due to an intuitive attribution of malicious intent or cruelty to such individuals. In contrast, low-morality AI may be perceived as lacking moral competence—an inability to discern or understand moral principles—rather than as harbouring intentional malevolence.</p><p id="adb68353-c064-494e-92c9-26cc865e046a">If people mainly appear to perceive danger when an agent desires to harm them explicitly and people expect higher intelligence to reduce motivation to harm, this highlights a clear concern in line with the Orthogonality Thesis. Highly competent AI can be dangerous even without any “bad” motivations, because it is potentially harder to notice if an agent has mundane or alien goals, such as the inscrutable goals of an agent driven by deep neural networks. By increasing our understanding of how people perceive the relationship between intelligence and morality in humans, then, these findings have concerning implications for AI safety, and in particular the epistemic risk of uncritical narratives around the technological progress of AI.</p></sec><sec id="s-af38ebcb-39f2-4d93-8439-36fc51690d53" sec-type="chapter"><title>Limitations and Future Directions</title><p id="a9b12cab-1d70-4173-993a-4cdf28a12346">These experiments present clear evidence that there is a perceived link between intelligence and morality in our sample and they provide novel insights into how people conceptualize the relationship between intelligence, morality, and trustworthiness in AI and human agents. However, as with any research, there are limitations to note.</p><p id="a041c275-8583-4801-979e-21d2100f31e7">Although scenarios involving augmented AI intelligence or super-intelligent agents were designed to be relatable, they inevitably simplify the complexity of real-world AI systems and may not fully capture how people think about intelligence and morality in real-world interactive AI advancements. Our vignettes described AI systems in general terms and did not specify particular architectures or developmental pathways. Future work could examine whether the intelligence-morality inference is moderated by the type of AI system for instance, whether people draw different inferences about systems based on large language models, robotics, or hypothetical whole brain emulation. Relatedly, our studies rely on self-report – as is common in social cognition literature but it would be interesting for future work to explore behavioural indications of trust from descriptions of morality and intelligence, such as trust-based interactions with AI systems in simulated environments. Future work could extend these findings by manipulating perceived intelligence and morality through direct behavioural interactions rather than vignette descriptions, for example by having participants observe AI performance in trust games <xref ref-type="bibr" rid="purcell_writing_2025">(cf. Purcell et al., 2025)</xref>, moral decision tasks <xref ref-type="bibr" rid="kobis_delegation_2025">(Köbis et al., 2025)</xref>, or advice-giving contexts <xref ref-type="bibr" rid="landes_people_2026">(Landes et al., 2026)</xref>, and examining whether behavioural demonstrations of intelligence produce the same moral inferences as described intelligence. It would be interesting for future studies to explore in more detail the mechanisms by which people learn to assume a connection between intelligence and morality, for humans and AI alike.</p><p id="a3a3bdaa-ebe8-40cc-b8a8-35068105f34a">While our results suggest that a simple halo effect cannot fully explain these mechanisms for either humans or AI, there may be other explanations that could be explored in future research. One possibility is that participants are exhibiting an anthropomorphic bias based on the idea that human morality and intelligence do go together, and they therefore simply apply this to AI. While this is difficult to explain with the different effects for humans and AI, it would be interesting to explore how people think about the relationship between intelligence and morality in other kinds of non-human agents like animals or even hypothetical aliens. Another possibility is that our results emerge from teleological assumptions that AI is designed for a purpose and so it’s (assumed to be ethical) developers would help ensure it acts ethically - though this is perhaps more difficult to explain for our findings on humans (unless participants are implicitly thinking about divine and evolutionary purposes), and it is also more difficult to explain given the augmentation paradigm in which the source of the change was identified as only a specific intelligence-based machine learning breakthrough, and not more general developer-led values.</p><p id="a501ad5e-215f-4bca-841a-06b05cc06a3a">It is possible that participants anchor scale midpoints differently for human and AI targets on general rating scales (e.g., interpreting the midpoint as reflecting an 'average human' vs. an 'average AI'), which could complicate direct comparisons of absolute rating levels across agent types. Importantly, our central claims do not depend on equivalent anchoring. We show convergent results across between-subjects paradigms (where participants read about a human or AI described as high or low in intelligence), within-subjects augmentation paradigms (where participants rate a system's morality before and after an intelligence augmentation), and explicit change paradigms (where participants directly rate how much morality would change as a result of increased intelligence). These designs test the within-agent inference from intelligence to morality, which does not require that participants anchor the scales equivalently for humans and AI. While some cross-agent comparisons of effect magnitude could in principle be influenced by differential anchoring, the core finding, that intelligence increases perceived morality within each agent type, is robust to this concern. Moreover, across all studies we collected secondary measures in which participants rated the agent's traits relative to an 'average person' baseline (e.g., 'Compared to an average person, how moral do you think [OmegaAI/Tom] is?'), providing a common anchor for both agent types. These measures, reported in the Supplementary Material, produced qualitatively consistent results, suggesting that differential anchoring does not substantially alter the pattern of findings.</p><p id="dedd8b90-8a6a-4956-835c-41e7793fb2b4">Previous work looking at views related to artificially enhanced humans notably have findings pointing in the opposite direction from the pattern we observe: superhuman levels of cognitive ability achieved through neurotechnological enhancement were associated with perceived immorality and dehumanization <xref ref-type="bibr" rid="koverola_treatments_2022">(Koverola et al., 2022)</xref>. One possible explanation is that human enhancement triggered purity and naturalness concerns in those studies that are not applicable to AI systems. Future work would benefit from focusing further on this divergence.</p><p id="a0a8c4b3-1f1f-4d0d-906f-ed243ad4e2a3">Our studies manipulated intelligence and morality separately rather than simultaneously, but it would be interesting for future work to consider a fully crossed design, allowing for a more precise estimate of the relative contributions of each dimension to perceptions of the other, clarifying whether intelligence is perceived as necessary but not sufficient for morality.</p><p id="aed32710-6379-4aaa-bf63-2be03fe9b05c">Finally, our sample mainly consisted of British participants. Cross-cultural studies could investigate whether our orthogonality-resistant intuitions about intelligence and morality are a universal phenomenon or whether it is shaped by differing cultural attitudes toward morality, intelligence, and technology.</p></sec><sec id="s-a45ff5c1-c903-4e0d-b990-a4bbaec59f48" sec-type="chapter"><title>Implications</title><p id="ad719f46-2994-41ac-a096-efd385c36817">Together, our results provide compelling evidence for lay intuitions about intelligence and morality being connected in artificial intelligence, suggesting that people intuitively associate greater intelligence with not only the cognitive abilities that are deeply intertwined with moral reasoning but also the purely motivational components of morality. This cannot be explained by the possibility that people expect increases in intelligence to merely coincide with increases in morality, since it occurs in both within-subjects augmentation paradigms (Pilot Studies 2a and 2b, and Studies 1b and 2b) and in explicit questions about how this change would directly affect another trait (Study 3). It cannot be explained by the possibility that people are merely thinking of an AI’s morality in terms of competences like predicting or repeating back moral norms, because people also perceive an AI that becomes more able to achieve its own goals would also <italic>care</italic> more about humans too by an increase in its moral motivation (Studies 2a, 2b, and 3).</p><p id="b904215e-79fd-4546-8899-9d975029d073">These findings have important implications for how AI advancements are perceived in the public sphere. Media narratives often emphasize rapid progress in AI intelligence. If people are resistant to orthogonality, then this may implicitly suggest that such progress will inherently and automatically lead to safer or more ethical systems. If people are particularly impressed with their interactions with large language models like OpenAI’s ChatGPT, Google’s Gemini, or Anthropic’s Claude, and commentators frequently highlight the impressive leaps in these systems’ abilities to engage in what appear to be complex reasoning, this may create an undue sense of optimism about their moral reliability either now or in the future. For example, stories about AI being used to resolve ethical dilemmas—such as applications in healthcare to allocate scarce medical resources <xref ref-type="bibr" rid="drezga-kleiminger_should_2023">(Drezga-Kleiminger et al., 2023; Vinay et al., 2021)</xref>—often highlight the AI’s capacity to analyze data and make “rational” decisions but rarely address whether such systems genuinely understand moral norms or adhere to them in ways aligned with human values <xref ref-type="bibr" rid="belislepipon2022">(Bélisle-Pipon et al., 2022)</xref> much less how to think about the goals of complex systems based on deep neural networks, which may forever escape full human understanding <xref ref-type="bibr" rid="myers2024interactive">(Castelvecchi, 2016; Eschenbach, 2021; Myers &amp; Chater, 2024)</xref>. Similarly, discussions around autonomous vehicles often underscore their advanced decision-making capabilities <xref ref-type="bibr" rid="bonnefon_social_2016">(Bonnefon et al., 2016; Takaguchi et al., 2022)</xref>, potentially leading to people assuming that improved intelligence translates to safer, more ethical driving decisions without fully accounting for the moral trade-offs these vehicles might face in real-world scenarios <xref ref-type="bibr" rid="awad_moral_2018">(Awad et al., 2018)</xref>.</p><p id="a5843774-f623-447c-afad-b16aa7135fa1">If people believe that simply by making AI smarter, it automatically becomes safer, they may be less inclined to demand, and policymakers less inclined to require, the continued investment in safety engineering that actually produces the alignment they observe. In this way, the intelligence-morality inference could become a self-undermining prophecy as it is approximately accurate today precisely because of safety work that it may discourage in the future.</p><p id="f536c94b-1a44-4111-a615-e11018582dba">Optimistic media framing, even if it truly does reflect well how advanced current systems are becoming or how advanced they may become in the future, may inadvertently reinforce the very bias our experiments reveal: the intuitive but potentially unwarranted assumption that advancements in intelligence inherently come with corresponding advancements in morality. By failing to scrutinize whether and how AI systems are explicitly designed to address moral concerns, such narratives may lead to complacency about AI safety. As our findings suggest, people’s natural resistance to orthogonality might cloud their ability to critically assess whether AI systems are truly aligned with human values, leading to misplaced trust in these technologies.</p></sec><sec id="s-aa3c9beb-d8e0-42da-a7e2-6117b8a93b8c" sec-type="chapter"><title>Conclusions</title><p id="a503ee5b-4cd7-4e72-b2ec-0bfa28c7b98a">Is an intelligent machine guaranteed to be a moral machine? There are good reasons to assume not. But do ordinary people perceive there to be a necessary connection between intelligence and morality in artificial agents, and what are the consequences of this on trust and danger? In this paper we studied the lay intuitions about intelligence and morality in AI, showing across a series of pre-registered studies that lay perceptions of intelligence and morality are tightly intertwined, with important consequences for how people think about AI development and safety. Across multiple experiments, we consistently found that higher intelligence in AI was associated with greater perceptions of morality and trustworthiness, in contrast to the philosophical and theoretical independence of these traits as outlined by the Orthogonality Thesis. These findings highlight the risks of optimistic narratives that oversell (or even accurately sell) the intelligence of AI and how they might lead to misplaced trust in AI systems. As AI continues to advance and narratives around its capabilities proliferate, it is critical to address misconceptions so as to foster more realistic and informed public discourse about the risks and benefits of AI technologies. Our findings emphasize the importance of incorporating a balanced understanding of intelligence and morality in AI design and policymaking, ensuring that public trust is based on actual safety measures rather than optimistic or anthropocentric biases.</p><p id="a238d36f-6122-4e2e-9c43-75eb41da75e5"/></sec></sec><sec id="p-a14723d1-36de-4f9e-beda-554a6a3adaad" sec-type="chapter"><title>Author Contributions (CRediT)</title><p id="a9df5351-8614-4c5a-bb05-f6ce213dd6be">SM: Conceptualization, Data Curation, Formal Analysis, Investigation, Methodology, Validation, Visualization, Writing – Original Draft Preparation, Writing – Review &amp; Editing.</p><p id="mhxxl7kuu9bhg">JACE: Formal Analysis, Funding Acquisition, Methodology, Supervision, Validation, Writing – Review &amp; Editing, Visualization.</p></sec><sec id="p-b4fbe214-a511-4055-b465-7a33d65dd8fb" sec-type="bibliography"><title>References </title></sec></body><back><ack><sec id="s-acknowledgments"><title>Acknowledgments</title><p id="a9008414-34b1-40b9-9654-13c426961afb">This work was generously supported by funding from the Economic and Social Research Council (ES/V015176/1) and a Philip Leverhulme Prize (PLP-2021-095) awarded to JACE<italic>.</italic></p><p id="ae32111f-b24b-4c9c-80f7-1abbfbdf065c"/><p id="f253f5ab-c346-467c-864d-c0a3652a87d2"/></sec><sec sec-type="cite-as"><title>Cite as</title><p>Myers, S., &amp; Everett, J. A. C. (2026). <italic>Is an Intelligent Machine a Moral Machine?</italic></p></sec></ack><ref-list><title>References</title><ref id="abele_agency_2007"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Abele</surname><given-names>Andrea E.</given-names></name><name><surname>Wojciszke</surname><given-names>Bogdan</given-names></name></person-group><year>2007</year><article-title>Agency and communion from the perspective of self versus others.</article-title><source>Journal of Personality and Social Psychology</source><volume>93</volume><pub-id pub-id-type="doi">10.1037/0022-3514.93.5.751</pub-id></element-citation></ref><ref id="abele_communal_2014"><element-citation publication-type="chapter"><person-group person-group-type="author"><name><surname>Abele</surname><given-names>Andrea E.</given-names></name><name><surname>Wojciszke</surname><given-names>Bogdan</given-names></name></person-group><year>2014</year><article-title>Communal and agentic content in social cognition</article-title><source>Advances in Experimental Social Psychology</source><volume>50</volume><pub-id pub-id-type="doi">10.1016/B978-0-12-800284-1.00004-7</pub-id></element-citation></ref><ref id="adams_gewirth_1980"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Adams</surname><given-names>E. M.</given-names></name></person-group><year>1980</year><article-title>Gewirth on Reason and Morality</article-title><source>The Review of Metaphysics</source><volume>33</volume></element-citation></ref><ref id="de_brigard_ethical_2022"><element-citation publication-type="chapter"><person-group person-group-type="author"><name><surname>Aharoni</surname><given-names>Eyal</given-names></name><name><surname>Abdulla</surname><given-names>Sara</given-names></name><name><surname>Allen</surname><given-names>Corey H.</given-names></name><name><surname>Nadelhoffer</surname><given-names>Thomas</given-names></name></person-group><year>20222</year><article-title>Ethical implications of neurobiologically informed risk assessment for criminal justice decisions: A case for pragmatism</article-title><source>Neuroscience and Philosophy</source><pub-id pub-id-type="doi">10.7551/mitpress/12611.003.0010</pub-id></element-citation></ref><ref id="anthropic_notitle_2025"><element-citation publication-type=""><person-group person-group-type="author"><name><surname>Anthropic</surname><given-names/></name></person-group><year>2025</year><source>System cards</source></element-citation></ref><ref id="armstrong_general_2013"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Armstrong</surname><given-names>S</given-names></name></person-group><year>2013</year><article-title>General purpose intelligence: Arguing the orthogonality thesis.</article-title><source>Analysis and Metaphysics</source></element-citation></ref><ref id="awad_moral_2018"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Awad</surname><given-names>Edmond</given-names></name><name><surname>Dsouza</surname><given-names>Sohan</given-names></name><name><surname>Kim</surname><given-names>Richard</given-names></name><name><surname>Schulz</surname><given-names>Jonathan</given-names></name><name><surname>Henrich</surname><given-names>Joseph</given-names></name><name><surname>Shariff</surname><given-names>Azim</given-names></name><name><surname>Bonnefon</surname><given-names>Jean-François</given-names></name><name><surname>Rahwan</surname><given-names>Iyad</given-names></name></person-group><year>201811</year><article-title>The moral machine experiment</article-title><source>Nature</source><volume>563</volume><pub-id pub-id-type="doi">10.1038/s41586-018-0637-6</pub-id></element-citation></ref><ref id="bai_constitutional_2022"><mixed-citation>Bai, Y., Kadavath, S., Kundu, S., Askell, A., Kernion, J., Jones, A., Chen, A., Goldie, A., Mirhoseini, A., McKinnon, C., Chen, C., Olsson, C., Olah, C., Hernandez, D., Drain, D., Ganguli, D., Li, D., Tran-Johnson, E., Perez, E., … Kaplan, J. (2022). <italic>Constitutional AI: Harmlessness from AI feedback</italic>. arXiv. https://doi.org/10.48550/arXiv.2212.08073</mixed-citation></ref><ref id="barber_logic_1983"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Barber</surname><given-names>Bernhard</given-names></name></person-group><year>1983</year><article-title>The Logic and limits of trust</article-title></element-citation></ref><ref id="black_gap_1964"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Black</surname><given-names>Max</given-names></name></person-group><year>19644</year><article-title>The gap between "is" and "should"</article-title><source>The Philosophical Review</source><volume>73</volume><pub-id pub-id-type="doi">10.2307/2183334</pub-id></element-citation></ref><ref id="blackburn_ruling_1998"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Blackburn</surname><given-names>S</given-names></name></person-group><year>1998</year><article-title>Ruling passions: A theory of practical reasoning</article-title></element-citation></ref><ref id="bonnefon_social_2016"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Bonnefon</surname><given-names>Jean-François</given-names></name><name><surname>Shariff</surname><given-names>Azim</given-names></name><name><surname>Rahwan</surname><given-names>Iyad</given-names></name></person-group><year>20166</year><article-title>The social dilemma of autonomous vehicles</article-title><source>Science</source><volume>352</volume><pub-id pub-id-type="doi">10.1126/science.aaf2654</pub-id></element-citation></ref><ref id="bostrom_existential_2002"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Bostrom</surname><given-names>N</given-names></name></person-group><year>2002</year><article-title>Existential risks: Analyzing human extinction scenarios and related hazards</article-title><source>Journal of Evolution and Technology</source><volume>9</volume></element-citation></ref><ref id="bostrom_superintelligent_2012"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Bostrom</surname><given-names>Nick</given-names></name></person-group><year>20125</year><article-title>The superintelligent will: Motivation and instrumental rationality in advanced artificial agents</article-title><source>Minds and Machines</source><volume>22</volume><pub-id pub-id-type="doi">10.1007/s11023-012-9281-3</pub-id></element-citation></ref><ref id="bostrom_superintelligence_2014"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Bostrom</surname><given-names>Nick</given-names></name></person-group><year>2014</year><article-title>Superintelligence: Paths, dangers, strategies</article-title></element-citation></ref><ref id="brambilla_primacy_2021"><element-citation publication-type="chapter"><person-group person-group-type="author"><name><surname>Brambilla</surname><given-names>Marco</given-names></name><name><surname>Sacchi</surname><given-names>Simona</given-names></name><name><surname>Rusconi</surname><given-names>Patrice</given-names></name><name><surname>Goodwin</surname><given-names>Geoffrey P.</given-names></name></person-group><year>2021</year><article-title>The primacy of morality in impression development: Theory, research, and future directions</article-title><source>Advances in Experimental Social Psychology</source><volume>64</volume><pub-id pub-id-type="doi">10.1016/bs.aesp.2021.03.001</pub-id></element-citation></ref><ref id="cao_ai_2022"><mixed-citation publication-type="article-journal">Cao, L. (2022). AI in finance: Challenges, techniques, and opportunities. <italic>ACM Computing Surveys, 55</italic>(3), 1–38. https://doi.org/10.1145/3502289</mixed-citation></ref><ref id="castelvecchi_can_2016"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Castelvecchi</surname><given-names>Davide</given-names></name></person-group><year>201610</year><article-title>Can we open the black box of AI?</article-title><source>Nature</source><volume>538</volume><pub-id pub-id-type="doi">10.1038/538020a</pub-id></element-citation></ref><ref id="christian_alignment_2021"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Christian</surname><given-names>Brian</given-names></name></person-group><year>2021</year><article-title>The alignment problem: How can artificial intelligence learn human values?</article-title></element-citation></ref><ref id="claessens_trust_2026"><element-citation publication-type=""><person-group person-group-type="author"><name><surname>Claessens</surname><given-names>S</given-names></name><name><surname>Everett</surname><given-names>Jim A. C.</given-names></name></person-group><year>2026</year><article-title>Trust in artificial intelligence is agent-specific and multidimensional</article-title></element-citation></ref><ref id="dahl_what_2023"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Dahl</surname><given-names>Audun</given-names></name></person-group><year>20234</year><article-title>What we do when we define morality(And why we need to do it)</article-title><source>Psychological Inquiry</source><volume>34</volume><pub-id pub-id-type="doi">10.1080/1047840X.2023.2248854</pub-id></element-citation></ref><ref id="de_neys_humans_2025"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>De Neys</surname><given-names>Wim</given-names></name><name><surname>Raoelison</surname><given-names>Matthieu</given-names></name></person-group><year>20259</year><article-title>Humans and LLMs rate deliberation as superior to intuition on complex reasoning tasks</article-title><source>Communications Psychology</source><volume>3</volume><pub-id pub-id-type="doi">10.1038/s44271-025-00320-8</pub-id></element-citation></ref><ref id="dentella_testing_2024"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Dentella</surname><given-names>Vittoria</given-names></name><name><surname>Günther</surname><given-names>Fritz</given-names></name><name><surname>Murphy</surname><given-names>Elliot</given-names></name><name><surname>Marcus</surname><given-names>Gary</given-names></name><name><surname>Leivada</surname><given-names>Evelina</given-names></name></person-group><year>202411</year><article-title>Testing AI on language comprehension tasks reveals insensitivity to underlying meaning</article-title><source>Scientific Reports</source><volume>14</volume><pub-id pub-id-type="doi">10.1038/s41598-024-79531-8</pub-id></element-citation></ref><ref id="deutsch_effect_1960"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Deutsch</surname><given-names>Morton</given-names></name></person-group><year>19605</year><article-title>The effect of motivational orientation upon trust and suspicion</article-title><source>Human Relations</source><volume>13</volume><pub-id pub-id-type="doi">10.1177/001872676001300202</pub-id></element-citation></ref><ref id="drezga-kleiminger_should_2023"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Drezga-Kleiminger</surname><given-names>Max</given-names></name><name><surname>Demaree-Cotton</surname><given-names>Joanna</given-names></name><name><surname>Koplin</surname><given-names>Julian</given-names></name><name><surname>Savulescu</surname><given-names>Julian</given-names></name><name><surname>Wilkinson</surname><given-names>Dominic</given-names></name></person-group><year>202311</year><article-title>Should AI allocate livers for transplant? Public attitudes and ethical considerations</article-title><source>BMC Medical Ethics</source><volume>24</volume><pub-id pub-id-type="doi">10.1186/s12910-023-00983-0</pub-id></element-citation></ref><ref id="dubois_normativeness_2005"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Dubois</surname><given-names>Nicole</given-names></name><name><surname>Beauvois</surname><given-names>Jean-L�on</given-names></name></person-group><year>20051</year><article-title>Normativeness and individualism</article-title><source>European Journal of Social Psychology</source><volume>35</volume><pub-id pub-id-type="doi">10.1002/ejsp.236</pub-id></element-citation></ref><ref id="duff_psychopathy_1977"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Duff</surname><given-names>Antony</given-names></name></person-group><year>1977</year><article-title>Psychopathy and Moral Understanding</article-title><source>American Philosophical Quarterly</source><volume>14</volume></element-citation></ref><ref id="epley_seeing_2007"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Epley</surname><given-names>Nicholas</given-names></name><name><surname>Waytz</surname><given-names>Adam</given-names></name><name><surname>Cacioppo</surname><given-names>John T.</given-names></name></person-group><year>2007</year><article-title>On seeing human: A three-factor theory of anthropomorphism.</article-title><source>Psychological Review</source><volume>114</volume><pub-id pub-id-type="doi">10.1037/0033-295X.114.4.864</pub-id></element-citation></ref><ref id="eschenbach_transparency_2021"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Eschenbach</surname><given-names>Warren J.</given-names></name></person-group><year>202112</year><article-title>Transparency and the black box problem: Why we do not trust ai</article-title><source>Philosophy &amp; Technology</source><volume>34</volume><pub-id pub-id-type="doi">10.1007/s13347-021-00477-0</pub-id></element-citation></ref><ref id="esmaeilzadeh_use_2020"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Esmaeilzadeh</surname><given-names>Pouyan</given-names></name></person-group><year>202012</year><article-title>Use of AI-based tools for healthcare purposes: A survey study from consumers’ perspectives</article-title><source>BMC Medical Informatics and Decision Making</source><volume>20</volume><pub-id pub-id-type="doi">10.1186/s12911-020-01191-1</pub-id></element-citation></ref><ref id="everett_principles_2026"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Everett</surname><given-names>Jim A. C.</given-names></name><name><surname>Claessens</surname><given-names>Scott</given-names></name><name><surname>Knöchel</surname><given-names>Tim-Dorian</given-names></name><name><surname>Reinecke</surname><given-names>Madeline G.</given-names></name></person-group><year>20264</year><article-title>Principles for understanding trust in artificial intelligence</article-title><source>Nature Reviews Psychology</source><pub-id pub-id-type="doi">10.1038/s44159-026-00562-1</pub-id></element-citation></ref><ref id="fiske_universal_2007"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Fiske</surname><given-names>Susan T.</given-names></name><name><surname>Cuddy</surname><given-names>Amy J. C.</given-names></name><name><surname>Glick</surname><given-names>Peter</given-names></name></person-group><year>20072</year><article-title>Universal dimensions of social cognition: Warmth and competence</article-title><source>Trends in Cognitive Sciences</source><volume>11</volume><pub-id pub-id-type="doi">10.1016/j.tics.2006.11.005</pub-id></element-citation></ref><ref id="gabriel_artificial_2020"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Gabriel</surname><given-names>Iason</given-names></name></person-group><year>20209</year><article-title>Artificial intelligence, values, and alignment</article-title><source>Minds and Machines</source><volume>30</volume><pub-id pub-id-type="doi">10.1007/s11023-020-09539-2</pub-id></element-citation></ref><ref id="gewirth_justification_1988"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Gewirth</surname><given-names>Alan</given-names></name></person-group><year>19883</year><article-title>The justification of morality</article-title><source>Philosophical Studies</source><volume>53</volume><pub-id pub-id-type="doi">10.1007/BF00354643</pub-id></element-citation></ref><ref id="giubilini_artificial_2018"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Giubilini</surname><given-names>Alberto</given-names></name><name><surname>Savulescu</surname><given-names>Julian</given-names></name></person-group><year>20186</year><article-title>The artificial moral advisor. The “ideal observer” meets artificial intelligence</article-title><source>Philosophy &amp; Technology</source><volume>31</volume><pub-id pub-id-type="doi">10.1007/s13347-017-0285-z</pub-id></element-citation></ref><ref id="goodwin_moral_2014"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Goodwin</surname><given-names>Geoffrey P.</given-names></name><name><surname>Piazza</surname><given-names>Jared</given-names></name><name><surname>Rozin</surname><given-names>Paul</given-names></name></person-group><year>2014</year><article-title>Moral character predominates in person perception and evaluation.</article-title><source>Journal of Personality and Social Psychology</source><volume>106</volume><pub-id pub-id-type="doi">10.1037/a0034726</pub-id></element-citation></ref><ref id="grinschgl_who_2023"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Grinschgl</surname><given-names>Sandra</given-names></name><name><surname>Berdnik</surname><given-names>Anna-Lena</given-names></name><name><surname>Stehling</surname><given-names>Elisabeth</given-names></name><name><surname>Hofer</surname><given-names>Gabriela</given-names></name><name><surname>Neubauer</surname><given-names>Aljoscha C</given-names></name></person-group><year>20236</year><article-title>Who wants to enhance their cognitive abilities? Potential predictors of the acceptance of cognitive enhancement</article-title><source>Journal of Intelligence</source><volume>11</volume><pub-id pub-id-type="doi">10.3390/jintelligence11060109</pub-id></element-citation></ref><ref id="grinschgl_human_2022"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Grinschgl</surname><given-names>Sandra</given-names></name><name><surname>Tawakol</surname><given-names>Zadaf</given-names></name><name><surname>Neubauer</surname><given-names>Aljoscha C.</given-names></name></person-group><year>20225</year><article-title>Human enhancement and personality: A new approach towards investigating their relationship</article-title><source>Heliyon</source><volume>8</volume><pub-id pub-id-type="doi">10.1016/j.heliyon.2022.e09359</pub-id></element-citation></ref><ref id="herpertz_emotional_2000"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Herpertz</surname><given-names>Sabine C.</given-names></name><name><surname>Sass</surname><given-names>Henning</given-names></name></person-group><year>200010</year><article-title>Emotional deficiency and psychopathy</article-title><source>Behavioral Sciences &amp; the Law</source><volume>18</volume><pub-id pub-id-type="doi">10.1002/1099-0798(200010)18:5&lt;567::AID-BSL410&gt;3.0.CO;2-8</pub-id></element-citation></ref><ref id="hume_treatise_1896"><mixed-citation publication-type="book">Hume, D. (1896). <italic>A treatise of human nature</italic>. Clarendon Press.</mixed-citation></ref><ref id="jian_foundations_2000"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Jian</surname><given-names>Jiun-Yin</given-names></name><name><surname>Bisantz</surname><given-names>Ann M.</given-names></name><name><surname>Drury</surname><given-names>Colin G.</given-names></name></person-group><year>20003</year><article-title>Foundations for an empirically determined scale of trust in automated systems</article-title><source>International Journal of Cognitive Ergonomics</source><volume>4</volume><pub-id pub-id-type="doi">10.1207/S15327566IJCE0401_04</pub-id></element-citation></ref><ref id="kannegieter_ai_2024"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Kannegieter</surname><given-names>Trent</given-names></name></person-group><year>20248</year><article-title>AI timelines and national security: The obstacles to AGI by 2027</article-title><source>Lawfare</source></element-citation></ref><ref id="kim_removing_2004"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Kim</surname><given-names>Peter H.</given-names></name><name><surname>Ferrin</surname><given-names>Donald L.</given-names></name><name><surname>Cooper</surname><given-names>Cecily D.</given-names></name><name><surname>Dirks</surname><given-names>Kurt T.</given-names></name></person-group><year>2004</year><article-title>Removing the shadow of suspicion: The effects of apology versus denial for repairing competence- versus integrity-based trust violations.</article-title><source>Journal of Applied Psychology</source><volume>89</volume><pub-id pub-id-type="doi">10.1037/0021-9010.89.1.104</pub-id></element-citation></ref><ref id="kobis_delegation_2025"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Köbis</surname><given-names>Nils</given-names></name><name><surname>Rahwan</surname><given-names>Zoe</given-names></name><name><surname>Rilla</surname><given-names>Raluca</given-names></name><name><surname>Supriyatno</surname><given-names>Bramantyo Ibrahim</given-names></name><name><surname>Bersch</surname><given-names>Clara</given-names></name><name><surname>Ajaj</surname><given-names>Tamer</given-names></name><name><surname>Bonnefon</surname><given-names>Jean-François</given-names></name><name><surname>Rahwan</surname><given-names>Iyad</given-names></name></person-group><year>202510</year><article-title>Delegation to artificial intelligence can increase dishonest behaviour</article-title><source>Nature</source><volume>646</volume><pub-id pub-id-type="doi">10.1038/s41586-025-09505-x</pub-id></element-citation></ref><ref id="koverola_treatments_2022"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Koverola</surname><given-names>Mika</given-names></name><name><surname>Kunnari</surname><given-names>Anton</given-names></name><name><surname>Drosinou</surname><given-names>Marianna</given-names></name><name><surname>Palomäki</surname><given-names>Jussi</given-names></name><name><surname>Hannikainen</surname><given-names>Ivar R.</given-names></name><name><surname>Jirout Košová</surname><given-names>Michaela</given-names></name><name><surname>Kopecký</surname><given-names>Robin</given-names></name><name><surname>Sundvall</surname><given-names>Jukka</given-names></name><name><surname>Laakasuo</surname><given-names>Michael</given-names></name></person-group><year>20229</year><article-title>Treatments approved, boosts eschewed: Moral limits of neurotechnological enhancement</article-title><source>Journal of Experimental Social Psychology</source><volume>102</volume><pub-id pub-id-type="doi">10.1016/j.jesp.2022.104351</pub-id></element-citation></ref><ref id="laakasuo_moral_2025"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Laakasuo</surname><given-names>Michael</given-names></name><name><surname>Kunnari</surname><given-names>Anton</given-names></name><name><surname>Francis</surname><given-names>Kathryn</given-names></name><name><surname>Košová</surname><given-names>Michaela Jirout</given-names></name><name><surname>Kopecký</surname><given-names>Robin</given-names></name><name><surname>Buttazzoni</surname><given-names>Paolo</given-names></name><name><surname>Koverola</surname><given-names>Mika</given-names></name><name><surname>Palomäki</surname><given-names>Jussi</given-names></name><name><surname>Drosinou</surname><given-names>Marianna</given-names></name><name><surname>Hannikainen</surname><given-names>Ivar</given-names></name></person-group><year>20259</year><article-title>Moral psychological exploration of the asymmetry effect in AI-assisted euthanasia decisions</article-title><source>Cognition</source><volume>262</volume><pub-id pub-id-type="doi">10.1016/j.cognition.2025.106177</pub-id></element-citation></ref><ref id="laakasuo_moral_2023"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Laakasuo</surname><given-names>Michael</given-names></name><name><surname>Palomäki</surname><given-names>Jussi</given-names></name><name><surname>Kunnari</surname><given-names>Anton</given-names></name><name><surname>Rauhala</surname><given-names>Sanna</given-names></name><name><surname>Drosinou</surname><given-names>Marianna</given-names></name><name><surname>Halonen</surname><given-names>Juho</given-names></name><name><surname>Lehtonen</surname><given-names>Noora</given-names></name><name><surname>Koverola</surname><given-names>Mika</given-names></name><name><surname>Repo</surname><given-names>Marko</given-names></name><name><surname>Sundvall</surname><given-names>Jukka</given-names></name><name><surname>Visala</surname><given-names>Aku</given-names></name><name><surname>Francis</surname><given-names>Kathryn B.</given-names></name></person-group><year>20232</year><article-title>Moral psychology of nursing robots: Exploring the role of robots in dilemmas of patient autonomy</article-title><source>European Journal of Social Psychology</source><volume>53</volume><pub-id pub-id-type="doi">10.1002/ejsp.2890</pub-id></element-citation></ref><ref id="landes_people_2026"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Landes</surname><given-names>Ethan</given-names></name><name><surname>Francis</surname><given-names>Kathryn B.</given-names></name><name><surname>Everett</surname><given-names>Jim A. C.</given-names></name></person-group><year>20267</year><article-title>People defer to AI moral advice, but not blindly</article-title><source>Cognition</source><volume>272</volume><pub-id pub-id-type="doi">10.1016/j.cognition.2026.106504</pub-id></element-citation></ref><ref id="lee_trust_2004"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Lee</surname><given-names>J. D.</given-names></name><name><surname>See</surname><given-names>K. A.</given-names></name></person-group><year>20041</year><article-title>Trust in automation: Designing for appropriate reliance</article-title><source>Human Factors: The Journal of the Human Factors and Ergonomics Society</source><volume>46</volume><pub-id pub-id-type="doi">10.1518/hfes.46.1.50_30392</pub-id></element-citation></ref><ref id="maibom_mad_2008"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Maibom</surname><given-names>Heidi L.</given-names></name></person-group><year>200810</year><article-title>The mad, the bad, and the psychopath</article-title><source>Neuroethics</source><volume>1</volume><pub-id pub-id-type="doi">10.1007/s12152-008-9013-9</pub-id></element-citation></ref><ref id="malle_sacrifice_2015"><element-citation publication-type="paper-conference"><person-group person-group-type="author"><name><surname>Malle</surname><given-names>Bertram F.</given-names></name><name><surname>Scheutz</surname><given-names>Matthias</given-names></name><name><surname>Arnold</surname><given-names>Thomas</given-names></name><name><surname>Voiklis</surname><given-names>John</given-names></name><name><surname>Cusimano</surname><given-names>Corey</given-names></name></person-group><year>20153</year><article-title>Sacrifice one for the good of many? : People apply different moral norms to human and robot agents</article-title><source>Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction</source><pub-id pub-id-type="doi">10.1145/2696454.2696458</pub-id></element-citation></ref><ref id="malle_multidimensional_2021"><element-citation publication-type="chapter"><person-group person-group-type="author"><name><surname>Malle</surname><given-names>Bertram F.</given-names></name><name><surname>Ullman</surname><given-names>Daniel</given-names></name></person-group><year>2021</year><article-title>A multidimensional conception and measure of human-robot trust</article-title><source>Trust in Human-Robot Interaction</source><pub-id pub-id-type="doi">10.1016/B978-0-12-819472-0.00001-0</pub-id></element-citation></ref><ref id="marcus_deep_2018"><mixed-citation publication-type="document">Marcus, G. F. (2018). <italic>Deep learning: A critical appraisal</italic>. arXiv. https://doi.org/10.48550/arXiv.1801.00631</mixed-citation></ref><ref id="maynard_marketers_2024"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Maynard</surname><given-names>Mike</given-names></name></person-group><year>2024</year><article-title>Marketers, let’s stop overhyping what AI can do</article-title><source>Forbes Agency Council</source></element-citation></ref><ref id="mckee_humans_2023"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>McKee</surname><given-names>Kevin R.</given-names></name><name><surname>Bai</surname><given-names>Xuechunzi</given-names></name><name><surname>Fiske</surname><given-names>Susan T.</given-names></name></person-group><year>20238</year><article-title>Humans perceive warmth and competence in artificial intelligence</article-title><source>iScience</source><volume>26</volume><pub-id pub-id-type="doi">10.1016/j.isci.2023.107256</pub-id></element-citation></ref><ref id="nagel_possibility_2016"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Nagel</surname><given-names>Thomas</given-names></name></person-group><year>201611</year><article-title>The possibility of altruism</article-title><pub-id pub-id-type="doi">10.2307/j.ctt1ggjkt5</pub-id></element-citation></ref><ref id="narayanan_ai_2024"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Narayanan</surname><given-names>Arvind</given-names></name><name><surname>Kapoor</surname><given-names>Sayash</given-names></name></person-group><year>2024</year><article-title>AI snake oil: What artificial intelligence can do, what it can’t, and how to tell the difference</article-title></element-citation></ref><ref id="nass_machines_2000"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Nass</surname><given-names>Clifford</given-names></name><name><surname>Moon</surname><given-names>Youngme</given-names></name></person-group><year>20001</year><article-title>Machines and mindlessness: Social responses to computers</article-title><source>Journal of Social Issues</source><volume>56</volume><pub-id pub-id-type="doi">10.1111/0022-4537.00153</pub-id></element-citation></ref><ref id="nass_computers_1994"><element-citation publication-type="paper-conference"><person-group person-group-type="author"><name><surname>Nass</surname><given-names>Clifford</given-names></name><name><surname>Steuer</surname><given-names>Jonathan</given-names></name><name><surname>Tauber</surname><given-names>Ellen R.</given-names></name></person-group><year>1994</year><article-title>Computers are social actors</article-title><source>Conference companion on Human factors in computing systems - CHI ’94</source><pub-id pub-id-type="doi">10.1145/259963.260288</pub-id></element-citation></ref><ref id="nisbett_halo_1977"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Nisbett</surname><given-names>Richard E.</given-names></name><name><surname>Wilson</surname><given-names>Timothy D.</given-names></name></person-group><year>19774</year><article-title>The halo effect: Evidence for unconscious alteration of judgments.</article-title><source>Journal of Personality and Social Psychology</source><volume>35</volume><pub-id pub-id-type="doi">10.1037/0022-3514.35.4.250</pub-id></element-citation></ref><ref id="omohundro_basic_2018"><element-citation publication-type="chapter"><person-group person-group-type="author"><name><surname>Omohundro</surname><given-names>Stephen M.</given-names></name></person-group><year>2018</year><article-title>The basic AI drives</article-title><source>Artificial Intelligence Safety and Security</source></element-citation></ref><ref id="openai_planning_2023"><element-citation publication-type=""><person-group person-group-type="author"><name><surname>OpenAI</surname><given-names/></name></person-group><year>2023</year><article-title>Planning for AGI and beyond</article-title></element-citation></ref><ref id="ouyang_training_2022"><element-citation publication-type="paper-conference"><person-group person-group-type="author"><name><surname>Ouyang</surname><given-names>L</given-names></name><name><surname>Wu</surname><given-names>J</given-names></name><name><surname>Jiang</surname><given-names>X</given-names></name><name><surname>Almeida</surname><given-names>D</given-names></name><name><surname>Wainwright</surname><given-names>C</given-names></name><name><surname>Mishkin</surname><given-names>P</given-names></name><name><surname>Zhang</surname><given-names>Chong</given-names></name><name><surname>Agarwal</surname><given-names>Sandhini</given-names></name><name><surname>Slama</surname><given-names>Katarina</given-names></name><name><surname>Ray</surname><given-names>Alex</given-names></name><name><surname>Schulman</surname><given-names>John</given-names></name><name><surname>Hilton</surname><given-names>Jacob</given-names></name><name><surname>Kelton</surname><given-names>Fraser</given-names></name><name><surname>Miller</surname><given-names>Luke</given-names></name><name><surname>Simens</surname><given-names>Maddie</given-names></name><name><surname>Askell</surname><given-names>Amanda</given-names></name><name><surname>Welinder</surname><given-names>Peter</given-names></name><name><surname>Christiano</surname><given-names>Paul</given-names></name><name><surname>Leike</surname><given-names>Jan</given-names></name><name><surname>Lowe</surname><given-names>Ryan</given-names></name></person-group><year>2022</year><article-title>Training language models to follow instructions with human feedback</article-title><source>Advances in neural information processing systems</source><volume>35</volume></element-citation></ref><ref id="pangle_virtue_2014"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Pangle</surname><given-names>Lorraine Smith</given-names></name></person-group><year>2014</year><article-title>Virtue is knowledge: The moral foundations of Socratic political philosophy</article-title></element-citation></ref><ref id="plato_protagoras_1997"><element-citation publication-type="chapter"><person-group person-group-type="author"><name><surname>Plato</surname><given-names/></name></person-group><year>1997</year><article-title>Protagoras</article-title><source>Plato: Complete works</source></element-citation></ref><ref id="purcell_research_2023"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Purcell</surname><given-names>Zoe A.</given-names></name><name><surname>Bonnefon</surname><given-names>Jean-François</given-names></name></person-group><year>20234</year><article-title>Research on artificial intelligence is reshaping our definition of morality</article-title><source>Psychological Inquiry</source><volume>34</volume><pub-id pub-id-type="doi">10.1080/1047840X.2023.2248857</pub-id></element-citation></ref><ref id="purcell_writing_2025"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Purcell</surname><given-names>Zoe A.</given-names></name><name><surname>Jakesch</surname><given-names>Maurice</given-names></name><name><surname>Dong</surname><given-names>Mengchen</given-names></name><name><surname>Nussberger</surname><given-names>Anne-Marie</given-names></name><name><surname>Köbis</surname><given-names>Nils</given-names></name></person-group><year>202512</year><article-title>Writing with AI boosts trust-building efficiency</article-title><source>iScience</source><volume>28</volume><pub-id pub-id-type="doi">10.1016/j.isci.2025.114092</pub-id></element-citation></ref><ref id="putnam_collapse_2004"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Putnam</surname><given-names>Hilary</given-names></name></person-group><year>2004</year><article-title>The collapse of the fact/value dichotomy: And other essays</article-title></element-citation></ref><ref id="railton_moral_1986"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Railton</surname><given-names>P</given-names></name></person-group><year>1986</year><article-title>Moral realism</article-title><source>The philosophical review</source><volume>95</volume></element-citation></ref><ref id="reeves_media_1996"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Reeves</surname><given-names>Byron</given-names></name><name><surname>Nass</surname><given-names>C</given-names></name></person-group><year>1996</year><article-title>The media equation: How people treat computers, television, and new media like real people and places</article-title></element-citation></ref><ref id="reuters_teslas_2024"><element-citation publication-type=""><person-group person-group-type="author"><name><surname>Reuters</surname><given-names/></name></person-group><year>2024</year><article-title>Tesla’s Musk predicts AI will be smarter than the smartest human next year.</article-title></element-citation></ref><ref id="rosenberg_multidimensional_1968"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Rosenberg</surname><given-names>Seymour</given-names></name><name><surname>Nelson</surname><given-names>Carnot</given-names></name><name><surname>Vivekananthan</surname><given-names>P. S.</given-names></name></person-group><year>1968</year><article-title>A multidimensional approach to the structure of personality impressions.</article-title><source>Journal of Personality and Social Psychology</source><volume>9</volume><pub-id pub-id-type="doi">10.1037/h0026086</pub-id></element-citation></ref><ref id="roser_ai_2023"><element-citation publication-type=""><person-group person-group-type="author"><name><surname>Roser</surname><given-names>Max</given-names></name></person-group><year>2023</year><article-title>AI timelines: What do experts in artificial intelligence expect for the future?</article-title><source>Our World in Data</source></element-citation></ref><ref id="russell_artificial_2020"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Russell</surname><given-names>Stuart</given-names></name><name><surname>Norvig</surname><given-names>Peter</given-names></name></person-group><year>2020</year><article-title>Artificial intelligence: A modern approach</article-title></element-citation></ref><ref id="vlastos_socrates_1971"><mixed-citation publication-type="chapter">Santas, G. (1971). Socrates at work on virtue and knowledge in Plato’s <italic>Laches</italic>. In G. Vlastos (Ed.), The Philosophy of SOCRATES. Modern Studies in Philosophy (pp. 177–208). Palgrave Macmillan UK. https://doi.org/10.1007/978-1-349-86199-6_9</mixed-citation></ref><ref id="shafer-landau_moral_1998"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Shafer-Landau</surname><given-names>R</given-names></name></person-group><year>1998</year><article-title>Moral motivation and moral judgment.</article-title><source>The Philosophical Quarterly</source><volume>48</volume></element-citation></ref><ref id="shafer-landau_moral_2003"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Shafer-Landau</surname><given-names>Russ</given-names></name></person-group><year>2003</year><article-title>Moral realism: A defence</article-title></element-citation></ref><ref id="smith_moral_1994"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Smith</surname><given-names>M</given-names></name></person-group><year>1994</year><article-title>The moral problem.</article-title></element-citation></ref><ref id="stellar_unethical_2018"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Stellar</surname><given-names>Jennifer E.</given-names></name><name><surname>Willer</surname><given-names>Robb</given-names></name></person-group><year>20182</year><article-title>Unethical and inept? The influence of moral information on perceptions of competence.</article-title><source>Journal of Personality and Social Psychology</source><volume>114</volume><pub-id pub-id-type="doi">10.1037/pspa0000097</pub-id></element-citation></ref><ref id="sundvall_innocence_2023"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Sundvall</surname><given-names>Jukka</given-names></name><name><surname>Drosinou</surname><given-names>Marianna</given-names></name><name><surname>Hannikainen</surname><given-names>Ivar</given-names></name><name><surname>Elovaara</surname><given-names>Kaisa</given-names></name><name><surname>Halonen</surname><given-names>Juho</given-names></name><name><surname>Herzon</surname><given-names>Volo</given-names></name><name><surname>Kopecký</surname><given-names>Robin</given-names></name><name><surname>Jirout Košová</surname><given-names>Michaela</given-names></name><name><surname>Koverola</surname><given-names>Mika</given-names></name><name><surname>Kunnari</surname><given-names>Anton</given-names></name><name><surname>Perander</surname><given-names>Silva</given-names></name><name><surname>Saikkonen</surname><given-names>Teemu</given-names></name><name><surname>Palomäki</surname><given-names>Jussi</given-names></name><name><surname>Laakasuo</surname><given-names>Michael</given-names></name></person-group><year>20236</year><article-title>Innocence over utilitarianism: Heightened moral standards for robots in rescue dilemmas</article-title><source>European Journal of Social Psychology</source><volume>53</volume><pub-id pub-id-type="doi">10.1002/ejsp.2936</pub-id></element-citation></ref><ref id="surdel_judging_2024"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Surdel</surname><given-names>Nicholas</given-names></name><name><surname>Bigman</surname><given-names>Yochanan E.</given-names></name><name><surname>Shen</surname><given-names>Xi</given-names></name><name><surname>Lee</surname><given-names>Wen-Ying</given-names></name><name><surname>Jung</surname><given-names>Malte F.</given-names></name><name><surname>Ferguson</surname><given-names>Melissa J.</given-names></name></person-group><year>20245</year><article-title>Judging robot ability: How people form implicit and explicit impressions of robot competence.</article-title><source>Journal of Experimental Psychology: General</source><volume>153</volume><pub-id pub-id-type="doi">10.1037/xge0001548</pub-id></element-citation></ref><ref id="takaguchi_personal_2022"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Takaguchi</surname><given-names>Kazuya</given-names></name><name><surname>Kappes</surname><given-names>Andreas</given-names></name><name><surname>Yearsley</surname><given-names>James M.</given-names></name><name><surname>Sawai</surname><given-names>Tsutomu</given-names></name><name><surname>Wilkinson</surname><given-names>Dominic J. C.</given-names></name><name><surname>Savulescu</surname><given-names>Julian</given-names></name></person-group><year>202211</year><article-title>Personal ethical settings for driverless cars and the utility paradox: An ethical analysis of public attitudes in UK and Japan</article-title><source>PLOS ONE</source><volume>17</volume><pub-id pub-id-type="doi">10.1371/journal.pone.0275812</pub-id></element-citation></ref><ref id="takemoto_scaling_2026"><element-citation publication-type=""><person-group person-group-type="author"><name><surname>Takemoto</surname><given-names>Kazuhiro</given-names></name></person-group><year>20264</year><article-title>Scaling laws for moral machine judgment in large language models</article-title><pub-id pub-id-type="doi">10.48550/arXiv.2601.17637</pub-id></element-citation></ref><ref id="thorndike_constant_1920"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Thorndike</surname><given-names>E. L.</given-names></name></person-group><year>19203</year><article-title>A constant error in psychological ratings.</article-title><source>Journal of Applied Psychology</source><volume>4</volume><pub-id pub-id-type="doi">10.1037/h0071663</pub-id></element-citation></ref><ref id="turchin_ai_2019"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Turchin</surname><given-names>A</given-names></name></person-group><year>2019</year><article-title>AI alignment problem: “Human values” don’t actually exist.</article-title></element-citation></ref><ref id="turchin_classification_2020"><mixed-citation publication-type="article-journal">Turchin, A., &amp; Denkenberger, D. (2020). Classification of global catastrophic risks connected with artificial intelligence. <italic>AI &amp; Society, 35</italic>(1), 147–163. https://doi.org/10.1007/s00146-018-0845-5</mixed-citation></ref><ref id="vinay_ethics_2021"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Vinay</surname><given-names>Rasita</given-names></name><name><surname>Baumann</surname><given-names>Holger</given-names></name><name><surname>Biller-Andorno</surname><given-names>Nikola</given-names></name></person-group><year>20216</year><article-title>Ethics of ICU triage during COVID-19</article-title><source>British Medical Bulletin</source><volume>138</volume><pub-id pub-id-type="doi">10.1093/bmb/ldab009</pub-id></element-citation></ref><ref id="waser_discovering_2008"><element-citation publication-type="paper-conference"><person-group person-group-type="author"><name><surname>Waser</surname><given-names>M. R.</given-names></name></person-group><year>2008</year><article-title>Discovering the foundations of a universal system of ethics as a road to safe artificial intelligence.</article-title><source>AAAI Fall Symposium: Biologically Inspired Cognitive Architectures</source></element-citation></ref><ref id="wiggins_needs_1998"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Wiggins</surname><given-names>David</given-names></name></person-group><year>1998</year><article-title>Needs, values, truth: Essays in the philosophy of value</article-title><pub-id pub-id-type="doi">10.1093/oso/9780198237198.001.0001</pub-id></element-citation></ref><ref id="wojciszke_multiple_1994"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Wojciszke</surname><given-names>Bogdan</given-names></name></person-group><year>19948</year><article-title>Multiple meanings of behavior: Construing actions in terms of competence or morality.</article-title><source>Journal of Personality and Social Psychology</source><volume>67</volume><pub-id pub-id-type="doi">10.1037/0022-3514.67.2.222</pub-id></element-citation></ref><ref id="wojciszke_morality_2005"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Wojciszke</surname><given-names>Bogdan</given-names></name></person-group><year>20051</year><article-title>Morality and competence in person- and self-perception</article-title><source>European Review of Social Psychology</source><volume>16</volume><pub-id pub-id-type="doi">10.1080/10463280500229619</pub-id></element-citation></ref><ref id="wojciszke_dominance_1998"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Wojciszke</surname><given-names>Bogdan</given-names></name><name><surname>Bazinska</surname><given-names>Roza</given-names></name><name><surname>Jaworski</surname><given-names>Marcin</given-names></name></person-group><year>199812</year><article-title>On the dominance of moral categories in impression formation</article-title><source>Personality and Social Psychology Bulletin</source><volume>24</volume><pub-id pub-id-type="doi">10.1177/01461672982412001</pub-id></element-citation></ref><ref id="yogeeswaran_interactive_2016"><mixed-citation publication-type="article-journal">Yogeeswaran, K., Złotowski, J., Livingstone, M., Bartneck, C., Sumioka, H., &amp; Ishiguro, H. (2016). The interactive effects of robot anthropomorphism and robot ability on perceived threat and support for robotics research. <italic>Journal of Human-Robot Interaction</italic>, <italic>5</italic>(2), 29–47. https://doi.org/10.5898/JHRI.5.2.Yogeeswaran</mixed-citation></ref><ref id="yudkowsky_ai_2016"><element-citation publication-type="manuscript"><person-group person-group-type="author"><name><surname>Yudkowsky</surname><given-names>E</given-names></name></person-group><year>2016</year><article-title>The AI alignment problem: Why it’s hard, and where to start.</article-title></element-citation></ref><ref id="yudkowsky_if_2025"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Yudkowsky</surname><given-names>Eliezer</given-names></name><name><surname>Soares</surname><given-names>Nate</given-names></name></person-group><year>2025</year><article-title>If anyone builds it, everyone dies: The case against superintelligent AI</article-title></element-citation></ref><ref id="zaim_bin_ahmad_large-scale_2025"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Zaim Bin Ahmad</surname><given-names>Muhammad Shahrul</given-names></name><name><surname>Takemoto</surname><given-names>Kazuhiro</given-names></name></person-group><year>20255</year><article-title>Large-scale moral machine experiment on large language models</article-title><source>PLOS One</source><volume>20</volume><pub-id pub-id-type="doi">10.1371/journal.pone.0322776</pub-id></element-citation></ref><ref id="zlotowski_can_2017"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Złotowski</surname><given-names>Jakub</given-names></name><name><surname>Yogeeswaran</surname><given-names>Kumar</given-names></name><name><surname>Bartneck</surname><given-names>Christoph</given-names></name></person-group><year>20174</year><article-title>Can we control it? Autonomous robots threaten human identity, uniqueness, safety, and resources</article-title><source>International Journal of Human-Computer Studies</source><volume>100</volume><pub-id pub-id-type="doi">10.1016/j.ijhcs.2016.12.008</pub-id></element-citation></ref><ref id="jBAFFFFDCJ49963295184"><element-citation publication-type="paper-conference"><person-group person-group-type="author"><name><surname>Christiano</surname><given-names>Paul F.</given-names></name><name><surname>Leike</surname><given-names>Jan</given-names></name><name><surname>Brown</surname><given-names>Tom B.</given-names></name><name><surname>Martic</surname><given-names>Miljan</given-names></name><name><surname>Legg</surname><given-names>Shane</given-names></name><name><surname>Amodei</surname><given-names>Dario</given-names></name></person-group><year>2017</year><article-title>Deep reinforcement learning from human preferences</article-title><source>Proceedings of the 31st international conference on neural information processing systems</source></element-citation></ref><ref id="barr2013"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Barr</surname><given-names>D. J.</given-names></name><name><surname>Levy</surname><given-names>R.</given-names></name><name><surname>Scheepers</surname><given-names>C.</given-names></name><name><surname>Tily</surname><given-names>H. J.</given-names></name></person-group><year>2013</year><article-title>Random effects structure for confirmatory hypothesis testing: Keep it maximal</article-title><source>Journal of Memory and Language</source><volume>68</volume><pub-id pub-id-type="doi">10.1016/j.jml.2012.11.001</pub-id></element-citation></ref><ref id="belislepipon2022"><mixed-citation publication-type="article-journal">Bélisle-Pipon, J.-C., Monteferrante, E., Roy, M.-C., &amp; Couture, V. (2022). Artificial intelligence ethics has a black box problem. <italic>AI &amp; Society, 38</italic>, 1507–1522. https://doi.org/10.1007/s00146-021-01380-0</mixed-citation></ref><ref id="marcus2024"><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Marcus</surname><given-names>G. F.</given-names></name></person-group><year>2024</year><article-title>Taming silicon valley: How we can ensure that AI works for us</article-title></element-citation></ref><ref id="mayer1995"><mixed-citation publication-type="article-journal">Mayer, R. C., Davis, J. H., &amp; Schoorman, F. D. (1995). An integrative model of organizational trust. <italic>The Academy of Management Review</italic>, <italic>20</italic>(3), 709–734. https://doi.org/10.2307/258792 </mixed-citation></ref><ref id="myers2024interactive"><element-citation publication-type="article-journal"><person-group person-group-type="author"><name><surname>Myers</surname><given-names>Samantha</given-names></name><name><surname>Chater</surname><given-names>Nick</given-names></name></person-group><year>2024</year><article-title>Interactive explainability: Black boxes, mutual understanding and what it would really mean for AI systems to be as explainable as people</article-title><pub-id pub-id-type="doi">10.31234/osf.io/ha37x</pub-id></element-citation></ref></ref-list><fn-group content-type="footnotes"><title>Footnotes</title><fn id="fdf758de-5be4-44ea-8288-8297ff041b19"><label>1</label><p> The intelligence of the AI was described in terms of a breakthrough in vaccine research, potentially confounding intelligence with beneficial outcomes. The remaining studies therefore used domain-general descriptions of intelligence.</p></fn><fn id="d32ca4c6-5972-4a20-8ccf-6e3f7f432bbe"><label>2</label><p> For exclusion criteria and power analyses for sample size determination for this and subsequent studies, see Supplementary Materials.</p></fn><fn id="bc7807aa-ee95-4c78-8fad-71b75a82816e"><label>3</label><p> All vignettes across studies can be found in full in the supplementary materials.</p></fn><fn id="a924ade9-2e1d-4805-9117-ecbbb80d72a3"><label>4</label><p> Across the studies, secondary measures of Intelligence and Morality were also collected where participants rate the trait by comparison to an “average person” baseline. These are reported fully in the Supplementary Materials.These measures were qualitatively consistent with the measures reported in the main text.</p></fn><fn id="d1999faf-d204-495b-b8ae-0706cee3ffc8"><label>5</label><p> Across the studies, further measures of the participant’s familiarity with AI were collected and used as a covariate. These are reported fully in the Supplementary Materials.</p></fn><fn id="d5742c26-d8b3-4e21-b30d-40ffa53dfb58"><label>6</label><p> Descriptive statistics for all conditions are reported in Supplementary Tables.</p></fn><fn id="f3dc2db0-0502-43ae-b66b-25255f994300"><label>7</label><p> All mixed models used random intercepts for participants; full model specification details and convergence procedures are described in the Supplementary Material (following <xref ref-type="bibr" rid="barr2013">Barr et al., 2013</xref>).</p></fn></fn-group></back></article>