The term Affective Computing is used here, for analytical reasons, as a superordinate term for various forms of data processing that allow conclusions to be drawn about emotions, affects or other inner states1 based on one or multiple data sources such as the computer user’s facial or bodily changes, that are documented as expressions (visual data), temperature changes (of the whole body or body parts, the face or hands), voice or sound data, heart rate (ECG), skin resistance, brain activity (e.g. EEG) and others (Afzal et al., 2024; See for an overwiew Ahmadpour et al., 2025). Affective computing encompasses the functions and services known as emotion recognition.

The areas of application include health care, intelligent driving, security and identity checks as well as surveillance in public spaces, behavioral research, advertising analysis and general sentiment analysis. When AI-powered and connected to artificial neural networks (ANN), these forms of computing are referred to as Emotion AI in this contribution. Affective Computing is the umbrella term for a historical change in computing since the 1990s, while Emotion AI is the newer notion that describes the powerful combination of ANNs with affective computing.

This contribution examines Affective Computing as a new technical form of encoding emotions and suggests that it should therefore be critically understood in terms of its conceptual normative implications as virtual semantics. Ethical and conceptual normative questions in the discussions about affective computing have so far either focused on the possible social, moral and legal consequences of this technology (Weber-Guskar & Menges, 2025) or the reductionist conceptions of emotion and affect applied in these forms of computing (Crawford, 2021; Leys, 2017; Tuschling, 2014) especially since the fundamental criticism of psychologist Lisa Feldman Barrett and her team (Barrett, 2006; Barrett et al., 2019).

In this contribution, however, Affective Computing itself and not its possible consequences or underlying concepts will be examined as conceptually normative by proposing that it has to be understood as a virtual semantics. Here, I define virtual semantics as a set of logical-functional rules that provide a framework within which syntactically operating systems link their processed objects—or, in this case, digital data—with categorized valences and horizons/spaces of meaning. Regarding Affective Computing, this signifies the systematic connection or matching of clustered sensor data of the physiological measurements of computer users or captured beings like pedestrians in a public square with defined states2 that are decoded as emotions and come with a space of meaning (as in a certain pattern of measurements are matched to a state called sadness that is set in a space of meaning with a spectrum of states3 and experiences called depression, grief, melancholy). The term “virtual” in virtual semantics is used here both in the philosophical sense, in the French tradition and that of Charles Sanders Peirce as something that has similar effects and functions (Chalmers, 2023, page 187–188; Peirce, 1920; Sprenger, 2023, page 167–168), but a different physical form than natural-language semantics as well as a term for Affective Computing’s embeddedness in virtual life worlds (Rieger et al., 2020) .

In its implementation, the article uses a media-semiotic and media-archaeological approach (Parikka, 2012; Weatherby, 2025) and develops its assumptions about Affective Computing as a new and normative form of coding emotions in three steps: 1) From the outset, the term “code” was necessarily polysemic, even with regard to its origin. 2) What we think of as emotion and affect has always referred to representations of inner states and historically changing forms of coding experiences (historical examples include love letters and novels, and now computer-based recodings such as Affective Computing and Emotion AI. On this basis, 3) Affective Computing can be understood as a system for recoding older forms of emotional expression and as virtual semantics. With reference to digital media as a new environment that Mark Weiser first described (Alpsancar, 2012; Gramelsberger, 2023), Affective Computing takes on the role of a virtual semantics in these environments or worlds.

Against this background I want to point out the importance of a media-archeological perspective to fully understand the development and functionality of Affective Computing and propose the notion of a quasi-semantics or virtual semantics. Insofar as Affective Computing relies on different traditions to encode emotions in a specific way, it is always to be considered normative—often without disclosing its normative implications. The contribution presents its notion of Affective Computing as virtual semantics in three steps:

The term code refers to many different practices and contexts in history, culture and technology: For example, we talk about a dress code when special attire is required for certain occasions (Fig. 1). We are all familiar with codes of conduct that apply to public spaces and, nowadays, to our digital social exchange on platforms.

In his large-scale study on code as a cultural form, media historian Bernhard Dionysius Geoghegan describes how the historically changing forms of encoding information, but also language and ultimately culture itself, were addressed with the mathematical information theory in the advent of modern computers (Geoghegan, 2023). What had previously appeared as different rules, conventions and habits concerning such diverse things as dialects, the exchange of goods, marriage customs and aesthetic expressions in highly diverse societies, appeared through the lens of the mathematical theory of communication as the patterning of scientists such as Gregory Bateson, Margaret Mead and their colleagues in the colonies, hospitals and suburbs (Geoghegan, 2023, page 53 f.). For Geoghegan, the concept of code refers to a distributed and non-organized tendency to formalize previously implicit relationships, things and objects, as did the standardized family tree representations in ethnography, and thus make them at least potentially technically processable.

Geoghegan’s “code” spans an arc from the linguistic turn in language theory (Ferdinand de Saussure and others) and anthropology (Claude Lévi-Strauss) to contemporary practices of coding as an extension of information-theoretical cultural analysis (Warren Weaver). In this chain of powerful coding practices, I include the form of coding of emotions, affects and moods in computing and artificial intelligence, often on the basis of various physiological parameters that are read out by sensors, as in Affective Computing.

According to Bernhard Siegert, the previous century saw the replacement of humanistic logic and reason by technology in the form of media codes (Siegert, 2018)—that includes all sorts of content even, as I want to add, such intimate things as bodily changes, moods, whims, hesitations, ticks and the like.

In terms of media-semiotics, it was developed that the concept of code has both a syntactic and, indirectly, a semantic function. Furthermore, code in the sense developed here refers to natural sign systems such as spoken language, but also to techniques such as alphabetic writing and, above all, to sign-processing machines such as electronic digital computers. Conversely, however, this means that it is not only Affective Computing and Emotion AI that discretize and code human experiences, but that emotional messages conveyed through language have always been encoded, whether in art in its various forms, and of course in poetry, literature and even in personal love letters. This section traces now parts of the history of emotion transmission in order to describe the historical change and transfer of these forms of coding. The analysis presented here brings the negotiation of one’s own emotions in written form, as for example known from the tradition of love letter writing, among other things, closer to AI. At the same time, the focus on interlocking forms of coding serves to understand the special form of coding emotion-related data in AI as normative and to make its functioning problematic, because it embodies a shift to automation in the history of encoding emotions.

The historical and technical transition from a handwritten, emotionally charged letter-writing culture to mechanical and ultimately electronic digital word processing (Heilmann, 2012) became a theme in early computer programming projects. Between 1953 and 1954, several typewritten love letters appeared on the bulletin boards of the University of Manchester, passionately addressed to an undefined, gender-ambiguous other and signed with the initials MUC. They read something like this:

Darling Sweetheart,You are my avid fellow feeling. My affection curiously clings to your passionate wish. My liking yearns for your heart. You are my wistful sympathy: my tender liking.Yours beautifully,M. U. C. (Strachey, 1954, page 26)

These were most likely the first love letters generated by a computer algorithm, written by programmer Christopher Strachey. Strachey later went down in history primarily as the inventor of time-sharing (Corbató et al., 1962). Originally a mathematician, Strachey came to computer development indirectly and after personal crises through a friendship with Alan Turing (Campbell-Kelly, 1985). Due to his unusual approach to computing, Christopher Strachey is increasingly being recognized in media and literary studies, although the amount of work on him remains relatively small in comparison. Noteworthy examples include an early short biography by Martin Campbell-Kelly, Jacob Gaboury’s appreciation of Strachey in the context of queer computing (Gaboury, 2013), and the discussion of his contribution to stochastic poetry (Bernhart & Richter, 2021).

During a period when he had little to do, his biographer Martin Campbell-Kelly reports that Strachey relieved his boredom by writing the so-called Love Letter Generator for the Manchester University computer (Ferranti Mark I) (Campbell-Kelly, 1985). His sister Barbara advised him on the selection of possible words from the Roget’s Thesaurus to be used by the algorithm (Strachey, 1954, page 27).

The algorithm worked with surprisingly few, but well-chosen words, which were divided into six groups: greetings (Saluations1, Salutations2), adjectives, nouns, adverbs, verbs (Strachey, 1954, page 26). The computer scientist with the handle gingerbeardman forensically reconstructed the code for an exhibition and documented it on a website and on Github (see https://www.gingerbeardman.com/loveletter/). If we now apply Eco’s media-semiotic concept of code from the previous section to the situation, these word groups not only represent the syntactic possibilities, but also define the semantic framework and thus, in terms of media technology, the affective spectrum of meaning of the “love” in the generated letters. Here we see a shift from the letter-writer’s own choice of words—that is, a form of self-coding—to a greater degree of external preselection through programming—that is, a form of external coding.

Strachey stands out from the group of early computer developers because he did not view programming as a purpose-driven activity. Toni Bernhart and Sandra Richter therefore attest to the “playful nature of his programming work” (Bernhart & Richter, 2021, page 14), and Gaboury sees a similarity between Strachey’s approach to technology and Turing’s recognition scene from The Imitation Game (Gaboury, 2013). For Turing and Strachey, technology is no longer the excluded other per se, but a possible object of affective and linguistic-social recognition. Jacob Gaboury also wants to understand Strachey's work in the context of queer computing as an ironization of conventional, heterosexual emotional language (Gaboury, 2013, 2022). Bernhart and Richter similarly emphasize the gap between convention and emotion in Strachey’s work:

In two respects, Strachey’s Love Letter Generator represents a historic shift in the coding of emotions that continues to have an impact on Affective Computing and Emotion AI: First, it highlights the major differences between simple and repetitive ways of generating expressive—in this case written—documents and their quite remarkable concrete form. Second, Strachey’s program demonstrates that technology is socially negotiated as a kind of performance (Gaboury, 2022). Strachey’s Love Letter Generator serves also to illustrate his assessment of the early possibilities of computers as intelligent actors.

Strachey clearly states that computers, though, do not “think”—and it can be added here that they do not “feel” either, in order to emphasize the importance of programming (Strachey, 1954, page 25). The statement that computers do not think and feel applies to all programs that seek to give the appearance of machine thinking—and, it should be added, perceiving or feeling—i.e., all computer programs and systems that we have since come to refer to as Artificial Intelligence. Nevertheless, their “tricks” would lead to unexpected and interesting results: “This is true of all programs which make the computer appear to think; (...) However, sometimes these tricks can lead to quite unexpected and interesting results.” (Strachey, 1954, page 27). The same applies to emotions or emotional intelligence, which are here imitated and recreated in a purely mechanical and performative manner. Compared to later forms of Affective Computing and Emotion AI, the Love Letter Generator does not analyze any user data and assign it to emotions. Rather, the written expression of infatuation and deep feeling suggests an emotional state on the part of the machine. Nevertheless, the Love Letter Generator deserves consideration within the genealogy of Emotion AI, precisely because it exhibits and utilizes the mutual attribution of various emotional states in the process of writing-based encoding of affects and emotions.

This does not imply a lack of humanity, but rather that technical evidence reflects something about human evidence that is often overlooked and ignored. Human declarations of love, especially in written form, may be unique and unforgettable expressions of the sender’s and recipient’s experiences and feelings. For third parties, however, they are often anything but free of patterns and schematic repetitions. Human expressions of love, as Strachey exposes with his Love Letter Generator, are as irreplaceable as they are interchangeable—and this is not a bad thing per se. Language has always been our technical Cyrano.

The Love Letter Generator is not just one example among many. Rather, it embodies the shift in emotion coding from analog writing to computer coding. Even if the Love Letter Generator cannot be technically compared with today’s AI, it is still one of the first technical systems that could synthesize emotional written testimonies. It is therefore permissible in the history of modern computing in general, and in the history of AI with a focus on emotion simulation and recognition in particular, to draw a historical arc from Strachey’s Love Letter Generator to ANN-based Affective Computing and Emotion AI.

On the one hand, in order to understand it, the history of emotion coding can be traced from gestural expression and verbal explanation to minnesong, poetry, and literature, and finally to computer-based emotion recognition. On the other hand, the last step in particular represents a turning point, as there is a significant shift from self-coding to external coding.

This contribution develops a perspective on Affective Computing as virtual semantics in a media-archeological and media-semiotic sense, focusing on the basic operation of a framework—a semantics—of encoded “emotions” in their socio-technological embeddings. Affective Computing and Emotion AI can be defined as a semantics because the basic syntactic principles of sensor-based and emotion-related data processing have—or can assume—strong semantic functions. These forms of computing are described here as virtual semantics for two reasons: First, the adjective “virtual” discursively marks the networked computer environment and lifeworlds in which these semantic effects emerge. Second, and of greater theoretical significance, it underscores a specific similarity and simultaneous difference from natural-language semantics. The virtual semantics of affective computing and Emotion AI is an “as-if” semantics. Charles Sanders Peirce expresses this in his concept of virtuality or even “enshrined” it with the following equation (Chalmers, 2023, page 187): “A virtual X (where X is a common noun) is something, not an X, which has the efficiency (virtus) of an X” (Peirce, 1920, page 763). The term “virtual” in virtual semantics also is part of the Deleuzian tradition of a virtual entity as something not yet realized but nevertheless real (Münker, 2005). To summarize and clarify this point: The term “virtual” in virtual semantics refers both to the digital environment of the forms and systems of encoding under consideration and to the fundamental difference that still exists from other linguistic semantics, precisely because the practices of encoding mark a significant shift from the written communication of emotional experience to sensor-based computing.

In the course of the 20th century, the term code changed its meaning with the development of modern computers, so that the word code eventually came to epitomize the practice of programming machines. When converting analog phenomena such as sounds, images and language into digital data, the process of a certain—but not necessarily binary—“numerical coding” must take place (Heilmann, 2010, page 128). The world’s libraries are no longer just filled with handwritten and printed artifacts, but also with computer code (https://codelibrary.opendatasoft.com/). The very data that makes up Affective Computing and is supposed to relate to emotionality must be coded and discretized in a certain way in order to make it machine-processable in the first place. As the media-semiotic framing makes clear, every form of coding involves a shaping process that cannot be circumvented. In the case of Affective Computing and AI systems, however, decisions about specific coding methods are made by third parties, consisting of scientists, programmers, nation states and corporate organizations (See for the highly problematic case of emotion recognition by nation states: Sanchez-Monedero & Dencik (2022)). These various third parties alone or combined can but do not necessarily have problematic or outright bad intentions, but as per today they don’t reflect the normative consequences of their technological pragmatism—but who determine what will later be defined as emotion in the context of technical recognition applied in their systems and embedded in social contexts. In concrete terms, this means, for example, that the services and systems offered under the name Emotion Recognition, Affective Computing, Empathic Technologies and others systematically implement which emotions are processed and how. Until now, the number of emotions was often fixed, usually remaining in the single digits and focusing on, for example, four states (clusters of data) defined as joy, anger, sadness and surprise (see for example the trial with an emotion recognition software in Berlin: Projekt zur Gesichtserkennung erfolgreich (2018)).

Specifically, this will now be examined step by step in terms of the data flow involved in emotion processing within potential applications of Affective Computing and Emotion AI (Fig. 6.). The steps here refer to the various components and stages of data collection and processing, which in some cases are mandatory and in others optional as part of the technical processes of Affective Computing and Emotion AI discussed here (see for recent overviews from the field: Afzal et al. (2024); Ahmadpour et al. (2025)). To this end, I propose an analysis scheme that embeds the data flow and device use in a general system in order to show the coding and recoding processes in each case: 1. Data sources: stimuli and environment, 2. Data collection: sensory conditions, 3. Data accumulation and modulation: sensor combination and time frames, 4. Data patterns: detection emotions, affects, sentiments and the like. I will now go through the steps in the system, characterize them, and examine the coding and recoding processes in each case in order to prepare the definition of Affective Computing and Emotion AI as virtual semantics. The steps do not necessarily have to be taken in every case of Affective Computing and Emotion AI, nor do they have to be taken in the same form or in the order specified. However, they provide a good overview of the schematics of Affective Computing and Emotion AI, which helps to substantiate the arguments presented.

The advances in the field of artificial intelligence (AI) over the past decade have been marked symbolically as a new insult to humanity, or “insult 4.0,” following the cosmological, biological, and psychological insults caused by the discoveries of the Copernican revolution in astronomy, evolution theory and psychoanalysis (Dotzler, 2024, page 59). All three discoveries before AI are said to have destroyed a form of human self-illusion of being the center of the universe (by Copernicus), a unique creation (by Charles Darwin) and the sole master of mental faculties (by Sigmund Freud).

Now, in the age of strong artificial intelligence, intelligent thinking and creative work no longer seem to be distinguishing features of the human species, especially in view of the impressive achievements of generative pretrained transformers in text production as well as speech and image synthesis. Individuality and “emotionality” appear in positions today as the last bastions for distinguishing humans from artificial beings, which Leif Weatherby critically frames as remainder humanism (2025).

Against this background, Affective Computing and Emotion AI have been assessed from two opposite perspectives as a shifting of these boundaries: firstly from a humanist perspective and secondly from a technicist perspective. From a humanist perspective, Affective Computing and Emotion AI can be understood in this context as a further transgression of the boundary between man and machine. From a technicist or techno-optimist perspective, on the other hand, Affective Computing and Emotion AI can be understood as a sufficient and often helpful imitation of human characteristics by digital technology, that bridges the distance between technological systems and lived experiences. A third perspective presented here can be described as analytical in general and media-semiotic and media-archaeological in detail. A media-semiotic and media-archeological perspective understands and problematizes Affective Computing and Emotion AI in their genesis and socio-technological setting. The goal is to show Affective Computing and Emotion AI as a powerful new form to encode emotions that takes on the function of virtual semantics in the contemporary digital culture. It emphasizes the still large differences between human individuals and their machines, but shows how Affective Computing and Emotion AI interweave computers and their users in a complex automatic or semi-automatic, not seldom highly problematic way that can be understood as a normative set of rules as virtual semantics. The paper makes the following points, which do not exclude other perspectives of a phenomenological and media-aesthetic nature, but rather seek to complement them:

Step 1I have applied a broad concept of code, following Umberto Eco, to emotions and affects. As soon as emotions and affects are shared, conveyed, expressed, written down, and, in short, communicated, they must be encoded. In principle, this can happen in very different ways, e.g., in gestural, spoken, or written form.

Step 2I have examined various historical techniques for encoding emotions, such as handwritten and computer-generated love letters, and described the shift toward Affective Computing and Emotion AI as a transition from self-encoding to external encoding.

Step 3Affective Computing and Emotion AI are described as virtual semantics based on the broad concept of code, as presented in step 1. “Virtual” refers both to the digital environment and to the quasi- or “as if” nature of the semantics described, since it comprises a technical-syntactic framework that exerts a strong semantic effect. The conceptual and normative issues identified concern, among others, the automation of emotional and affective communication.