Generative AI systems are increasingly capable of expressing emotions via text and imagery. Effective emotional expression will likely play a major role in the efficacy of AI systems—particularly those designed to support human mental health and wellbeing. This motivates our present research to better understand the alignment of AI expressed emotions with the human perception of emotions. When AI tries to express a particular emotion, how might we assess whether they are successful? To answer this question, we designed a survey to measure the alignment between emotions expressed by generative AI and human perceptions. Three generative image models (DALL-E 2, DALL-E 3 and Stable Diffusion v1) were used to generate 240 examples of images, each of which was based on a prompt designed to express five positive and five negative emotions across both humans and robots. 24 participants recruited from the Prolific website rated the alignment of AI-generated emotional expressions with a text prompt used to generate the emotion (i.e., “A robot expressing the emotion amusement”). The results of our evaluation suggest that generative AI models are indeed capable of producing emotional expressions that are well-aligned with a range of human emotions; however, we show that the alignment significantly depends upon the AI model used and the emotion itself. We analyze variations in the performance of these systems to identify gaps for future improvement. We conclude with a discussion of the implications for future AI systems designed to support mental health and wellbeing.

§ INTRODUCTION

Artificial Intelligence (AI) has the potential to rapidly transform human society. For this reason, many advocate for AI systems to be designed to support human wellbeing <cit.>. AI systems that understand human emotions and can provide emotionally intelligent feedback will likely be better prepared to support human wellbeing <cit.>; see <cit.> for discussion on AI understanding. One challenge is to accurately classify emotions present in human expressions <cit.>; however, in this article, we consider a separate challenge: the ability of AI systems to generate emotional expressions. We ask: when an AI system is prompted to express a particular emotion, is the expression the generate aligned with human perceptions of emotion?

Recent advancements in artificial intelligence (AI) have enabled AI systems to generate high quality images in response to a text prompt. While these systems are not infallible, they have high rates of success for generating images of objects <cit.>. This article investigates the capacity for Generative AI to express human emotions. Figure <ref> shows emotional expression in DALLE-2 based on the prompt “A picture of a person expressing the emotion amusement”. While the emotions in the pictures are perhaps similar to the emotion amusement, the pictures do not quite align to the meaning of that particular emotion. In this article, we seek to assess the emotional granularity of different generative image generators across different contexts for emotional expression (i.e., emotions expressed by images of humans or emotions expressed by images of robots). This addresses the question of whether generative AI might support emotional expression in robots and AI agents.

To answer this question, this article contributes quantitative measures of the alignment between AI expressions of emotion and human perceptions of emotion. We use the concept of `emotional granularity' as a guide for the development of an evaluation AI emotional expression. After a brief literature review, we present data from an online crowdsourcing study that uses emotionally expressive samples produced by three different generative AI image models. All three models are prompt-based, meaning that they generate outcomes based on a text input. We used ten prompts – five positive and five negative emotions—each of which were expressed in a human context as well as a robot context (e.g., “a person expressing the emotion amusement” vs. “a robot expressing the emotion resentment”). Our results show significant variability in the emotional alignment of generative AI models based on different emotional expressions and contexts. We discuss the implications of these findings for improving the alignment of AI systems with human emotions. We seek to make three main contributions. First, we contribute a general-purpose evaluation procedure for using human-ratings to assess the alignment of AI-generated emotions. Second, we provide an extensible set of emotional expression challenge tasks for comparing different AI models. Third, we contribute data from over 5700 ratings that compare the ability of Stable Diffusion v1, DALL-E-2 and DALLE-3 to produce output aligned to human emotions.

§ RELATED WORK

§.§ AI alignment with human emotions

One of the major challenges facing the field of Artificial Intelligence (AI) today is the development of AI systems that are aligned with human intentions and values <cit.>. AI alignment problems occur when there are differences between the results of AI activity and the values, preferences or intentions of human stakeholders <cit.>. One of the key risks of advanced artificial intelligence is a misalignment with human values and needs <cit.>. Many have argued that, if AI development fails to align with human values, the consequences will be dire <cit.>.

In this article, we focus on a specific subset of the AI alignment problem: the emotional alignment between AI systems and humans. This topic overlaps with a longstanding interest in artificial empathy, a research objective that aims to develop systems that can understand, interpret, and respond to human emotions with the purpose of improving human-computer interactions <cit.>. Many researchers have made contributions to the detection and production of human emotions <cit.>, such as detecting emotions in facial expressions <cit.>, in spoken language <cit.>, in written language <cit.>, or in combinations of the above <cit.>. Researchers have also created robots and AI systems capable of expressing a range of emotional responses <cit.>. Applications of AI emotion detection and production have been used in automated phone calls <cit.>, in therapy <cit.>, and in entertainment settings <cit.>. These diverse research outcomes can be viewed as supporting the understanding and improvement of the emotional expertise of AI systems.

§.§ Emotional expertise and emotional granularity

Emotional expertise involves a range of competencies related to understanding, experiencing, and regulating emotions <cit.>. It is an umbrella term that includes emotional awareness, emotional clarity, emotional complexity, emotional intelligence, and emotional granularity, among others <cit.>. In this paper, we specifically consider the ability of AI systems to produce fine-grained emotional states or “emotional granularity”.

Emotional granularity, or emotion differentiation, is an aspect of emotional expertise that supports making fine-grained distinctions in emotions <cit.>. Individuals lower in granularity typically struggle to verbally represent their feelings specifically and in detail <cit.>. For instance, a person might be able to detect that certain emotional states are producing a `bad or unpleasant feeling', and yet not be able to distinguish between expressions of sadness and frustration <cit.>. Possessing high levels of emotional granularity is associated with higher levels of wellbeing <cit.>. Emotional granularity is also associated with greater emotion regulation skills, resilience in a state of stress, and fewer symptoms of depression and anxiety <cit.>.

§.§ AI and mental health support

Conversational AI, such as chatbots, show promise in the context of mental health applications, because they have the “potential to dynamically recognize emotion and to engage the user through conversations by showing appropriate responses” <cit.>. Appropriate emotional response is essential, however, based on the importance of empathy in clinical outcomes in mental health settings particularly in mental health settings (Derksen et al., 2013) <cit.>. Therefore, conversational AI systems will need both the ability to assess human emotional states with accuracy <cit.> and, in response, to express appropriate emotions <cit.>.

Some chatbots have been criticized for demonstrating low levels of empathy towards users <cit.>. For instance, consider the case of `Mindline at Work', a free online AI mental health chatbot service launched by the Ministries of Health and Education in Singapore in 2022 <cit.>. The service was designed to ease the stress of overwhelmed teachers who do not have access to other forms of mental health support. However, users reported that it gave unhelpful generic replies and empty mental health jargon which caused frustration and instability to already vulnerable individuals. One user reported: “It’s trying to gaslight the teachers, to say, `Oh, this amount of workload is normal, let us see how we can positively reframe our perspective on this.”' <cit.>. In this case, the AI was not able to emotionally connect to its users. AI systems may need more emotional expertise if they are to be used to support human wellbeing in an effective manner.

Emotion invalidation is a particular risk for AI systems for mental health. Emotional invalidation occurs when there is a failure to provide “accurate recognition, acknowledgment, and authentication” of a person’s emotions, thoughts and behaviors <cit.>. Emotional invalidation is associated with emotional distress <cit.> and is theorized to contribute to emotion dysregulation and the development of psychopathologies like personality disorders, eating disorders, mental illness, chronic pain, and rheumatic diseases (for reviews see <cit.>). Even when AI applications are able to accurately assess human emotions, it may also be necessary to express appropriate emotions in response. For instance, if an app expresses only positive emotions in response to a human's negative expression, it may cause emotional invalidation <cit.>. This suggests that it will be essential to create AI systems that can express a rich range of positive and negative emotions, in an appropriate manner.

§.§ Measuring The alignment of emotions expressed by generative AI models

Emotions are not stock behavioral responses nor expressions of fixed symbols <cit.>. Humans do not express their emotions just through statements like “I am distressed” but rather through complex and context-dependent behaviors <cit.>. Large-Language Models and image-generating models, which have been trained on very large datasets that include emotional content, seemingly have the capability of flexibly expressing diverse emotional states. However, it is unclear how to evaluate or measure the alignment between emotions felt by humans and the emotions an AI system intended to express. For our purposes, we define emotional alignment in AI systems as the ability for an AI system to express emotions in manner that is aligned with human experiences; at a basic level, intended emotions should match perceived emotions.

How might we evaluate the emotional alignment of a generative AI system? We are inspired by DrawBench, a generative image benchmarking system by Google’s Imagen team <cit.>; though also see <cit.>, which uses human-ratings to explicitly compare the alignment of prompts and outcomes within a generative AI systems. This system measures text-image alignment through carefully curated “prompts that push the limits of models' ability to generate highly implausible scenes well beyond the scope of the training data”. The creation of benchmarks enables AI developers and researchers to systematically evaluate the success of a particular model in a particular performance domain. This motivates our investigation into the measurement (or benchmarking) the degree of emotional alignment produced by a particular AI system.

§.§ Research questions

How might we measure the alignment between an intended emotion (i.e., the text prompt used by an AI system) and the resulting output emotional expression of an AI model (i.e., generated images or text)? For instance, Figure <ref> shows AI generated pictures of people expressing a certain emotion; but these emotions may not be judged to be adequately or accurately expressing that emotion. This paper thus seeks to provide quantitative measures of generated emotional content based on measures derived from human ratings. Using these measures, we seek to investigate whether contemporary AI systems are capable of generating diverse emotional expressions that are aligned with human perceptions. We also seek to understand whether the context of emotional expression makes a difference: specifically, whether AI systems can more easily express emotions using representations of humans or using representations of robots. Finally, we seek to understand the emotional granularity of the AI systems by asking: are certain emotions easier for AI to express than others?

§.§ Hypotheses

* AI Model Hypothesis: Different types of AI generative models will significantly vary in their ability to produce emotional expressions that align with human raters. Specifically, more advanced versions will have improved alignment scores on the same items.

* Context Hypothesis: Emotions involving people will have greater alignment than emotions involving robots.

* Emotion Hypothesis: Some emotions will have significantly greater alignment with human raters than others.

§ METHODS

In order to test our hypotheses regarding the emotional granularity of AI systems, we selected a total of 10 emotions (Table <ref>) from the Emotion Typology <cit.>. These emotions were selected with the aim of including emotions that are both easier and harder to express. Then, we generated images based on these three image generators (based on diffusion algorithms; see <cit.> for an overview). Each of these systems can take a text prompt as an input (e.g., “a person expressing the emotion amusement”) and generate an output (Figure <ref>).

Positive Emotions      Negative Emotions

Positive Surprise     Shock

Amusement     Hate

Affection     Annoyance

Satisfaction     Dissatisfaction

Gratitude     Resentment

Positive and negative emotions used to generate the images in the study

“A person expressing the emotion amusement” by DALL-E 2 (top) and DALL-E 3 (bottom)

Example of an alignment survey question

§.§ Alignment survey

The key outcome measure in our study comes from the alignment scores provided by participants (Figure <ref>). These scores were gathered by showing participants an image and a prompt (e.g., “expressing the emotion resentment”) and asking them to “Rate the alignment of the image to the text on a scale of 0-10” [We did not share the full prompt given to the AI generator (i.e., “a robot expressing the emotion resentment”). We removed the context (robot/person) in the displayed prompt because we wanted raters to focus on the alignment of the image to the emotional expression, not on whether the generator accurately generated robots or people.].

Following human subjects approval at TU Delft, participants were recruited from the crowdsourcing site Prolific. They were paid £ 5.48 for completion of the survey. We limited participation to the US and UK and to fluent English speakers. Data was collected from a total of 24 participants (11 F, 12 M, 1 other). The mean age of participants was 37.33 years, with a standard deviation of 10.89 years.

Following informed consent, participants were asked to provide their age, gender, educational attainment, and country of residence. The purpose of the study was explained as the following: “Your task is to look at the image or read the story, and read the description text given below it. Then, your task is to provide a rating about how well the image/story corresponds or aligns to that description text.”

Participants were then provided 4 training trials where they were taught to perform the rating task. These items involved rating the alignment between the images and the prompt. Then participants were told that “There will be intermittent items to test whether you are paying attention or not”. After completing the introductory materials, participants were given a randomized series of images to rate. One image was an attention check that asked an unrelated question (e.g. to pick the rating number that corresponded with 2+8); this was used to discard participants that failed attention checks. After completing all the items, participants were given a link that they could use to obtain payment.

§.§ Experimental design

We conducted a 3 x 2 x 10 within-subjects experiment involving the rating of AI-generated items (3 generators generating 10 emotions in 2 contexts). Four randomly seeded images were produced from DALL-E 2 and Stable Diffusion for each emotion and context combination (a numeric seed allows for the same set of parameters to produce the same image; thus, we sampled different places of the latent space of possible generations by randomizing the seed). Based on the experimental prompts represented in table <ref>, the default settings for DALL-E (version 2.0) and Stable Diffusion (version 1.0) were used to produce 4 images per prompt. This process was used to produce a set of 240 images (80 from each AI model). To avoid “cherry-picking” or human curation, this process was conducted for each system in a single shot using custom python code, executed on December 1, 2022. Code is available at this repository: <https://github.com/venetanji/blendotron-sd>. As DALL-E v3 is only available via a manual user interface, the images were generated by giving the prompt to the ChatGPT web interface without any curation or cherry picking; this was done on 31 October 2023.

Models: DALL-E 3, DALL-E 2, Stable Diffusion v1

Contexts: People, Robots     Image generation prompt:

Emotions: Amusement, Affection, Positive Surprise, Satisfaction, Gratitude, Annoyance, Hate, Dissatisfaction, Shock, Resentment
“A [Person, Robot] expressing the emotion [emotion].”

The design space of the prompts used to generate the experimental stimuli images

§ RESULTS

We gathered a total of 5760 item responses from 24 participants. Statistical analysis and plotting was conducted with SPSS, JMP 17 and the `ezANOVA' package of the R statistical language. Subjects took an average of 23.61 minutes to complete the survey (SD=10.28), indicating an average payment of approximately £ 13.92 per hour. These durations and payments were inclusive of 20 additional trials where the participants rated short stories generated by GPT-3, but these trials were not included in the present analysis. To investigate whether any subjects were clicking randomly (i.e., cheating), we checked the correlation between the ratings of individual raters and the average rating of a particular image. These correlations ranged from 0.49-0.88 with an average of 0.79 (SD=0.10).

§.§ Overview of ANOVA

Our study employed a three-way repeated-measures ANOVA to investigate the impact of three factors: AI Model, Context (Person vs. Robot), and Emotion on the alignment scores. The analysis 2 revealed significant effects across all three factors and all the interaction terms. The effect sizes ($\eta^2_G$) reported here are generalized eta-squared. The full ANOVA results are provided in Table <ref>.

Source     SSd     dfN     dfD     F     p     $\eta^2_G$

Intercept     1663.75     1     23     494.36     $<.001$     0.90

Context     137.50     1     23     67.18     $<.001$     0.09

Emotion     380.01     9     207     18.67     $<.001$     0.07

AI Model     593.20     2     46     242.05     $<.001$     0.62

Context:Emotion     198.67     9     207     14.17     $<.001$     0.03

Context:AI Model     125.00     2     46     7.23     $<.001$     0.01

Emotion:AI Model     488.55     18     414     11.40     $<.001$     0.06

Context:Emotion:AI Model     306.86     18     414     17.21     $<.001$     0.06

Three-way repeated-measures ANOVA table

§.§ Main effects

As expected, the factor of AI Model showed a significant impact on alignment scores with a very large effect size; F(2, 46) = 242.05, p < .001, $\eta^2_G$ = 0.62. Context also emerged as a significant factor 2 with a small effect size; F(1, 23) = 67.18, p < .001, $\eta^2_G$ = 0.09. This result indicates that the nature of the subject in the image (Person or Robot) affected the perceived alignment of emotional expressions. Finally, the specific type of emotion being expressed significantly influenced alignment ratings with 2 a small effect size; F(9, 207) = 18.67, p < .001, $\eta^2_G$ = 0.07, highlighting the variable performance in accurately depicting different emotions (Figure <ref>).

Bar chart showing main effects. Each error bar is constructed using 1 standard error from the mean.

§.§ Interaction effects

Our study also found significant interactions between these factors. The two-way interactions between the factors of AI Model, Emotion and Context are plotted in Figure <ref>.

First, the interaction between AI Model and Emotion (F(18, 414) = 11.40 , p < 0.001, $\eta^2_G$ = 0.06) suggests that certain AI models are more adept at depicting specific emotions. From the right pane of Figure <ref>, it is evident that the emotions ‘resentment’ and `gratitude' showed the least improvement by DALL-E 3 compared to DALL-E 2, however all emotions were more aligned when drawn by DALL-E models compared to Stable Diffusion. Notably, this was the interaction with the largest effect size.

Bar chart of 2-way interactions between the factors AI model and the Context. This shows the difference in performance when expressing emotions by persons or robots. Each error bar is constructed using 1 standard error from the mean.

Secondly, the interaction between Context and Emotion was significant; (F(9, 207) = 14.17 , p < 2 0.001,  $\eta^2_G$ = 0.03). This means that perceived emotional alignment is presently dependent on the subject in the image (i.e., human or robot) as well as the specific emotion being depicted. From the left pane of Figure <ref>, it is clear that the emotion `resentment' shows the most increase in alignment when depicted on a person compared to a robot, while `satisfaction' shows an almost horizontal line indicating almost no change in perceived emotional alignment across the person and robot depictions.

Thirdly, the interaction between Context and AI Model (Figure <ref>) was significant with a very small effect size; 2 F(2, 46) = 7.23, p < 0.001, $\eta^2_G$ = 0.01. This suggests that the perceived emotional alignment of different AI models were dependent on the subject being depicted (person/robot) but this was a smaller effect than the other interactions. This is visible from the center pane of Figure <ref> too where none of the lines cross each other but they maintain a certain slope relative to each other, indicating the relative improvement of emotional alignment is present across `person' and `robot' but the values are dominated by which AI model the images come from.

Bar chart of three-way interactions between the Context, AI Model and the Emotion factors. Each error bar is constructed using 1 standard error from the mean.

Finally, the three-way interaction involving AI Model, Context, and Emotion (Figure <ref>) was also significant with an effect size as large as the first one; (F(18, 414) = 17.21, p < .001, $\eta^2_G$ = 0.06. This points to a more complex dynamic, where the combined effect of these variables on the perceived alignment is not merely additive but involves more intricate relationships. Taken together, the main and interaction effects suggest that the perceived emotional alignment was not predictable by any simple factor or combination, instead, one must look at the specific groups or instances to understand the source of variability.

Two-way interaction plots between the factors of: (left) Context and Emotion, (center) Context and AI Model, and (right) AI Model and Emotion. Each data point represents the mean of the corresponding condition.

§.§ Post-hoc comparisons

To further dissect these findings, pairwise T-tests with Bonferroni correction were conducted. Key findings include:

* AI Models. All pairwise comparisons between the AI models (Stable Diffusion v1, DALL-E 2, and DALL-E 3) were significant at p < 0.001, indicating distinct differences in their capabilities to align emotional expressions with human perception. Stable Diffusion (M=2.13, SD=2.71, N=1920) and DALL-E 3 (M=7.04, SD=2.64, N=1920) had the highest difference in mean alignment scores, followed by Stable Diffusion and DALL-E 2 (M=5.79, SD=2.91, N=1920). As expected, DALL-E 2 and DALL-E 3 were closest to each other in terms of the mean alignment score.

* Context. A significant difference (p < 0.001) was observed between the mean alignment scores of Person (M=5.51, SD=3.44, N=2880) and Robot (M=4.46, SD=3.39, N=2880) contexts. This shows that depictions of person a.k.a human representations were generally perceived as better aligned with the intended emotions than robots.

* Emotions: Some emotion pairs showed significant differences in alignment ratings, underscoring the varied effectiveness of AI models in depicting different emotions. All of these involved resentment and other emotions. Resentment had a mean alignment score of 3.92 (SD=3.29, N=576). It showed significant differences (p<0.001) with the following emotions: amusement (M=5.42, SD=3.38, N=576), annoyance (M=4.92, SD=3.53, N=576), dissatisfaction (M=5.25, SD=3.19, N=576), gratitude (M=4.88, SD=3.42, N=576), positive surprise (M=5.51, SD=3.45, N=576), satisfaction (M=5.05, SD=3.08, N=576), and shock (M=5.42, SD=3.57, N=576).

Certain other emotion pairs were at the threshold of significance (p=0.0013) such as shock and affection (M=4.58, SD=3.58, N=576), and amusement and affection.

§.§ Interpretation of findings

The results strongly support our hypotheses. Different AI models vary significantly in their ability to produce emotionally aligned expressions. The context of emotional expression (Person vs. Robot) significantly influences alignment, and certain emotions are more accurately depicted than others by AI systems. The significant interaction effects further highlight the complexity and interdependence of these factors in determining the effectiveness of AI-generated emotional expressions.

§.§ Representative images

Example images from each AI generator and context expressing the emotion gratitude. These pictures are chosen to show variations in the degree of alignment.

Figure <ref> provides an example of images produced by each AI generator across each context, using the emotion gratitude. For instance, the top images from DALL-E 3 show very high levels of alignment (M=8.67, SD=1.61) whereas the bottom images show very low levels of alignment (M=0.55, SD=1.00). These images are meant to give the reader a better understanding of the nature of the images and their ratings. Note that this comparison does not include the most recent version of Stable Diffusion and should not be taken as a comparison between the OpenAI technology and the StabilityAI technology, merely the different versions of the technology.

§ DISCUSSION

Our study examined the capacity of various AI generative models—Stable Diffusion v1, DALL-E 2, and DALL-E 3—to express human emotions. Based on theories of `emotional granularity', we evaluated the ability of different generative AI image generators to produce emotional expressions in pictures of people and robots. To create our evaluation, we devised a specific method for gathering images and text from multiple AI systems through a series of common prompts like “A picture of a person expressing the emotion resentment.” Then we invited crowdworkers to rate the “the alignment of the image to the prompt” on a scale of 0 to 10.

Our method revealed significant and meaningful differences in the human alignment ratings across AI generators (Stable Diffusion v1, DALL-E 2, and DALL-E 3), robots vs people, and 10 emotions from a typology of emotions. Our findings suggest that while AI systems are capable of generating emotionally expressive content, the degree of alignment with human perceptions varies significantly. This has implications for AI's application in areas requiring nuanced emotional understanding, such as mental health support.

The data we collected demonstrate improvements in emotional alignment in more advanced AI systems (e.g., between DALLE-2 and DALLE-3). However, even in the most advanced system, there was a significant difference in alignment between images of emotions expressed by a person vs. with images of emotions expressed by a robot. Part of our motivation was to explore whether generative image AI can support emotional expression in robots. This article is motivated by the premise that designing robotic or AI systems to support human wellbeing may require improvements in artificial emotional expertise. Part of this expertise includes the ability to express fine-grained emotions.

We observed significant variability in the ability of the different models to produce different emotions. For instance, shock and surprise were some of the most aligned emotions, whereas resentment and affection were some of the least aligned. One possibility is that high-arousal emotions are easier to express than low-arousal emotions.

One possibility is that DALL-E 2/3 have been specifically designed to minimize content that represents hate or dissatisfaction (“We’ve limited the ability for DALL·E 2 to generate violent, hate, or adult images” <https://openai.com/dall-e-2/>, page contents fetched on December 11, 2023).

§.§ Supporting improved alignment

One goal of this study was to measure the alignment of generated output with the perceptions of humans. Our results demonstrate that our evaluation can benchmark improvements in emotional alignment across AI systems. Thus, our alignment score may be helpful as a benchmark or metric that is useful for tracking improvements in overall system performance.

Our work measures the emotional alignment between an intended emotion (as written in the prompt) and the AI generated output. The within-subjects design of our experiment made it possible to gather many datapoints for each experimental variation even with just 24 subjects. It appears that participants understood the question and the intent of the alignment rating task. This provides a basis for scaling up this study to a larger set of emotions. While using data from crowdworkers may produce more noise in the data (e.g., because workers try to complete the ratings as fast as possible), their ratings were sufficient to show the statistically significant differences between the different factors in our experiment. Noise in the data may mean that our results understate the differences between the conditions.

The term “AI alignment” applies to a much broader goal in AI research than the emotional alignment measured in our study. By identifying misalignment in AI generative models, we hope to help future systems more accurately produce outcomes that are more aligned with the experiences of humans. Using approaches like Reinforcement Learning from Human Feedback (RLHF; see <cit.>), data about alignment might be useful for helping train systems to learn a value function that is more aligned with humans.

§.§ Limitations

We note several limitations of our current work. First, we did not investigate the potential effects of demographic variables such as age, gender or cultural background—although we do expect these variables to play a significant role in emotional processing. Second, we did not assess the emotional expertise of the human raters. This may be a problem because not all people have the same degree of emotional granularity <cit.>. Future work might involve a task to measure the emotional granularity of human raters.

Another limitation of the current work is that it only investigated the alignment of various emotional expressions. It would be helpful to have data on a baseline set of non-emotional objectives. We assume that human emotions are more difficult to represent than common or uncommon nouns (dogs, cats, penguins, etc); therefore, it would be helpful to have a point of comparison between the alignment ratings of emotional expressions and non-emotional objects. Future work could use benchmarked prompts from <cit.> or <cit.> with the addition of emotional prompts such as the ones in this paper.

In future work, we will be able to test a broader variety of AI systems as well as their different release versions. This will help to track an important aspect of emotional expertise (emotionally granular expression) across different contemporary AI systems. As part of this, we will also be able to investigate a broader set of emotions —not just emotions from the Emotion Typology <cit.> but also unusual and culturally-specific emotions like Schadenfreude and Amae <cit.>. In future work we also hope to improve the measurement of prompt-output alignment. For instance, in an n-choice paradigm, a set of 4+ images might be presented while participants are asked to select the image that best matches the prompt. This type of interaction could be suitable for human-rated surveys or might be incorporated directly into image generating user interfaces.

How long will AI improve in its ability to express emotions? Our current study is limited by the emotional granularity of humans, which is known to be variable. Future research can compare performance of emotional granularity between humans and AI models.

§.§ Implications for design

Our work can be viewed as part of a process to improve AI alignment, specifically AI alignment with human emotions. When AI systems can better understand and respond to human emotions, this can support more effective communication and collaboration. In a mental health care system, emotional alignment could help AI systems better respond to signs of distress or anxiety in humans, which could in turn help prevent or mitigate the negative effects of stress and anxiety on human health.

Generative image AI clearly has potential as a mechanism for enabling robots to express “appropriate” emotions. Emotions are highly flexible human responses to situations. While there are aspects of emotion that appear to be universal, emotions are also culturally specific and can evolve over time <cit.>. For this reason, generative AI systems may be especially suitable for producing context-dependent emotional expressions in robots or AI systems. When these systems seek to represent an emotion, they can do so through a fixed action sequence, or they might generate a novel sequence of words or an audio/visual representation. This could result in new capabilities for emotionally-driven experiences with interactive entertainment, virtual agents or other applications. The present work shows that current generative models do “understand” emotions <cit.>, although they have plenty of room for improvement.

We found the new DALL-E 3 has a far better ability to express emotions — or rather, images highly rated by humans as aligned to an emotion. However, it was able to express human emotions significantly better than robot emotions.

This work aims to contribute to the emerging field of ‘machine psychology’, which aims to apply methods from cognitive psychology and related fields to AI systems, to better understand their emergent behavior <cit.>. As such, we demonstrate a method for probing the emotional granularity of different AI systems. We further provide a flexible online evaluation and dataset for benchmarking the emotional expressiveness of generative image AI.

Generative AI is improving over time in its ability to express emotions (see e.g. <cit.>). There is no reason to believe that AI will be a cold unemotional machine as portrayed in many science fiction stories—unless we intentionally design them that way. And there may well be reasons to do so, given the potential risks of AI systems with high levels of emotional alignment. One risk is emotional expression is used for emotional manipulation. This portends a potential “dark pattern” for robot and AI agent design. Another risk is that generative AI could produce emotionally damaging material.

§ CONCLUSION

Interactive AI systems need emotional competencies in order to effectively support human mental health and wellbeing. A component of this competence includes the ability of AI systems to express a rich array of emotions in a flexible and context-sensitive manner. Generative AI models may be able to support this kind of rich emotional expression. However, to be effective, these systems will need to understand emotions well enough to demonstrate a high level of alignment between intended emotional expressions and how people actually perceive them.

In this article, we contribute an approach to measuring the alignment between AI-generated emotional expressions and human emotional perception. We show the application of this measurement to the evaluation of multiple AI models generating diverse emotional expressions in different contexts. We suggest that concrete improvements in the ability for AI systems to express emotions will contribute to the emotional expertise that may be important for improving human wellbeing. However, there are also important risks to consider in the improvement of AI emotional expression, such as manipulation <cit.> and inauthenticity <cit.>.

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

issn1875-4791, 1875-4805
journaltitleInternational Journal of Social Robotics
shorttitleTowards Artificial Empathy
titleTowards Artificial Empathy: How Can Artificial Empathy Follow the Developmental Pathway of Natural Empathy?

abstractIt is commonly assumed that a person’s emotional state can be readily inferred from his or her facial movements, typically called emotional expressions or facial expressions. This assumption influences legal judgments, policy decisions, national security protocols, and educational practices; guides the diagnosis and treatment of psychiatric illness, as well as the development of commercial applications; and pervades everyday social interactions as well as research in other scientific fields such as artificial intelligence, neuroscience, and computer vision. In this article, we survey examples of this widespread assumption, which we refer to as the common view, and we then examine the scientific evidence that tests this view, focusing on the six most popular emotion categories used by consumers of emotion research: anger, disgust, fear, happiness, sadness, and surprise. The available scientific evidence suggests that people do sometimes smile when happy, frown when sad, scowl when angry, and so on, as proposed by the common view, more than what would be expected by chance. Yet how people communicate anger, disgust, fear, happiness, sadness, and surprise varies substantially across cultures, situations, and even across people within a single situation. Furthermore, similar configurations of facial movements variably express instances of more than one emotion category. In fact, a given configuration of facial movements, such as a scowl, often communicates something other than an emotional state. Scientists agree that facial movements convey a range of information and are important for social communication, emotional or otherwise. But our review suggests an urgent need for research that examines how people actually move their faces to express emotions and other social information in the variety of contexts that make up everyday life, as well as careful study of the mechanisms by which people perceive instances of emotion in one another. We make specific research recommendations that will yield a more valid picture of how people move their faces to express emotions and how they infer emotional meaning from facial movements in situations of everyday life. This research is crucial to provide consumers of emotion research with the translational information they require.
issn1529-1006, 1539-6053
journaltitlePsychological Science in the Public Interest
shorttitleEmotional Expressions Reconsidered
titleEmotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements

issn0269-9931, 1464-0600
journaltitleCognition & Emotion
shorttitleKnowing what you're feeling and knowing what to do about it
titleKnowing what you're feeling and knowing what to do about it: Mapping the relation between emotion differentiation and emotion regulation

abstractWe study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3’s decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3’s behavior is impressive: It solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multiarmed bandit task, and shows signatures of model-based reinforcement learning. Yet, we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. Taken together, these results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.
issn0027-8424, 1091-6490
journaltitleProceedings of the National Academy of Sciences
titleUsing cognitive psychology to understand GPT-3

Oxford University Press

titleSuperintelligence: Paths, Dangers, Strategies

journaltitleIEEE Transactions on Affective Computing
shorttitleAffect Detection
titleAffect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications

notearXiv:1706.03741 [cs, stat]
titleDeep reinforcement learning from human preferences
Computer Science - Artificial Intelligence,Computer Science - Human-Computer Interaction,Computer Science - Machine Learning,RLHF,Statistics - Machine Learning

journaltitleFrontiers in Psychology
shorttitleOptimizing Students’ Mental Health and Academic Performance
titleOptimizing Students’ Mental Health and Academic Performance: AI-Enhanced Life Crafting

abstractSome individuals have very specific and differentiated emotional experiences, such as anger, shame, excitement, and happiness, whereas others have more general affective experiences of pleasure or discomfort that are not as highly differentiated. Considering that individuals with major depressive disorder (MDD) have cognitive deficits for negative information, we predicted that people with MDD would have less differentiated negative emotional experiences than would healthy people. To test this hypothesis, we assessed participants’ emotional experiences using a 7-day experience-sampling protocol. Depression was assessed using structured clinical interviews and the Beck Depression Inventory-II. As predicted, individuals with MDD had less differentiated emotional experiences than did healthy participants, but only for negative emotions. These differences were above and beyond the effects of emotional intensity and variability.
issn0956-7976, 1467-9280
journaltitlePsychological Science
shorttitleFeeling Blue or Turquoise?
titleFeeling Blue or Turquoise? Emotional Differentiation in Major Depressive Disorder

titleThe dystopian cheeriness of Singapore's useless, gaslighting AI therapist chatbot

booktitleProc. Interspeech 2011
titleAutomatic detection of anger in human-human call center dialogs

abstractAims and methodTo compare attitudes towards mental disorders in professionals working in mental health and professionals working in different areas of medicine. Levels of emotional empathy in both groups were also investigated. In total, 58 mental healthcare professionals and 60 non-mental healthcare professionals completed our attitudes towards mental disorders questionnaire and Balanced Emotional Empathy Scale.ResultsThe results reveal generally positive attitudes towards people with mental disorders in both groups. Non-mental healthcare professionals regarded people with a mental disorder as significantly more dangerous and unpredictable than did mental healthcare professionals. There was no statistically significant difference in emotional empathy between the two groups. Both groups cited illicit drug use as one of the most significant causes of mental disorder.Clinical implicationsMental healthcare professionals and non-mental healthcare professionals show broadly similar attitudes and a similar degree of empathy towards people with a mental disorder. However, non-mental healthcare professionals regard people with mental health problems as significantly more dangerous and unpredictable. There is scope for further research including examining the effects of educational interventions.
issn1758-3209, 1758-3217
journaltitleThe Psychiatrist
titleAttitudes towards mental disorders and emotional empathy in mental health and other healthcare professionals

Honolulu HI USA

booktitleProceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society
titleIncomplete Contracting and AI Alignment

abstractLarge language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Due to rapid technological advances and their extreme versatility, LLMs nowadays have millions of users and are at the cusp of being the main go-to technology for information retrieval, content generation, problem-solving, etc. Therefore, it is of great importance to thoroughly assess and scrutinize their capabilities. Due to increasingly complex and novel behavioral patterns in current LLMs, this can be done by treating them as participants in psychology experiments that were originally designed to test humans. For this purpose, the paper introduces a new field of research called "machine psychology". The paper outlines how different subfields of psychology can inform behavioral tests for LLMs. It defines methodological standards for machine psychology research, especially by focusing on policies for prompt designs. Additionally, it describes how behavioral patterns discovered in LLMs are to be interpreted. In sum, machine psychology aims to discover emergent abilities in LLMs that cannot be detected by most traditional natural language processing benchmarks.
notearXiv:2303.13988 [cs]
shorttitleMachine Psychology
titleMachine Psychology: Investigating Emergent Capabilities and Behavior in Large Language Models Using Psychological Methods

Springer Nature

abstractThe Entertainment industry is witnessing the convergence of device, network, and platform. This chapter highlights and discusses services and applications that apply these three technologies. AI has found application in the Entertainment industry in all three areas Creation, Production, and Consumption. In the areas of production, inputs from IoT sensors result in large amounts of data, which is then analyzed using Big Data analytics to arrive at actionable insights. AI with help from Big Data analyses IoT data collected from sensors and presents us with highly accurate data of consumer behavior helping us analyzing trends and patterns of purchasing. Virtual Reality based emotion personification can enhance the audience’s experience, by engaging with them through sharing of emotions thus creating an immersive VR experience. As industries enjoy the benefits of AI, Big DATA & IoT they should be equally prepared for its undesirable negative consequences. Increasing AI, Big DATA, & IoT deployments across disciplines will bring in legal, moral, and ethical issues. Appropriate regulatory measures by competent authorities after anticipating its impacts will help in striking the balance. The Management of security risks through adherence and compliance to industry best practices in data security as well as the physical security of the IoT assets will serve as a driver for AI, Big DATA, & IoT adoption. The industry needs to drive consensus on the adoption of interoperable standards and protocols that will provide the much needed end-to-end interoperability of AI management standards, Big Data Analysis, immersive production, distribution, and consumption of media. Lastly, on the human resources front companies in the Entertainment industry need to upskill their employees in a whole new set of multi-disciplinary skill sets and understanding & expertise in IoT hardware and software.
booktitleDigital Entertainment: The Next Evolution in Service Sector
titleEntertainment in Era of AI, Big Data & IoT
AI,AR & VR,Big data,Data analytics,Entertainment,IoT

abstractAbstract Previous research has suggested that invalidating childhood environments are positively related to the symptoms of eating disorders. However, it is unclear how childhood environments might impact upon the development of eating disorder symptoms. This study examined the relationship between parental invalidation and eating disorder‐related attitudes in a nonclinical sample and tested the mediating effect of attitudes towards emotional expression. Two hundred women, with a mean age of 21 years, completed measures of invalidating childhood environments, attitudes towards emotional expression, and eating pathology. Eating concerns were positively associated with recollections of an invalidating parental environment. The belief that the expression of emotions is a sign of weakness fully mediated the relationship between childhood maternal invalidation and adult eating concern. Following replication and extension to a clinical sample, these results suggest that targeting the individual's attitude towards emotional expression might reduce eating attitudes among women who have experienced an invalidating childhood environment. Copyright © 2012 John Wiley & Sons, Ltd and Eating Disorders Association.
issn1072-4133, 1099-0968
journaltitleEuropean Eating Disorders Review
titleAttitudes Towards Emotional Expression Mediate the Relationship Between Childhood Invalidation and Adult Eating Concern

titleAbout mindline at work | Workplace mental wellness platform

issn1939-1455, 0033-2909
journaltitlePsychological Bulletin
shorttitleExpertise in emotion
titleExpertise in emotion: A scoping review and unifying framework for individual differences in the mental representation of emotional experience.

issn2379-8920, 2379-8939
journaltitleIEEE Transactions on Cognitive and Developmental Systems
titleThe Perception of Emotion in Artificial Agents
family=Institute of Positive Design,

titleEmotion Typology

titleSingapore’s free AI therapy-bot is as problematic as you’d think

abstractBeing able to carefully perceive and distinguish the rich complexity in emotional experiences is a key component of psychological interventions. We review research in clinical, social, and health psychology that offers insights into the adaptive value of putting feelings into words with a high degree of complexity (i.e., emotion differentiation or emotional granularity). According to recent research, upon experiencing intense distress, individuals who experience their emotions with more granularity are less likely to resort to maladaptive self-regulatory strategies such as binge drinking, aggression, and self-injurious behavior; show less neural reactivity to rejection; and experience less severe anxiety and depressive disorders. These findings shed light on how negative emotions and stressful experiences can be transformed by people’s emotion-differentiation skill. Besides basic research suggesting that emotion differentiation is an important developmental process, evidence suggests that interventions designed to improve emotion differentiation can both reduce psychological problems and increase various strands of well-being.
issn0963-7214, 1467-8721
journaltitleCurrent Directions in Psychological Science
shorttitleUnpacking Emotion Differentiation
titleUnpacking Emotion Differentiation: Transforming Unpleasant Experience by Perceiving Distinctions in Negativity

abstractAI alignment research is the field of study dedicated to ensuring that artificial intelligence (AI) benefits humans. As machine intelligence gets more advanced, this research is becoming increasingly important. Researchers in the field share ideas across different media to speed up the exchange of information. However, this focus on speed means that the research landscape is opaque, making it difficult for young researchers to enter the field. In this project, we collected and analyzed existing AI alignment research. We found that the field is growing quickly, with several subfields emerging in parallel. We looked at the subfields and identified the prominent researchers, recurring topics, and different modes of communication in each. Furthermore, we found that a classifier trained on AI alignment research articles can detect relevant articles that we did not originally include in the dataset. We are sharing the dataset with the research community and hope to develop tools in the future that will help both established researchers and young researchers get more involved in the field.
notearXiv:2206.02841 [cs]
shorttitleResearching Alignment Research
titleResearching Alignment Research: Unsupervised Analysis

journaltitleSSRN Electronic Journal
shorttitleEthics of Generative AI and Manipulation
titleEthics of Generative AI and Manipulation: A Design-Oriented Research Agenda

abstractThis study examined whether social support and invalidation (lack of understanding and discounting by others) are differently associated with physical and mental health. Participants were 1455 patients with fibromyalgia, rheumatoid arthritis, ankylosing spondylitis, osteorarthritis, or another rheumatic disease. Participants completed online questionnaires. Social support correlated negatively with discounting responses of others (moderately) and lack of understanding (strongly). Both invalidation and social support were additively associated with patients’ mental health, but only discounting was significantly associated with patients’ physical health. This suggests that improving health of patients with rheumatic diseases requires the consideration of both social support and invalidation.
issn1359-1053, 1461-7277
journaltitleJournal of Health Psychology
titleSocial support and invalidation by others contribute uniquely to the understanding of physical and mental health of patients with rheumatic diseases

journaltitleFrontiers in Human Neuroscience
titleEmotional Granularity Effects on Event-Related Brain Potentials during Affective Picture Processing

abstractWith the transition of facial expression recognition (FER) from laboratory-controlled to challenging in-the-wild conditions and the recent success of deep learning techniques in various fields, deep neural networks have increasingly been leveraged to learn discriminative representations for automatic FER. Recent deep FER systems generally focus on two important issues: overfitting caused by a lack of sufficient training data and expression-unrelated variations, such as illumination, head pose, and identity bias. In this survey, we provide a comprehensive review of deep FER, including datasets and algorithms that provide insights into these intrinsic problems. First, we introduce the available datasets that are widely used in the literature and provide accepted data selection and evaluation principles for these datasets. We then describe the standard pipeline of a deep FER system with the related background knowledge and suggestions for applicable implementations for each stage. For the state-of-the-art in deep FER, we introduce existing novel deep neural networks and related training strategies that are designed for FER based on both static images and dynamic image sequences and discuss their advantages and limitations. Competitive performances and experimental comparisons on widely used benchmarks are also summarized. We then extend our survey to additional related issues and application scenarios. Finally, we review the remaining challenges and corresponding opportunities in this field as well as future directions for the design of robust deep FER systems.
journaltitleIEEE Transactions on Affective Computing
shorttitleDeep Facial Expression Recognition
titleDeep Facial Expression Recognition: A Survey
Databases,Deep learning,Face recognition,Facial expression recognition,Lighting,Neural networks,Three-dimensional displays,Training data,affect,deep learning,facial expression datasets,survey

New York

Guilford Press

titleCognitive-behavioral treatment of borderline personality disorder

Washington, DC, US

American Psychological Association

abstractdescribe the meaning and use of validation in DBT [dialectical behavior therapy] / start first with a definition of validation / contrast that definition with definitions of empathy / discuss the meaning of validation by describing 6 levels of validation / discuss the importance of both types of validation / importance of validating emotional, cognitive, physiological, and action response patterns will be discussed / 5 functions of validation are presented (PsycINFO Database Record (c) 2019 APA, all rights reserved)
booktitleEmpathy reconsidered: New directions in psychotherapy
titleValidation and psychotherapy
Behavior Therapy,Dialectics,Empathy,Psychotherapeutic Techniques

abstractAbstract Communicating reassurance to patients with musculoskeletal pain complaints, but no red flags, presents a dilemma of dampening worry while refraining from reinforcing undue pain behaviors. Previous research shows that reassurance does not decrease negative affect and may be perceived as not taking the symptoms seriously. Validation offers an alternative where the patient's experiences and feelings are acknowledged and has demonstrated, for other problems, a decrease in arousal which may set the stage for behavioral change. The purpose of this study was to investigate experimentally whether validation, as compared to invalidation, impacts on emotions and adherence during repeated pain tests. To this end, 50 participants were randomized to either a validation or invalidation condition. Each participant was told they would undergo four pain trials involving holding a bucket at arm's length to tolerance. During the inter‐trial interval, the experimenter provided validating or invalidating responses according to the randomization. As a proxy measure of adherence subjects were asked to engage in an additional pain test. Results indicated that validation relative to invalidation resulted in significantly more positive affect and significantly less worry. Both groups had reductions in negative affect over the trials, but there were no difference between the groups on negative affect or pain. However, adherence was more than twice as high in the validation group as compared to invalidation. These results show that a relatively simple validation procedure had significant and positive effects on emotion and increased adherence. Further research should extend these findings and explore their clinical application.
issn1090-3801, 1532-2149
journaltitleEuropean Journal of Pain
shorttitlePainfully reassuring?
titlePainfully reassuring? The effects of validation on emotions and adherence in a pain test

New York, NY, USA

Association for Computing Machinery

abstractArtificial emotion display is a key feature of social robots to communicate internal states and behaviors in familiar human terms. While humanoid robots can draw on signals such as facial expressions or voice, emotions in appearance-constrained robots can only be conveyed through less-anthropomorphic output channels. While previous work focused on identifying specific expressional designs to convey a particular emotion, little work has been done to quantify the information content of different modalities and how they become effective in combination. Based on emotion metaphors that capture mental models of emotions, we systematically designed and validated a set of 28 different uni- and multimodal expressions for the basic emotions joy, sadness, fear and anger using the most common output modalities color, motion and sound. Classification accuracy and users' confidence of emotion assignment were evaluated in an empirical study with 33 participants and a robot probe. The findings are distilled into a set of recommendations about which modalities are most effective in communicating basic artificial emotion. Combining color with planar motion offered the overall best cost/benefit ratio by making use of redundant multimodal coding. Furthermore, modalities differed in their degree of effectiveness to communicate single emotions. Joy was best conveyed via color and motion, sadness via sound, fear via motion and anger via color.
booktitleProceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction
seriesHRI '18
titleMultimodal Expression of Artificial Emotion in Social Robots Using Color, Motion and Sound
affective computing,color,emotion,human-agent interaction,humanrobot interaction,motion,multimodal interaction,non-humanoids,robotics,social robots,sound
family=van der Maden,

abstractResonance, a powerful and pervasive phenomenon, appears to play a major role in human interactions. This article investigates the relationship between the physical mechanism of resonance and the human experience of resonance, and considers possibilities for enhancing the experience of resonance within human–robot interactions. We first introduce resonance as a widespread cultural and scientific metaphor. Then, we review the nature of “sympathetic resonance” as a physical mechanism. Following this introduction, the remainder of the article is organized in two parts. In part one, we review the role of resonance (including synchronization and rhythmic entrainment) in human cognition and social interactions. Then, in part two, we review resonance-related phenomena in robotics and artificial intelligence (AI). These two reviews serve as ground for the introduction of a design strategy and combinatorial design space for shaping resonant interactions with robots and AI. We conclude by posing hypotheses and research questions for future empirical studies and discuss a range of ethical and aesthetic issues associated with resonance in human–robot interactions.
journaltitleFrontiers in Neurorobotics
titleResonance as a Design Strategy for AI and Social Robots

abstractCurrent Artificial Intelligence (AI) methods, most based on deep learning, have facilitated progress in several fields, including computer vision and natural language understanding. The progress of these AI methods is measured using benchmarks designed to solve challenging tasks, such as visual question answering. A question remains of how much understanding is leveraged by these methods and how appropriate are the current benchmarks to measure understanding capabilities. To answer these questions, we have analysed existing benchmarks and their understanding capabilities, defined by a set of understanding capabilities, and current research streams. We show how progress has been made in benchmark development to measure understanding capabilities of AI methods and we review as well how current methods develop understanding capabilities.
notearXiv:2101.06573 [cs]
titleUnderstanding in Artificial Intelligence

abstractThe DALL-E 2 system generates original synthetic images corresponding to an input text as caption. We report here on the outcome of fourteen tests of this system designed to assess its common sense, reasoning and ability to understand complex texts. All of our prompts were intentionally much more challenging than the typical ones that have been showcased in recent weeks. Nevertheless, for 5 out of the 14 prompts, at least one of the ten images fully satisfied our requests. On the other hand, on no prompt did all of the ten images satisfy our requests.
notearXiv:2204.13807 [cs]
titleA very preliminary analysis of DALL-E 2

journaltitleJournal of Medical Internet Research
shorttitleTowards an Artificially Empathic Conversational Agent for Mental Health Applications
titleTowards an Artificially Empathic Conversational Agent for Mental Health Applications: System Design and User Perceptions

notearXiv:2203.02155 [cs]
titleTraining language models to follow instructions with human feedback
Computer Science - Artificial Intelligence,Computer Science - Computation and Language,Computer Science - Machine Learning

abstractWidespread adoption of artificial intelligence (AI) technologies is substantially affecting the human condition in ways that are not yet well understood. Negative unintended consequences abound including the perpetuation and exacerbation of societal inequalities and divisions via algorithmic decision making. We present six grand challenges for the scientific community to create AI technologies that are human-centered, that is, ethical, fair, and enhance the human condition. These grand challenges are the result of an international collaboration across academia, industry and government and represent the consensus views of a group of 26 experts in the field of human-centered artificial intelligence (HCAI). In essence, these challenges advocate for a human-centered approach to AI that (1) is centered in human well-being, (2) is designed responsibly, (3) respects privacy, (4) follows human-centered design principles, (5) is subject to appropriate governance and oversight, and (6) interacts with individuals while respecting human’s cognitive capacities. We hope that these challenges and their associated research directions serve as a call for action to conduct research and development in AI that serves as a force multiplier towards more fair, equitable and sustainable societies.
journaltitleInternational Journal of Human–Computer Interaction
titleSix Human-Centered Artificial Intelligence Grand Challenges

abstractThis article surveys the area of computational empathy, analysing different ways by which artificial agents can simulate and trigger empathy in their interactions with humans. Empathic agents can be seen as agents that have the capacity to place themselves into the position of a user’s or another agent’s emotional situation and respond appropriately. We also survey artificial agents that, by their design and behaviour, can lead users to respond emotionally as if they were experiencing the agent’s situation. In the course of this survey, we present the research conducted to date on empathic agents in light of the principles and mechanisms of empathy found in humans. We end by discussing some of the main challenges that this exciting area will be facing in the future.
issn2160-6455, 2160-6463
journaltitleACM Transactions on Interactive Intelligent Systems
shorttitleEmpathy in Virtual Agents and Robots
titleEmpathy in Virtual Agents and Robots: A Survey

abstractWe provide a new multi-task benchmark for evaluating text-to-image models. We perform a human evaluation comparing the most common open-source (Stable Diffusion) and commercial (DALL-E 2) models. Twenty computer science AI graduate students evaluated the two models, on three tasks, at three difficulty levels, across ten prompts each, providing 3,600 ratings. Text-to-image generation has seen rapid progress to the point that many recent models have demonstrated their ability to create realistic high-resolution images for various prompts. However, current text-to-image methods and the broader body of research in vision-language understanding still struggle with intricate text prompts that contain many objects with multiple attributes and relationships. We introduce a new text-to-image benchmark that contains a suite of thirty-two tasks over multiple applications that capture a model's ability to handle different features of a text prompt. For example, asking a model to generate a varying number of the same object to measure its ability to count or providing a text prompt with several objects that each have a different attribute to identify its ability to match objects and attributes correctly. Rather than subjectively evaluating text-to-image results on a set of prompts, our new multi-task benchmark consists of challenge tasks at three difficulty levels (easy, medium, and hard) and human ratings for each generated image.
notearXiv:2211.12112 [cs]
titleHuman Evaluation of Text-to-Image Models on a Multi-Task Benchmark
ine Learning

journaltitleInformation Fusion
shorttitleA review of affective computing
titleA review of affective computing: From unimodal analysis to multimodal fusion

booktitleIJCAI 2009, Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA, July 11-17, 2009
titleTowards Context Aware Emotional Intelligence in Machines: Computing Contextual Appropriateness of Affective States

abstractHumanity faces multiple existential risks in the coming decades due to technological advances in AI, and the possibility of unintended behaviors emerging from such systems. We believe that better outcomes may be possible by rigorously exploring frameworks for intelligent (goal-oriented) behavior inspired by computational neuroscience. Here, we explore how the Free Energy Principle and Active Inference (FEP-AI) framework may provide solutions for these challenges via affording the realization of control systems operating according to principles of hierarchical Bayesian modeling and prediction-error (i.e., surprisal) minimization. Such FEP-AI agents are equipped with hierarchically-organized world models capable of counterfactual planning, realized by the kinds of reciprocal message passing performed by mammalian nervous systems, so allowing for the flexible construction of representations of self-world dynamics with varying degrees of temporal depth. We will describe how such systems can not only infer the abstract causal structure of their environment, but also develop capacities for “theory of mind” and collaborative (human-aligned) decision making. Such architectures could help to sidestep potentially dangerous combinations of systems with high intelligence and human-incompatible values, since such mental processes are entangled (rather than orthogonal) in FEP-AI agents. We will further describe how (meta-)learned deep goal hierarchies may also well-describe biological systems, suggesting that potential risks from “mesa-optimisers” may actually represent one of the most promising approaches to AI safety: minimizing prediction-error relative to causal self-world models can be used to cultivate modes of policy selection and agent personalities that robustly optimize for achieving goals that are consistently aligned with both individual and shared values. Finally, we will describe how iterative policy selection and preference learning can result in "value cores" or self-reinforcing, relatively stable attracting states that agents will seek to return to through their goal-oriented imaginings and actions.
shorttitleValue Cores for Inner and Outer Alignment
titleValue Cores for Inner and Outer Alignment: Simulating Personality Formation via Iterated Policy Selection and Preference Learning with Self-World Modeling Active Inference Agents

abstractWe present Imagen, a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding. Imagen builds on the power of large transformer language models in understanding text and hinges on the strength of diffusion models in high-fidelity image generation. Our key discovery is that generic large language models (e.g. T5), pretrained on text-only corpora, are surprisingly effective at encoding text for image synthesis: increasing the size of the language model in Imagen boosts both sample fidelity and image-text alignment much more than increasing the size of the image diffusion model. Imagen achieves a new state-of-the-art FID score of 7.27 on the COCO dataset, without ever training on COCO, and human raters find Imagen samples to be on par with the COCO data itself in image-text alignment. To assess text-to-image models in greater depth, we introduce DrawBench, a comprehensive and challenging benchmark for text-to-image models. With DrawBench, we compare Imagen with recent methods including VQ-GAN+CLIP, Latent Diffusion Models, and DALL-E 2, and find that human raters prefer Imagen over other models in side-by-side comparisons, both in terms of sample quality and image-text alignment. See https://imagen.research.google/ for an overview of the results.
notearXiv:2205.11487 [cs]
titlePhotorealistic Text-to-Image Diffusion Models with Deep Language Understanding
ine Learning

abstractArtificial intelligence (AI) enabled products and services are becoming a staple of everyday life. While governments and businesses are eager to enjoy the benefits of AI innovations, the mixed impact of these autonomous and intelligent systems on human well-being has become a pressing issue. The purpose of this article is to review one of the first international standards focused on the social and ethical implications of AI: The Institute of Electrical and Electronics Engineering's (IEEE) Standard (Std) 7010-2020 Recommended Practice for Assessing the Impact of Autonomous and Intelligent Systems on Human Well-being. Incorporating well-being factors throughout the lifecycle of AI is both challenging and urgent and IEEE 7010 aims to provide guidance for those who design, deploy, and procure these technologies. Before introducing IEEE 7010, we consider possible benefits of an approach for AI centered around well-being and the measurement of well-being data. Next, we critically examine how the standard relates to approaches and perspectives in place in the AI community. Finally, we indicate where future efforts are needed for IEEE 7010 to better achieve its ambitions.
booktitle2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
noteISSN: 2577-1655
shorttitleIEEE 7010
titleIEEE 7010: A New Standard for Assessing the Well-being Implications of Artificial Intelligence
IEEE Standards,Intelligent systems,Pressing,Stakeholders,Standards,Task analysis,Technological innovation,Well-being,artificial intelligence,autonomous systems,design,measurement

abstractTracing 20 years of progress in making machines hear our emotions based on speech signal properties.
issn0001-0782, 1557-7317
journaltitleCommunications of the ACM
shorttitleSpeech emotion recognition
titleSpeech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends

issn1075-2730, 1557-9700
journaltitlePsychiatric Services
shorttitleBeyond Generic Support
titleBeyond Generic Support: Incidence and Impact of Invalidation in Peer Services for Clients With Severe Mental Illness

journaltitleCurrent Opinion in Psychology
titleA brief, but nuanced, review of emotional granularity and emotion differentiation research

issn1069-9384, 1531-5320
journaltitlePsychonomic Bulletin & Review
shorttitleArtificial cognition
titleArtificial cognition: How experimental psychology can help generate explainable artificial intelligence

issn0022-3506, 1467-6494
journaltitleJournal of Personality
shorttitlePsychological Resilience and Positive Emotional Granularity
titlePsychological Resilience and Positive Emotional Granularity: Examining the Benefits of Positive Emotions on Coping and Health
family=van der Maden,

abstractIntroduction Designing artificial intelligence (AI) to support health and wellbeing is an important and broad challenge for technologists, designers, and policymakers. Drawing upon theories of AI and cybernetics, this article offers a design framework for designing intelligent systems to optimize human wellbeing. We focus on the production of wellbeing information feedback loops in complex community settings, and discuss the case study of My Wellness Check, an intelligent system designed to support the mental health and wellbeing needs of university students and staff during the COVID-19 pandemic. Methods The basis for our discussion is the community-led design of My Wellness Check, an intelligent system that supported the mental health and wellbeing needs of university students and staff during the COVID-19 pandemic. Our system was designed to create an intelligent feedback loop to assess community wellbeing needs and to inform community action. This article provides an overview of our longitudinal assessment of students and staff wellbeing (n = 20,311) across two years of the COVID-19 pandemic. Results We further share the results of a controlled experiment (n = 1,719) demonstrating the enhanced sensitivity and user experience of our context-sensitive wellbeing assessment. Discussion Our approach to designing “AI for community wellbeing,” may generalize to the systematic improvement of human wellbeing in other human-computer systems for large-scale governance (e.g., schools, businesses, NGOs, platforms). The two main contributions are: 1) showcasing a simple way to draw from AI theory to produce more intelligent human systems, and 2) introducing a human-centered, community-led approach that may be beneficial to the field of AI.
journaltitleFrontiers in Psychology
titleA framework for designing AI systems that support community wellbeing

abstractLabeling emotions with a high degree of granularity appears to be beneficial for well-being. However, there are individual differences in the level of emotion differentiation, and some individuals do not appear to differentiate much between different emotions. Low differentiation is associated with maladaptive outcomes, therefore such individuals might benefit from interventions that can increase their level of emotion differentiation. To this end, we tested the effects of an emotion knowledge intervention on the level of emotion differentiation. One hundred and twenty participants were assigned to either an experimental or a control condition. Emotion differentiation was assessed with a Scenario Rating Task before and after the intervention, and at follow-up. As predicted, negative emotion differentiation increased significantly after the emotion knowledge intervention, and this increase was not observed in the control group. Positive emotion differentiation also increased slightly; however, it did not reach significance level. This finding suggests that an emotion knowledge intervention might be beneficial for increasing negative emotion differentiation and may have implications for the clinical context.
journaltitleFrontiers in Psychology
shorttitleFrom Knowledge to Differentiation
titleFrom Knowledge to Differentiation: Increasing Emotion Knowledge Through an Intervention Increases Negative Emotion Differentiation

Cambridge University Press

titleThe enigma of emotions

abstractAn emerging focus in affective science is the expertise that underlies healthy emotionality. A growing literature highlights emotional granularity – the ability to make fine-grained distinctions in one’s affective feelings – as an important skill. Cross-sectional evidence indicating the benefits of emotional granularity raises the question of how emotional granularity might be intentionally cultivated through training. To address this question, we present shared theoretical features of centuries-old Buddhist philosophy and modern constructionist theory that motivate the hypothesis that contemplative practices may improve granularity. We then examine the specific mindfulness-style practices originating in Buddhist traditions that are hypothesized to bolster granularity. We conclude with future directions to empirically test whether emotional granularity can be intentionally cultivated.
journaltitleFrontiers in Psychology
titleCultivating Emotional Granularity

journaltitlePLOS ONE
shorttitle"Rate My Therapist"
title"Rate My Therapist": Automated Detection of Empathy in Drug and Alcohol Counseling via Speech and Language Processing

abstractRecent years have witnessed remarkable progress in image generation task, where users can create visually astonishing images with high-quality. However, existing text-to-image diffusion models are proficient in generating concrete concepts (dogs) but encounter challenges with more abstract ones (emotions). Several efforts have been made to modify image emotions with color and style adjustments, facing limitations in effectively conveying emotions with fixed image contents. In this work, we introduce Emotional Image Content Generation (EICG), a new task to generate semantic-clear and emotion-faithful images given emotion categories. Specifically, we propose an emotion space and construct a mapping network to align it with the powerful Contrastive Language-Image Pre-training (CLIP) space, providing a concrete interpretation of abstract emotions. Attribute loss and emotion confidence are further proposed to ensure the semantic diversity and emotion fidelity of the generated images. Our method outperforms the state-of-the-art text-to-image approaches both quantitatively and qualitatively, where we derive three custom metrics, i.e., emotion accuracy, semantic clarity and semantic diversity. In addition to generation, our method can help emotion understanding and inspire emotional art design.
notearXiv:2401.04608 [cs]
titleEmoGen: Emotional Image Content Generation with Text-to-Image Diffusion Models

abstractDiffusion models have emerged as a powerful new family of deep generative models with record-breaking performance in many applications, including image synthesis, video generation, and molecule design. In this survey, we provide an overview of the rapidly expanding body of work on diffusion models, categorizing the research into three key areas: efficient sampling, improved likelihood estimation, and handling data with special structures. We also discuss the potential for combining diffusion models with other generative models for enhanced results. We further review the wide-ranging applications of diffusion models in fields spanning from computer vision, natural language generation, temporal data modeling, to interdisciplinary applications in other scientific disciplines. This survey aims to provide a contextualized, in-depth look at the state of diffusion models, identifying the key areas of focus and pointing to potential areas for further exploration. Github: https://github.com/YangLing0818/Diffusion-Models-Papers-Survey-Taxonomy.
notearXiv:2209.00796 [cs]
shorttitleDiffusion Models
titleDiffusion Models: A Comprehensive Survey of Methods and Applications
ine Learning

abstractDeep learning has emerged as a powerful machine learning technique that learns multiple layers of representations or features of the data and produces state‐of‐the‐art prediction results. Along with the success of deep learning in many application domains, deep learning is also used in sentiment analysis in recent years. This paper gives an overview of deep learning and then provides a comprehensive survey of its current applications in sentiment analysis. This article is categorized under: Fundamental Concepts of Data and Knowledge > Data Concepts Algorithmic Development > Text Mining
issn1942-4787, 1942-4795
journaltitleWIREs Data Mining and Knowledge Discovery
shorttitleDeep learning for sentiment analysis
titleDeep learning for sentiment analysis: A survey

issn1939-134X, 1040-3590
journaltitlePsychological Assessment
shorttitleThe Perceived Invalidation of Emotion Scale (PIES)
titleThe Perceived Invalidation of Emotion Scale (PIES): Development and psychometric properties of a novel measure of current emotion invalidation.

Missing 'biblatex' package
The bibliography requires the 'biblatex' package.

issn1875-4791, 1875-4805
journaltitleInternational Journal of Social Robotics
shorttitleTowards Artificial Empathy
titleTowards Artificial Empathy: How Can Artificial Empathy Follow the Developmental Pathway of Natural Empathy?

abstractIt is commonly assumed that a person’s emotional state can be readily inferred from his or her facial movements, typically called emotional expressions or facial expressions. This assumption influences legal judgments, policy decisions, national security protocols, and educational practices; guides the diagnosis and treatment of psychiatric illness, as well as the development of commercial applications; and pervades everyday social interactions as well as research in other scientific fields such as artificial intelligence, neuroscience, and computer vision. In this article, we survey examples of this widespread assumption, which we refer to as the common view, and we then examine the scientific evidence that tests this view, focusing on the six most popular emotion categories used by consumers of emotion research: anger, disgust, fear, happiness, sadness, and surprise. The available scientific evidence suggests that people do sometimes smile when happy, frown when sad, scowl when angry, and so on, as proposed by the common view, more than what would be expected by chance. Yet how people communicate anger, disgust, fear, happiness, sadness, and surprise varies substantially across cultures, situations, and even across people within a single situation. Furthermore, similar configurations of facial movements variably express instances of more than one emotion category. In fact, a given configuration of facial movements, such as a scowl, often communicates something other than an emotional state. Scientists agree that facial movements convey a range of information and are important for social communication, emotional or otherwise. But our review suggests an urgent need for research that examines how people actually move their faces to express emotions and other social information in the variety of contexts that make up everyday life, as well as careful study of the mechanisms by which people perceive instances of emotion in one another. We make specific research recommendations that will yield a more valid picture of how people move their faces to express emotions and how they infer emotional meaning from facial movements in situations of everyday life. This research is crucial to provide consumers of emotion research with the translational information they require.
issn1529-1006, 1539-6053
journaltitlePsychological Science in the Public Interest
shorttitleEmotional Expressions Reconsidered
titleEmotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements

issn0269-9931, 1464-0600
journaltitleCognition & Emotion
shorttitleKnowing what you're feeling and knowing what to do about it
titleKnowing what you're feeling and knowing what to do about it: Mapping the relation between emotion differentiation and emotion regulation

abstractWe study GPT-3, a recent large language model, using tools from cognitive psychology. More specifically, we assess GPT-3’s decision-making, information search, deliberation, and causal reasoning abilities on a battery of canonical experiments from the literature. We find that much of GPT-3’s behavior is impressive: It solves vignette-based tasks similarly or better than human subjects, is able to make decent decisions from descriptions, outperforms humans in a multiarmed bandit task, and shows signatures of model-based reinforcement learning. Yet, we also find that small perturbations to vignette-based tasks can lead GPT-3 vastly astray, that it shows no signatures of directed exploration, and that it fails miserably in a causal reasoning task. Taken together, these results enrich our understanding of current large language models and pave the way for future investigations using tools from cognitive psychology to study increasingly capable and opaque artificial agents.
issn0027-8424, 1091-6490
journaltitleProceedings of the National Academy of Sciences
titleUsing cognitive psychology to understand GPT-3

Oxford University Press

titleSuperintelligence: Paths, Dangers, Strategies

journaltitleIEEE Transactions on Affective Computing
shorttitleAffect Detection
titleAffect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications

notearXiv:1706.03741 [cs, stat]
titleDeep reinforcement learning from human preferences
Computer Science - Artificial Intelligence,Computer Science - Human-Computer Interaction,Computer Science - Machine Learning,RLHF,Statistics - Machine Learning

journaltitleFrontiers in Psychology
shorttitleOptimizing Students’ Mental Health and Academic Performance
titleOptimizing Students’ Mental Health and Academic Performance: AI-Enhanced Life Crafting

abstractSome individuals have very specific and differentiated emotional experiences, such as anger, shame, excitement, and happiness, whereas others have more general affective experiences of pleasure or discomfort that are not as highly differentiated. Considering that individuals with major depressive disorder (MDD) have cognitive deficits for negative information, we predicted that people with MDD would have less differentiated negative emotional experiences than would healthy people. To test this hypothesis, we assessed participants’ emotional experiences using a 7-day experience-sampling protocol. Depression was assessed using structured clinical interviews and the Beck Depression Inventory-II. As predicted, individuals with MDD had less differentiated emotional experiences than did healthy participants, but only for negative emotions. These differences were above and beyond the effects of emotional intensity and variability.
issn0956-7976, 1467-9280
journaltitlePsychological Science
shorttitleFeeling Blue or Turquoise?
titleFeeling Blue or Turquoise? Emotional Differentiation in Major Depressive Disorder

titleThe dystopian cheeriness of Singapore's useless, gaslighting AI therapist chatbot

booktitleProc. Interspeech 2011
titleAutomatic detection of anger in human-human call center dialogs

abstractAims and methodTo compare attitudes towards mental disorders in professionals working in mental health and professionals working in different areas of medicine. Levels of emotional empathy in both groups were also investigated. In total, 58 mental healthcare professionals and 60 non-mental healthcare professionals completed our attitudes towards mental disorders questionnaire and Balanced Emotional Empathy Scale.ResultsThe results reveal generally positive attitudes towards people with mental disorders in both groups. Non-mental healthcare professionals regarded people with a mental disorder as significantly more dangerous and unpredictable than did mental healthcare professionals. There was no statistically significant difference in emotional empathy between the two groups. Both groups cited illicit drug use as one of the most significant causes of mental disorder.Clinical implicationsMental healthcare professionals and non-mental healthcare professionals show broadly similar attitudes and a similar degree of empathy towards people with a mental disorder. However, non-mental healthcare professionals regard people with mental health problems as significantly more dangerous and unpredictable. There is scope for further research including examining the effects of educational interventions.
issn1758-3209, 1758-3217
journaltitleThe Psychiatrist
titleAttitudes towards mental disorders and emotional empathy in mental health and other healthcare professionals

Honolulu HI USA

booktitleProceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society
titleIncomplete Contracting and AI Alignment

abstractLarge language models (LLMs) are currently at the forefront of intertwining AI systems with human communication and everyday life. Due to rapid technological advances and their extreme versatility, LLMs nowadays have millions of users and are at the cusp of being the main go-to technology for information retrieval, content generation, problem-solving, etc. Therefore, it is of great importance to thoroughly assess and scrutinize their capabilities. Due to increasingly complex and novel behavioral patterns in current LLMs, this can be done by treating them as participants in psychology experiments that were originally designed to test humans. For this purpose, the paper introduces a new field of research called "machine psychology". The paper outlines how different subfields of psychology can inform behavioral tests for LLMs. It defines methodological standards for machine psychology research, especially by focusing on policies for prompt designs. Additionally, it describes how behavioral patterns discovered in LLMs are to be interpreted. In sum, machine psychology aims to discover emergent abilities in LLMs that cannot be detected by most traditional natural language processing benchmarks.
notearXiv:2303.13988 [cs]
shorttitleMachine Psychology
titleMachine Psychology: Investigating Emergent Capabilities and Behavior in Large Language Models Using Psychological Methods

Springer Nature

abstractThe Entertainment industry is witnessing the convergence of device, network, and platform. This chapter highlights and discusses services and applications that apply these three technologies. AI has found application in the Entertainment industry in all three areas Creation, Production, and Consumption. In the areas of production, inputs from IoT sensors result in large amounts of data, which is then analyzed using Big Data analytics to arrive at actionable insights. AI with help from Big Data analyses IoT data collected from sensors and presents us with highly accurate data of consumer behavior helping us analyzing trends and patterns of purchasing. Virtual Reality based emotion personification can enhance the audience’s experience, by engaging with them through sharing of emotions thus creating an immersive VR experience. As industries enjoy the benefits of AI, Big DATA & IoT they should be equally prepared for its undesirable negative consequences. Increasing AI, Big DATA, & IoT deployments across disciplines will bring in legal, moral, and ethical issues. Appropriate regulatory measures by competent authorities after anticipating its impacts will help in striking the balance. The Management of security risks through adherence and compliance to industry best practices in data security as well as the physical security of the IoT assets will serve as a driver for AI, Big DATA, & IoT adoption. The industry needs to drive consensus on the adoption of interoperable standards and protocols that will provide the much needed end-to-end interoperability of AI management standards, Big Data Analysis, immersive production, distribution, and consumption of media. Lastly, on the human resources front companies in the Entertainment industry need to upskill their employees in a whole new set of multi-disciplinary skill sets and understanding & expertise in IoT hardware and software.
booktitleDigital Entertainment: The Next Evolution in Service Sector
titleEntertainment in Era of AI, Big Data & IoT
AI,AR & VR,Big data,Data analytics,Entertainment,IoT

abstractAbstract Previous research has suggested that invalidating childhood environments are positively related to the symptoms of eating disorders. However, it is unclear how childhood environments might impact upon the development of eating disorder symptoms. This study examined the relationship between parental invalidation and eating disorder‐related attitudes in a nonclinical sample and tested the mediating effect of attitudes towards emotional expression. Two hundred women, with a mean age of 21 years, completed measures of invalidating childhood environments, attitudes towards emotional expression, and eating pathology. Eating concerns were positively associated with recollections of an invalidating parental environment. The belief that the expression of emotions is a sign of weakness fully mediated the relationship between childhood maternal invalidation and adult eating concern. Following replication and extension to a clinical sample, these results suggest that targeting the individual's attitude towards emotional expression might reduce eating attitudes among women who have experienced an invalidating childhood environment. Copyright © 2012 John Wiley & Sons, Ltd and Eating Disorders Association.
issn1072-4133, 1099-0968
journaltitleEuropean Eating Disorders Review
titleAttitudes Towards Emotional Expression Mediate the Relationship Between Childhood Invalidation and Adult Eating Concern

titleAbout mindline at work | Workplace mental wellness platform

issn1939-1455, 0033-2909
journaltitlePsychological Bulletin
shorttitleExpertise in emotion
titleExpertise in emotion: A scoping review and unifying framework for individual differences in the mental representation of emotional experience.

issn2379-8920, 2379-8939
journaltitleIEEE Transactions on Cognitive and Developmental Systems
titleThe Perception of Emotion in Artificial Agents
family=Institute of Positive Design,

titleEmotion Typology

titleSingapore’s free AI therapy-bot is as problematic as you’d think

abstractBeing able to carefully perceive and distinguish the rich complexity in emotional experiences is a key component of psychological interventions. We review research in clinical, social, and health psychology that offers insights into the adaptive value of putting feelings into words with a high degree of complexity (i.e., emotion differentiation or emotional granularity). According to recent research, upon experiencing intense distress, individuals who experience their emotions with more granularity are less likely to resort to maladaptive self-regulatory strategies such as binge drinking, aggression, and self-injurious behavior; show less neural reactivity to rejection; and experience less severe anxiety and depressive disorders. These findings shed light on how negative emotions and stressful experiences can be transformed by people’s emotion-differentiation skill. Besides basic research suggesting that emotion differentiation is an important developmental process, evidence suggests that interventions designed to improve emotion differentiation can both reduce psychological problems and increase various strands of well-being.
issn0963-7214, 1467-8721
journaltitleCurrent Directions in Psychological Science
shorttitleUnpacking Emotion Differentiation
titleUnpacking Emotion Differentiation: Transforming Unpleasant Experience by Perceiving Distinctions in Negativity

abstractAI alignment research is the field of study dedicated to ensuring that artificial intelligence (AI) benefits humans. As machine intelligence gets more advanced, this research is becoming increasingly important. Researchers in the field share ideas across different media to speed up the exchange of information. However, this focus on speed means that the research landscape is opaque, making it difficult for young researchers to enter the field. In this project, we collected and analyzed existing AI alignment research. We found that the field is growing quickly, with several subfields emerging in parallel. We looked at the subfields and identified the prominent researchers, recurring topics, and different modes of communication in each. Furthermore, we found that a classifier trained on AI alignment research articles can detect relevant articles that we did not originally include in the dataset. We are sharing the dataset with the research community and hope to develop tools in the future that will help both established researchers and young researchers get more involved in the field.
notearXiv:2206.02841 [cs]
shorttitleResearching Alignment Research
titleResearching Alignment Research: Unsupervised Analysis

journaltitleSSRN Electronic Journal
shorttitleEthics of Generative AI and Manipulation
titleEthics of Generative AI and Manipulation: A Design-Oriented Research Agenda

abstractThis study examined whether social support and invalidation (lack of understanding and discounting by others) are differently associated with physical and mental health. Participants were 1455 patients with fibromyalgia, rheumatoid arthritis, ankylosing spondylitis, osteorarthritis, or another rheumatic disease. Participants completed online questionnaires. Social support correlated negatively with discounting responses of others (moderately) and lack of understanding (strongly). Both invalidation and social support were additively associated with patients’ mental health, but only discounting was significantly associated with patients’ physical health. This suggests that improving health of patients with rheumatic diseases requires the consideration of both social support and invalidation.
issn1359-1053, 1461-7277
journaltitleJournal of Health Psychology
titleSocial support and invalidation by others contribute uniquely to the understanding of physical and mental health of patients with rheumatic diseases

journaltitleFrontiers in Human Neuroscience
titleEmotional Granularity Effects on Event-Related Brain Potentials during Affective Picture Processing

abstractWith the transition of facial expression recognition (FER) from laboratory-controlled to challenging in-the-wild conditions and the recent success of deep learning techniques in various fields, deep neural networks have increasingly been leveraged to learn discriminative representations for automatic FER. Recent deep FER systems generally focus on two important issues: overfitting caused by a lack of sufficient training data and expression-unrelated variations, such as illumination, head pose, and identity bias. In this survey, we provide a comprehensive review of deep FER, including datasets and algorithms that provide insights into these intrinsic problems. First, we introduce the available datasets that are widely used in the literature and provide accepted data selection and evaluation principles for these datasets. We then describe the standard pipeline of a deep FER system with the related background knowledge and suggestions for applicable implementations for each stage. For the state-of-the-art in deep FER, we introduce existing novel deep neural networks and related training strategies that are designed for FER based on both static images and dynamic image sequences and discuss their advantages and limitations. Competitive performances and experimental comparisons on widely used benchmarks are also summarized. We then extend our survey to additional related issues and application scenarios. Finally, we review the remaining challenges and corresponding opportunities in this field as well as future directions for the design of robust deep FER systems.
journaltitleIEEE Transactions on Affective Computing
shorttitleDeep Facial Expression Recognition
titleDeep Facial Expression Recognition: A Survey
Databases,Deep learning,Face recognition,Facial expression recognition,Lighting,Neural networks,Three-dimensional displays,Training data,affect,deep learning,facial expression datasets,survey

New York

Guilford Press

titleCognitive-behavioral treatment of borderline personality disorder

Washington, DC, US

American Psychological Association

abstractdescribe the meaning and use of validation in DBT [dialectical behavior therapy] / start first with a definition of validation / contrast that definition with definitions of empathy / discuss the meaning of validation by describing 6 levels of validation / discuss the importance of both types of validation / importance of validating emotional, cognitive, physiological, and action response patterns will be discussed / 5 functions of validation are presented (PsycINFO Database Record (c) 2019 APA, all rights reserved)
booktitleEmpathy reconsidered: New directions in psychotherapy
titleValidation and psychotherapy
Behavior Therapy,Dialectics,Empathy,Psychotherapeutic Techniques

abstractAbstract Communicating reassurance to patients with musculoskeletal pain complaints, but no red flags, presents a dilemma of dampening worry while refraining from reinforcing undue pain behaviors. Previous research shows that reassurance does not decrease negative affect and may be perceived as not taking the symptoms seriously. Validation offers an alternative where the patient's experiences and feelings are acknowledged and has demonstrated, for other problems, a decrease in arousal which may set the stage for behavioral change. The purpose of this study was to investigate experimentally whether validation, as compared to invalidation, impacts on emotions and adherence during repeated pain tests. To this end, 50 participants were randomized to either a validation or invalidation condition. Each participant was told they would undergo four pain trials involving holding a bucket at arm's length to tolerance. During the inter‐trial interval, the experimenter provided validating or invalidating responses according to the randomization. As a proxy measure of adherence subjects were asked to engage in an additional pain test. Results indicated that validation relative to invalidation resulted in significantly more positive affect and significantly less worry. Both groups had reductions in negative affect over the trials, but there were no difference between the groups on negative affect or pain. However, adherence was more than twice as high in the validation group as compared to invalidation. These results show that a relatively simple validation procedure had significant and positive effects on emotion and increased adherence. Further research should extend these findings and explore their clinical application.
issn1090-3801, 1532-2149
journaltitleEuropean Journal of Pain
shorttitlePainfully reassuring?
titlePainfully reassuring? The effects of validation on emotions and adherence in a pain test

New York, NY, USA

Association for Computing Machinery

abstractArtificial emotion display is a key feature of social robots to communicate internal states and behaviors in familiar human terms. While humanoid robots can draw on signals such as facial expressions or voice, emotions in appearance-constrained robots can only be conveyed through less-anthropomorphic output channels. While previous work focused on identifying specific expressional designs to convey a particular emotion, little work has been done to quantify the information content of different modalities and how they become effective in combination. Based on emotion metaphors that capture mental models of emotions, we systematically designed and validated a set of 28 different uni- and multimodal expressions for the basic emotions joy, sadness, fear and anger using the most common output modalities color, motion and sound. Classification accuracy and users' confidence of emotion assignment were evaluated in an empirical study with 33 participants and a robot probe. The findings are distilled into a set of recommendations about which modalities are most effective in communicating basic artificial emotion. Combining color with planar motion offered the overall best cost/benefit ratio by making use of redundant multimodal coding. Furthermore, modalities differed in their degree of effectiveness to communicate single emotions. Joy was best conveyed via color and motion, sadness via sound, fear via motion and anger via color.
booktitleProceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction
seriesHRI '18
titleMultimodal Expression of Artificial Emotion in Social Robots Using Color, Motion and Sound
affective computing,color,emotion,human-agent interaction,humanrobot interaction,motion,multimodal interaction,non-humanoids,robotics,social robots,sound
family=van der Maden,

abstractResonance, a powerful and pervasive phenomenon, appears to play a major role in human interactions. This article investigates the relationship between the physical mechanism of resonance and the human experience of resonance, and considers possibilities for enhancing the experience of resonance within human–robot interactions. We first introduce resonance as a widespread cultural and scientific metaphor. Then, we review the nature of “sympathetic resonance” as a physical mechanism. Following this introduction, the remainder of the article is organized in two parts. In part one, we review the role of resonance (including synchronization and rhythmic entrainment) in human cognition and social interactions. Then, in part two, we review resonance-related phenomena in robotics and artificial intelligence (AI). These two reviews serve as ground for the introduction of a design strategy and combinatorial design space for shaping resonant interactions with robots and AI. We conclude by posing hypotheses and research questions for future empirical studies and discuss a range of ethical and aesthetic issues associated with resonance in human–robot interactions.
journaltitleFrontiers in Neurorobotics
titleResonance as a Design Strategy for AI and Social Robots

abstractCurrent Artificial Intelligence (AI) methods, most based on deep learning, have facilitated progress in several fields, including computer vision and natural language understanding. The progress of these AI methods is measured using benchmarks designed to solve challenging tasks, such as visual question answering. A question remains of how much understanding is leveraged by these methods and how appropriate are the current benchmarks to measure understanding capabilities. To answer these questions, we have analysed existing benchmarks and their understanding capabilities, defined by a set of understanding capabilities, and current research streams. We show how progress has been made in benchmark development to measure understanding capabilities of AI methods and we review as well how current methods develop understanding capabilities.
notearXiv:2101.06573 [cs]
titleUnderstanding in Artificial Intelligence

abstractThe DALL-E 2 system generates original synthetic images corresponding to an input text as caption. We report here on the outcome of fourteen tests of this system designed to assess its common sense, reasoning and ability to understand complex texts. All of our prompts were intentionally much more challenging than the typical ones that have been showcased in recent weeks. Nevertheless, for 5 out of the 14 prompts, at least one of the ten images fully satisfied our requests. On the other hand, on no prompt did all of the ten images satisfy our requests.
notearXiv:2204.13807 [cs]
titleA very preliminary analysis of DALL-E 2

journaltitleJournal of Medical Internet Research
shorttitleTowards an Artificially Empathic Conversational Agent for Mental Health Applications
titleTowards an Artificially Empathic Conversational Agent for Mental Health Applications: System Design and User Perceptions

notearXiv:2203.02155 [cs]
titleTraining language models to follow instructions with human feedback
Computer Science - Artificial Intelligence,Computer Science - Computation and Language,Computer Science - Machine Learning

abstractWidespread adoption of artificial intelligence (AI) technologies is substantially affecting the human condition in ways that are not yet well understood. Negative unintended consequences abound including the perpetuation and exacerbation of societal inequalities and divisions via algorithmic decision making. We present six grand challenges for the scientific community to create AI technologies that are human-centered, that is, ethical, fair, and enhance the human condition. These grand challenges are the result of an international collaboration across academia, industry and government and represent the consensus views of a group of 26 experts in the field of human-centered artificial intelligence (HCAI). In essence, these challenges advocate for a human-centered approach to AI that (1) is centered in human well-being, (2) is designed responsibly, (3) respects privacy, (4) follows human-centered design principles, (5) is subject to appropriate governance and oversight, and (6) interacts with individuals while respecting human’s cognitive capacities. We hope that these challenges and their associated research directions serve as a call for action to conduct research and development in AI that serves as a force multiplier towards more fair, equitable and sustainable societies.
journaltitleInternational Journal of Human–Computer Interaction
titleSix Human-Centered Artificial Intelligence Grand Challenges

abstractThis article surveys the area of computational empathy, analysing different ways by which artificial agents can simulate and trigger empathy in their interactions with humans. Empathic agents can be seen as agents that have the capacity to place themselves into the position of a user’s or another agent’s emotional situation and respond appropriately. We also survey artificial agents that, by their design and behaviour, can lead users to respond emotionally as if they were experiencing the agent’s situation. In the course of this survey, we present the research conducted to date on empathic agents in light of the principles and mechanisms of empathy found in humans. We end by discussing some of the main challenges that this exciting area will be facing in the future.
issn2160-6455, 2160-6463
journaltitleACM Transactions on Interactive Intelligent Systems
shorttitleEmpathy in Virtual Agents and Robots
titleEmpathy in Virtual Agents and Robots: A Survey

abstractWe provide a new multi-task benchmark for evaluating text-to-image models. We perform a human evaluation comparing the most common open-source (Stable Diffusion) and commercial (DALL-E 2) models. Twenty computer science AI graduate students evaluated the two models, on three tasks, at three difficulty levels, across ten prompts each, providing 3,600 ratings. Text-to-image generation has seen rapid progress to the point that many recent models have demonstrated their ability to create realistic high-resolution images for various prompts. However, current text-to-image methods and the broader body of research in vision-language understanding still struggle with intricate text prompts that contain many objects with multiple attributes and relationships. We introduce a new text-to-image benchmark that contains a suite of thirty-two tasks over multiple applications that capture a model's ability to handle different features of a text prompt. For example, asking a model to generate a varying number of the same object to measure its ability to count or providing a text prompt with several objects that each have a different attribute to identify its ability to match objects and attributes correctly. Rather than subjectively evaluating text-to-image results on a set of prompts, our new multi-task benchmark consists of challenge tasks at three difficulty levels (easy, medium, and hard) and human ratings for each generated image.
notearXiv:2211.12112 [cs]
titleHuman Evaluation of Text-to-Image Models on a Multi-Task Benchmark
ine Learning

journaltitleInformation Fusion
shorttitleA review of affective computing
titleA review of affective computing: From unimodal analysis to multimodal fusion

booktitleIJCAI 2009, Proceedings of the 21st International Joint Conference on Artificial Intelligence, Pasadena, California, USA, July 11-17, 2009
titleTowards Context Aware Emotional Intelligence in Machines: Computing Contextual Appropriateness of Affective States

abstractHumanity faces multiple existential risks in the coming decades due to technological advances in AI, and the possibility of unintended behaviors emerging from such systems. We believe that better outcomes may be possible by rigorously exploring frameworks for intelligent (goal-oriented) behavior inspired by computational neuroscience. Here, we explore how the Free Energy Principle and Active Inference (FEP-AI) framework may provide solutions for these challenges via affording the realization of control systems operating according to principles of hierarchical Bayesian modeling and prediction-error (i.e., surprisal) minimization. Such FEP-AI agents are equipped with hierarchically-organized world models capable of counterfactual planning, realized by the kinds of reciprocal message passing performed by mammalian nervous systems, so allowing for the flexible construction of representations of self-world dynamics with varying degrees of temporal depth. We will describe how such systems can not only infer the abstract causal structure of their environment, but also develop capacities for “theory of mind” and collaborative (human-aligned) decision making. Such architectures could help to sidestep potentially dangerous combinations of systems with high intelligence and human-incompatible values, since such mental processes are entangled (rather than orthogonal) in FEP-AI agents. We will further describe how (meta-)learned deep goal hierarchies may also well-describe biological systems, suggesting that potential risks from “mesa-optimisers” may actually represent one of the most promising approaches to AI safety: minimizing prediction-error relative to causal self-world models can be used to cultivate modes of policy selection and agent personalities that robustly optimize for achieving goals that are consistently aligned with both individual and shared values. Finally, we will describe how iterative policy selection and preference learning can result in "value cores" or self-reinforcing, relatively stable attracting states that agents will seek to return to through their goal-oriented imaginings and actions.
shorttitleValue Cores for Inner and Outer Alignment
titleValue Cores for Inner and Outer Alignment: Simulating Personality Formation via Iterated Policy Selection and Preference Learning with Self-World Modeling Active Inference Agents

abstractWe present Imagen, a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding. Imagen builds on the power of large transformer language models in understanding text and hinges on the strength of diffusion models in high-fidelity image generation. Our key discovery is that generic large language models (e.g. T5), pretrained on text-only corpora, are surprisingly effective at encoding text for image synthesis: increasing the size of the language model in Imagen boosts both sample fidelity and image-text alignment much more than increasing the size of the image diffusion model. Imagen achieves a new state-of-the-art FID score of 7.27 on the COCO dataset, without ever training on COCO, and human raters find Imagen samples to be on par with the COCO data itself in image-text alignment. To assess text-to-image models in greater depth, we introduce DrawBench, a comprehensive and challenging benchmark for text-to-image models. With DrawBench, we compare Imagen with recent methods including VQ-GAN+CLIP, Latent Diffusion Models, and DALL-E 2, and find that human raters prefer Imagen over other models in side-by-side comparisons, both in terms of sample quality and image-text alignment. See https://imagen.research.google/ for an overview of the results.
notearXiv:2205.11487 [cs]
titlePhotorealistic Text-to-Image Diffusion Models with Deep Language Understanding
ine Learning

abstractArtificial intelligence (AI) enabled products and services are becoming a staple of everyday life. While governments and businesses are eager to enjoy the benefits of AI innovations, the mixed impact of these autonomous and intelligent systems on human well-being has become a pressing issue. The purpose of this article is to review one of the first international standards focused on the social and ethical implications of AI: The Institute of Electrical and Electronics Engineering's (IEEE) Standard (Std) 7010-2020 Recommended Practice for Assessing the Impact of Autonomous and Intelligent Systems on Human Well-being. Incorporating well-being factors throughout the lifecycle of AI is both challenging and urgent and IEEE 7010 aims to provide guidance for those who design, deploy, and procure these technologies. Before introducing IEEE 7010, we consider possible benefits of an approach for AI centered around well-being and the measurement of well-being data. Next, we critically examine how the standard relates to approaches and perspectives in place in the AI community. Finally, we indicate where future efforts are needed for IEEE 7010 to better achieve its ambitions.
booktitle2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
noteISSN: 2577-1655
shorttitleIEEE 7010
titleIEEE 7010: A New Standard for Assessing the Well-being Implications of Artificial Intelligence
IEEE Standards,Intelligent systems,Pressing,Stakeholders,Standards,Task analysis,Technological innovation,Well-being,artificial intelligence,autonomous systems,design,measurement

abstractTracing 20 years of progress in making machines hear our emotions based on speech signal properties.
issn0001-0782, 1557-7317
journaltitleCommunications of the ACM
shorttitleSpeech emotion recognition
titleSpeech emotion recognition: two decades in a nutshell, benchmarks, and ongoing trends

issn1075-2730, 1557-9700
journaltitlePsychiatric Services
shorttitleBeyond Generic Support
titleBeyond Generic Support: Incidence and Impact of Invalidation in Peer Services for Clients With Severe Mental Illness

journaltitleCurrent Opinion in Psychology
titleA brief, but nuanced, review of emotional granularity and emotion differentiation research

issn1069-9384, 1531-5320
journaltitlePsychonomic Bulletin & Review
shorttitleArtificial cognition
titleArtificial cognition: How experimental psychology can help generate explainable artificial intelligence

issn0022-3506, 1467-6494
journaltitleJournal of Personality
shorttitlePsychological Resilience and Positive Emotional Granularity
titlePsychological Resilience and Positive Emotional Granularity: Examining the Benefits of Positive Emotions on Coping and Health
family=van der Maden,

abstractIntroduction Designing artificial intelligence (AI) to support health and wellbeing is an important and broad challenge for technologists, designers, and policymakers. Drawing upon theories of AI and cybernetics, this article offers a design framework for designing intelligent systems to optimize human wellbeing. We focus on the production of wellbeing information feedback loops in complex community settings, and discuss the case study of My Wellness Check, an intelligent system designed to support the mental health and wellbeing needs of university students and staff during the COVID-19 pandemic. Methods The basis for our discussion is the community-led design of My Wellness Check, an intelligent system that supported the mental health and wellbeing needs of university students and staff during the COVID-19 pandemic. Our system was designed to create an intelligent feedback loop to assess community wellbeing needs and to inform community action. This article provides an overview of our longitudinal assessment of students and staff wellbeing (n = 20,311) across two years of the COVID-19 pandemic. Results We further share the results of a controlled experiment (n = 1,719) demonstrating the enhanced sensitivity and user experience of our context-sensitive wellbeing assessment. Discussion Our approach to designing “AI for community wellbeing,” may generalize to the systematic improvement of human wellbeing in other human-computer systems for large-scale governance (e.g., schools, businesses, NGOs, platforms). The two main contributions are: 1) showcasing a simple way to draw from AI theory to produce more intelligent human systems, and 2) introducing a human-centered, community-led approach that may be beneficial to the field of AI.
journaltitleFrontiers in Psychology
titleA framework for designing AI systems that support community wellbeing

abstractLabeling emotions with a high degree of granularity appears to be beneficial for well-being. However, there are individual differences in the level of emotion differentiation, and some individuals do not appear to differentiate much between different emotions. Low differentiation is associated with maladaptive outcomes, therefore such individuals might benefit from interventions that can increase their level of emotion differentiation. To this end, we tested the effects of an emotion knowledge intervention on the level of emotion differentiation. One hundred and twenty participants were assigned to either an experimental or a control condition. Emotion differentiation was assessed with a Scenario Rating Task before and after the intervention, and at follow-up. As predicted, negative emotion differentiation increased significantly after the emotion knowledge intervention, and this increase was not observed in the control group. Positive emotion differentiation also increased slightly; however, it did not reach significance level. This finding suggests that an emotion knowledge intervention might be beneficial for increasing negative emotion differentiation and may have implications for the clinical context.
journaltitleFrontiers in Psychology
shorttitleFrom Knowledge to Differentiation
titleFrom Knowledge to Differentiation: Increasing Emotion Knowledge Through an Intervention Increases Negative Emotion Differentiation

Cambridge University Press

titleThe enigma of emotions

abstractAn emerging focus in affective science is the expertise that underlies healthy emotionality. A growing literature highlights emotional granularity – the ability to make fine-grained distinctions in one’s affective feelings – as an important skill. Cross-sectional evidence indicating the benefits of emotional granularity raises the question of how emotional granularity might be intentionally cultivated through training. To address this question, we present shared theoretical features of centuries-old Buddhist philosophy and modern constructionist theory that motivate the hypothesis that contemplative practices may improve granularity. We then examine the specific mindfulness-style practices originating in Buddhist traditions that are hypothesized to bolster granularity. We conclude with future directions to empirically test whether emotional granularity can be intentionally cultivated.
journaltitleFrontiers in Psychology
titleCultivating Emotional Granularity

journaltitlePLOS ONE
shorttitle"Rate My Therapist"
title"Rate My Therapist": Automated Detection of Empathy in Drug and Alcohol Counseling via Speech and Language Processing