Fin G, Moreno-Murcia JA, León J, Baretta E and Nodari Júnior RJ (2019) Teachers' Interpersonal Style in Physical Education: Exploring Patterns of Students' Self-Determined Motivation and Enjoyment of Physical Activity in a Longitudinal Study. Front. Psychol. 9:2721. doi: 10.3389/fpsyg.2018.02721 This longitudinal study explored patterns of basic psychological needs and self-determined motivation, as well as its association with the teaching style and the physical activity enjoyment in a group of students. The sample consisted of 200 secondary education students (105 girls and 95 boys) aged 11 to 13 years (M = 12.65, SD = 0.79) at the start of the study. Students were assessed twice in a 22 month-period. Descriptive analyses were conducted between major variables at both time points, and to explore the number and nature of clusters, we relied on latent profile analysis. The statistical analyses showed three different patterns: downward, stable and upward, with different outcomes and determinants. The downward pattern was associated with more negative enjoyment and a controlling style, while the upward pattern with more positive enjoyment and autonomy support. This study, which was based on a person-centered approach, provided a bigger picture of the interplay between autonomy, competence, relatedness, self-determined motivation, autonomy/control support, and physical activity enjoyment.

Keywords: motivation, self-determination, physical activity, enjoyment, adolescents

### INTRODUCTION

The importance of regular exercise is indisputable in maintaining quality of life and health. Aspects related to physical exercise are particularly investigated in childhood and adolescence, since during this period a significant relationship is established between acquired behaviors and the resulting actions in adulthood (Hallal et al., 2012). Thus, a gradual increase in physical activity would be beneficial; however, a decline in these activities has been observed from adolescence onward, with high rates of physical inactivity in young people (Sousa and Hallal, 2015).

The factors that interfere in exercise adherence and a healthy lifestyle are related to motivational aspects. In this respect, the self-determination theory (SDT; Deci and Ryan, 1985, 2000), and the hierarchical model of intrinsic and extrinsic motivation (HMIEM) (Vallerand, 1997, 2001, 2007) offer an explanation of factors that promote or inhibit a healthy lifestyle, such as enjoyment during physical activity classes.

Basic psychological needs (autonomy, competence, and relatedness) and motivation are factors that have been extensively studied (Tessier et al., 2010; Cheon et al., 2014; Sparks et al., 2015). Cross-sectionals and longitudinal studies have focused on the linear relationships between autonomy, competence, relatedness or self-determined motivation and positive outcomes during physical activity classes (e.g., enjoyment, engagement or satisfaction). Similar relationships have been observed between contextual factors (e.g., teachers' control or autonomy support) and autonomy, competence, relatedness or self-determined motivation (Lim and Wang, 2009; Cairney et al., 2012; Cheon et al., 2012).

Unfortunately, these studies only provide information on the relationship between the determinant and the outcome, such as high values on autonomy correspond to high values on enjoyment. These types of studies rely on a variable-centered approach. This approach provides a small picture of the interplay among the studied variables, however, a person-centered approach allows the study of different variable configurations whilst providing a bigger picture (Kimiecik and Horn, 2017; León and Liew, 2017). For instance, in longitudinal research we could forecast groups of students based on autonomy, competence, relatedness and self-determined motivation changes, and then test how these groups differ on the outcomes (e.g., enjoyment) and the determinants (e.g., autonomy support).

The present study addresses this gap in the literature by examining students' patterns of autonomy, competence, relatedness and self-determined motivation changes, and how such patterns predict enjoyment/disenjoyment, and how these patterns relate to autonomy support.

### Self-Determination Theory and Basic Education

The SDT, proposed by Deci and Ryan (1985, 2017), aims to explain human behavior, based on different motivational styles, influences of the context, and interpersonal perceptions. Three basic psychological needs are related to motivation: autonomy, which is related to the level of independence and control of the choices made by an individual; competence, which refers to a person's ability to perform a task; and relatedness, which is linked to the perception of a sense of connection with other people (Deci and Ryan, 2012).

Motivation varies from the most self-determined form, called intrinsic motivation, when the choice is personal, characterizing total autonomy in terms of the activity, which generates interest, enjoyment and satisfaction inherent to the activity (Deci and Ryan, 2000) to the least self-determined levels, which are an extrinsic motivation and amotivation (Deci and Ryan, 2000).

Satisfaction of basic psychological needs and self-determined motivation are associated with enjoyment and effort in performing activities and other positives outcomes (Deci and Ryan, 2017). For instance, Brunet et al. (2016) observed in adolescents that autonomy, competence and relatedness predicted moderate-to-vigorous-intensity physical activity 4 months later. Similarly, Richards et al. (2017) observed that self-determined motivation predicted physical activity.

Unfortunately, these longitudinal studies relied on a variable-centered approach, and thus, it is not possible to see a big picture of the interplay among the studied variables. Nevertheless, there are cross-sectional studies that relied on a person-centered approach to increase adolescents' motivation. Most of these studies analyzed the coexistence of motivational regulations (Ullrich-French and Cox, 2009; Wang et al., 2016), but little information is available about the interplay between autonomy, competence, relatedness and self-determined motivation. To the best of our knowledge, only Kimiecik and Horn (2017) explored different clusters based on adolescents' needs, observing three clusters based on students' autonomy, competence, relatedness, subjective well-being and self-awareness, and perceived choice as indicators of self-determination. In this cross-sectional study, they observed three different clusters: low, moderate and high. Interestingly, students in the high cluster reported the highest levels of moderate-to-vigorous physical activity and the most healthy eating patterns.

### Interpersonal Teacher Style in Physical Education

In the educational and behavioral context, the SDT was broadened by Vallerand (1997, 2001, 2007) through the hierarchical model of intrinsic and extrinsic motivation (HMIEM). This model analyzes motivational changes (intrinsic, extrinsic and amotivation) that may occur over time, depending on three levels of social factors. The global level refers to overall motivation, which is related to family and cultural aspects developed in the first socialization processes. The contextual level refers to a specific context, such as physical activities. Finally, the situational level is influenced by the global level, and as a function of the learning styles displayed in the latter, perceptions and future styles at the contextual level may vary, in a specific situation, a particular physical exercise, for example.

During physical education classes, one of the contextual factors that can influence motivation is the teacher's interpersonal style. Teachers offer support along a continuum, which ranges from extreme control to total support for autonomy (Tessier et al., 2010). According to Reeve (2009), supporting autonomy consists of nurturing the student's inner motivational resources, providing basic explanations, using non-controlling language, and showing patience in allowing them the necessary time to learn at their own pace. As such, students become more involved in decision making and use an inquisitive methodology, attributing more importance to the process and encouraging effort and personal improvement (Moreno-Murcia et al., 2014).

In contrast to an interpersonal style that supports autonomy in the classroom, teachers with a controlling style ensure that activities are performed exactly according to their way of thinking, feeling and behaving. When teachers use controlling support, they make students abandon their own inner motivational resources in order to undertake activities and seek to solve problems based on the teachers' needs (Moreno-Murcia et al., 2012).

The motor and affective experiences perceived by students during class are influenced by the teachers' interaction and how they present class content (Jang et al., 2012; Ntoumanis et al., 2018). The teaching strategies used can affect how competent students feel to execute activities and pursue results, leading to a positive or negative change in behavior in relation to the proposed objective (Sparks et al., 2015).

Investigations on the SDT indicate that a teaching style aimed at supporting autonomy results in improved student motivation. Wang et al. (2016) observed that students in clusters characterized by the highest levels of autonomous regulation were associated with teachers with a more autonomy-supportive style.

### Physical Activity Enjoyment

A determining factor in physical education classes and physical activity in general is the feeling of joy. Studies show that self-determined motivation to engage in physical activity is influenced by perceived enjoyment and challenges that activities can generate in children and adolescents (Motl et al., 2001; Cairney et al., 2012). The pleasure experienced during physical education classes was also identified as a predictor of future physical activity in both children and adolescents (Sallis et al., 2000).

The SDT highlights the positive effect of enjoyment and the feeling of pleasure as a crucial point for physical education self-determined motivation. Perceptions of competence, autonomy, success and good relationships with others increase pleasure during physical activity and reduce negative perceptions such as boredom (Baron and Downey, 2007). Taken together, these findings suggest that autonomy, competence and relatedness may be important factors that affect the pleasure experienced during physical activity.

## The Present Study

Understanding different patterns of self-determined motivation in physical education classes may help teachers improve the quality of interactions with students, favoring and increasing positive experiences during classes (Moreno-Murcia and Sánchez-Latorre, 2016). Different researchers studied the relationship between motivational variables within the SDT, unfortunately there is lack of research focusing on patterns of longitudinal changes. Therefore, in this longitudinal study we aim to explore patterns of change of motivational variables (autonomy, competence, relatedness and self-determined motivation), and how these patterns differ on enjoyment and on teacher style (autonomy/control support).

We hypothesize the existence of different clusters and that the students in motivational clusters with more self-determination will show more pleasure in practicing physical activity and more autonomy support from their teachers. Similarly, we expect that motivational clusters with less self-determination will show less pleasure in practicing the activity and will receive more controlled classes from their teachers.

## MATERIALS AND METHODS

### Participants

The sample consisted of 200 schoolchildren, 105 girls, and 95 boys, in the final grades of elementary schooling at four public schools located in the urban zone of three municipalities in Midwest Santa Catarina state, Brazil. The students were 11–13 years old (M = 12.65, SD = 0.79) at the beginning of the study. For the data analysis, the same individuals were considered for collection 1 when they were still in the sixth, seventh and eighth grades, and collection 2 were in the seventh, eighth and ninth grade, respectively.

### Measurements

### Autonomy Support

The Learning Climate Questionnaire (LCQ; Williams and Deci, 1996) was used to determine the students' perception of their teachers. As recommended by Núñez et al. (2012) we applied the short version, with the adaptation for Brazilian studies in Fin et al. (2017). This scale consists of 5 items preceded by the stem "My physical education teacher...," which evaluate autonomy support (e.g., "Tries to understand how I feel before suggesting a new way of doing things"). Answers were scored on a Likert-type scale ranging from 1 (I completely disagree) to 7 (I completely agree). Since this instrument has not yet been adapted to the Brazilian educational setting, Hambleton's back-translation method was used (Hambleton, 1996). Items were first translated into Portuguese and then translated back into English. Next, the questionnaires were applied to a small group of students to check for understanding and make any necessary corrections. Internal consistency of the scale was calculated using Cronbach's alpha, obtaining an alpha value of 0.91. We tested the factor structure by means of a confirmatory factor analysis. All standardized loadings were between 0.661 and 0.870. Regarding the CFA, the χ 2 value and fit indexes were: χ 2 (199, 5) = 9.686 (p = 0.08), RMSEA = 0.068 [0.000, 0.133] and CFI = 0.986.

### Control Support

The Controlling Teacher Questionnaire (CTQ), from Jang et al. (2009), was used in its modified version for physical education (Cheon et al., 2014). This scale consists of four items, preceded by "My physical education teacher is . . .," which assesses teacher control during class (e.g., "Seeks/intends to control everything I do"). The answers were scored on a Likert-type scale, whose scores varied from 1 (I completely disagree) to 7 (I completely agree). The same procedure used in the previous scale was followed. Cronbach's alpha was 0.86. All standardized loadings were between 0.270 and 0.565. Regarding the CFA, the χ 2 value and fit indexes were: χ 2 (199, 2) = 0.130 (p = 0.94), RMSEA = 0.000 [0.000, 0.035] and CFI = 0.999.

### Basic Psychological Needs

A questionnaire was applied to assess basic psychological needs in physical education (NPBEF), adapted for Portuguese by Pires et al. (2010) from the Basic Psychological Needs in Exercise Scale (BPNESp) (Vlachopoulos and Michailidou, 2006). The questionnaire consists of 12 items encompassing three dimensions: autonomy (e.g., "I feel I do activities the way I want to"), competence (e.g., "I feel I complete class activities successfully"), and relatedness (e.g., "I feel good with my classmates"). Items are preceded by the statement "Generally, in physical education..." and are scored on a 5-point Likert scale from 1 (I completely disagree) to 5 (I completely agree). Cronbach's alpha was 0.73, 0.71, and 0.85, respectively.

#### Self-Determined Motivation

fpsyg-09-02721 January 5, 2019 Time: 19:7 # 4

The Perceived Locus of Causality Questionnaire (PLOCQ) (Goudas and Biddle, 1994) was used, translated into Portuguese and validated for the Brazilian population (Tenório, 2014). The questionnaire contains 20 items and is subdivided into five dimensions: intrinsic motivation, identified regulation, introjected regulation, external regulation, and amotivation. Items are preceded by the stem "I do physical education..." and are scored on a 7-point Likert scale ranging from 1 (I completely disagree) to 5 (I completely agree). Internal consistency was 0.81, 0.76, 0.76, 0.69, and 0.74, respectively. To determine the total score of self-determined motivation, the index of self-determination (IAD): (2 × intrinsic motivation + identified regulation) – [(introjected regulation + external regulation) / 2 + 2 × amotivation] (Vallerand and Rousseau, 2001) was used.

### Physical Activity Enjoyment Scale

We applied the Physical Activity Enjoyment Scale (PACES) (Motl et al., 2001), translated by Montanha (2013), to measure the enjoyment for physical activity. The scale consists of 16 items preceded by the statement "When I am physically active..." which are divided into two groups composed of eight items each. The items of one group directly assess enjoyment, with affirmative sentences (e.g., "I enjoy it," "It's very pleasant," "It gives me energy"), and the items of the other group indirectly assess enjoyment, with negative sentences (e.g., "It makes me sad," "I dislike it," "It's not fun at all"). Answers were scored on a Likert-type scale, rated from 1 (I completely disagree) to 5 (I completely agree). Cronbach's alpha was 0.91.

### Procedure

Questionnaires were administered individually during class time on two time points. The first data collection took place in March 2015 and the second in December 2016. Both evaluations had the same sample of individuals, so the data represent the answers of the students at the beginning of the 2015 school year while they were in the sixth, seventh and eighth grades. At the end of 2016, these same students were in the seventh, eighth and ninth grade, respectively. It is important to consider that the Brazilian school year begins in February and ends in December. The questionnaires were applied in the classroom under the supervision of the authors of this study. Prior authorization was requested from management staff and teachers at the schools involved in the study, as well as the parents and/or guardians of the participants. The questionnaires were answered during physical education (PE) classes under the supervision of the researcher, who explained how subjects should complete the instrument and remained available to answer any questions that might arise during the process. Completion time was approximately 35 min, and anonymity was respected to ensure sincere answers. Whilst the students answered the questionnaire, only the researchers were in the classroom. The teachers were not present during the data collection. This study was carried out in accordance with the recommendations of Human Research Ethics Committee of Unoesc/Hust with written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Human Research Ethics Committee of Unoesc/Hust, under protocol number 937.597 on December 19, 2014.

### Data Analyses

#### Preliminary Analyses

Descriptive analyses were conducted, including Pearson's correlations between major variables at both time points (see **Table 1**). To compute variables' indicators, we began by estimating the mean for each variable. Then, we computed the difference between each time point (T2 – T1). Finally, to ease interpretation, we standardized these values (mean = 0 and standard deviation = 1).

### Latent Profile Analysis

To explore the number and nature of clusters, we relied on Latent Profile Analysis (Collins and Lanza, 2010; Berlin et al., 2014). Following the recommendations of Marsh et al. (2009) and Collins and Lanza (2010) to choose the number of clusters, we analyzed: (1) Akaike information criterion (AIC), (2) sample-size-adjusted Bayesian information criterion (SSA-BIC),

All correlations were significant (p < 0.05).

TABLE 2 | Goodness of fit for models with latent groups.

fpsyg-09-02721 January 5, 2019 Time: 19:7 # 5

AIC, Akaike Information Criteria; SSA-BIC, Sample size adjusted Bayes Information Criteria; PB-LRT, Parametric Bootstrapped Likelihood Ratio Test.

(3) Parametric Bootstrapped Likelihood Ratio Test (PB-LRT), and (4) because solutions with few participants (e.g., 1 or 5% of the total sample) may not truly represent a unique cluster, we also relied on the percentage of cases in the smallest latent subgroup of each model. All of the calculations were done with Mplus 7.4 (Muthén and Muthén, 2017).

### RESULTS

### Preliminary Analyses

Means for variables at both time points and correlations between the standardized differences scores (T2–T1) are displayed in **Table 1**. The means varied between 1.625 (T1 Negative Enjoyment) and 10.061 (T1 Self-determination). With regard to correlations, they ranged from 0.614 (Self-determination with Positive Enjoyment) to −0.007 (Teacher Controlling Style with Positive Enjoyment). All correlations were significant (p < 0.05).

### Latent Profile Analysis Identification of Latent Groups

We compared models between one and four clusters. As can be seen in **Table 2**, the solution with lower values for AIC and SSA-BIC is the 3 clusters option, which is significantly better than the four clusters option. Based on these results, the three-factor model best represented the dataset.

### Description of Latent Groups

**Figure 1** and **Table 3** present the cluster solution. When the changes between the collection 1 and the collection 2 are compared, it is observed that for the first cluster, comprised of 28 students (14%), there were decreases in self-determination and need-fulfillment (low). The second cluster, comprised of 46 (63%), is characterized by no changes in self-determination and need-fulfillment (average). And the third cluster, comprised of 126 (23%), is characterized by increases in self-determination, autonomy, competence, but not in relatedness (high).

### Comparison of Autonomy Support and Positive Enjoyment Across Latent Groups

Subjects in the low cluster had the lowest Autonomy Support (M = −1.238) and Positive Enjoyment (M = −1.261) variation,

M, Mean; SE, Standard error.

subjects in the average cluster showed more Autonomy Support (M = 0.0881) and Positive Enjoyment (M = −0.024) than subjects in the low cluster, while subjects in the high cluster were characterized by more Positive Enjoyment (M = 0.786), but not more Autonomy Support (M = 0.493). Results are displayed in **Table 4**.

### Comparison of Control Support and Negative Enjoyment Across Latent Groups

As can be seen in **Table 5**, subjects in the low cluster had the highest Teacher Controlling style (M = 0.773) and Negative Enjoyment variation (M = 1.351), subjects in the average cluster showed less Teacher Controlling style (M = −0.092) and Negative Enjoyment (M = 0.009) than subjects in the low cluster, while subjects in the high cluster were characterized by less Negative Enjoyment (M = −0.800), but not less Teacher Controlling style (M = −0.214).

TABLE 4 | Mean and standard error for autonomy support and positive enjoyment in each cluster.

M, Mean; SE, Standard error. Numbers in superscript refers to groups significantly different (Confidence level = 95%).

TABLE 5 | Mean and standard for error teacher controlling style and negative enjoyment in each cluster.

M, Mean; SE, Standard error. Numbers in superscript refers to groups significantly different (Confidence level = 95%).

### Comparison of Gender Across Latent Groups

With regard to gender differences, all clusters were significantly different. The low and high cluster were characterized by more females, while more males were observed in the average cluster (**Table 6**).

### DISCUSSION

The aim of this longitudinal study was twofold: (1) explore different clusters based on autonomy, competence, relatedness and self-determined motivation changes over a 22-months period, and (2) test how these clusters differ on enjoyment and on teacher style (autonomy/control support). The hypothesis presented seeks for the existence of patterns of change and for the association of these patterns with an outcome and a determinant.

### Patterns on Students' Self-Determined Motivation

Three patterns were identified: decreases in all psychological needs and self-determination; no changes, and increases in self-determination, competence, autonomy but no changes in relatedness. The downward pattern was associated with more Negative Enjoyment and a Controlling Style, while the upward pattern, with more Positive Enjoyment and Autonomy Support. In a similar study, Kimiecik and Horn (2017) observed three different clusters: low, moderate and high, being that the students in the high cluster reported the highest levels of moderate-to-vigorous physical activity and most healthy eating patterns. Kimiecik and Horn's (2017) findings are significant to understand adolescent self-determined motivation for the participation in health-promoting behavior.

Importantly, those three patterns differed on the outcome (positive and negative enjoyment), and the determinant (autonomy/control support). In a cross-sectional study Wang et al. (2016), observed that students in clusters characterized by the highest levels of autonomous regulation were associated with teachers with a more autonomy-supportive style. Wang et al. (2016) as well as the present study support the SDT since different subtypes of self-determined motivation differentially predict students' engagement and experience of physical education. It is possible to affirm that it is not only how motivated students are, but in what ways they are motivated, that explain their persistence in physical activities.

The group with low self-determination also showed low fulfillment of the basic psychological needs of autonomy, competence and relatedness with moderate self-determination

TABLE 6 | Proportion of males and females in each cluster.

exhibited moderate fulfillment of these needs. Changes in satisfaction related to basic psychological needs were associated with self-determined motivation, and it is important to examine competence, autonomy and the relationship with others separately, in order to obtain a more complete understanding of the role played by the teaching style, fulfillment of psychological needs and enjoyment of physical activity in adolescents.

Earlier studies gathered evidence of the positive influence of teaching style on autonomy on self-determined motivation, basic psychological needs and satisfaction with physical activity (Lim and Wang, 2009; Cheon et al., 2012; Ntoumanis et al., 2018), since these students are also prone to participate in the proposed tasks, more satisfied with their life and more committed to their activities, in addition to having greater perception of competence (Reeve et al., 2004).

### Patterns on Students' Enjoyment of Physical Activity

It is important to underscore the relationship between enjoyment and physical activities, given that the sense of fulfillment and pleasure in performing an activity promotes adherence and regularity (Vlachopoulos and Michailidou, 2006). Enjoying physical education classes may increase student commitment to overall physical activities, which increases their desire to remain active outside the school environment, and later become active adults concerned about their health (Ntoumanis, 2005).

There is also an intergroup difference between the sexes. In this study the boys obtained moderate results, while the girls showed both extremes (low and high self-determination). Most of the studies that analyze self-determined motivation to attend physical education classes, considering the students' sex, found that boys are more intrinsically motivated than girls (Cairney et al., 2012). Moreno-Murcia et al. (2006) observed that the attitudes of female adolescents or adults regarding sport and physical activity follow a more esthetic pattern, while boys display attitudes in relation to team and competitive sports. These differences may be related to attitudes labeled according to sex, stimulating a more competitive attitude in boys, which could cause a feeling of lower competence and less fun during activities.

Educational programs that stimulate the development of self-determined motivation may lead students to habitually engage in physical activities so that they are less likely to discontinue them after the school year ends. Thus, enjoyment in physical education classes results from a more self-determined behavior and fulfillment of the basic psychological needs for competence, autonomy and relationships with others, with an influence of interpersonal teaching style. The present findings support the use of educational strategies that favor autonomy and influence self-determined motivation, thereby promoting commitment to physical activity. In this respect, a number of aspects are important in improving self-determined motivation in physical education classes, such as varying activities, transmitting the feeling of responsibility, enabling student decision making, and recognizing efforts and personal improvement (González-Cutre et al., 2011). As such, it might be interesting for physical education teachers to concentrate on activities that students deem important, interesting and useful as well as stimulate feelings of competence, thereby promoting the perception of success during activities (Gu and Solmon, 2016).

### Limitations

A study limitation was the fact that the authors used the self-determination index, as observed in earlier studies. However, recent research indicates that using a continuum as a general index may dilute the richness of the findings obtained, given that the model can be tested by considering each type of self-determined motivation or separating it into two large categories (autonomous and controlled). Nevertheless, the information collected may help future studies, where it will be important to design interventions with teachers in order to enhance experiences in physical education classes, thereby improving adherence to physical activities.

### CONCLUSION

In conclusion, more self-determined motivation is related to greater teacher support for autonomy, greater fulfillment of basic psychological needs and increased enjoyment with physical activity. It is important to underscore that the group characterized by high self-determination exhibited high competence values. As such, it is recommended that physical education teachers use a style that supports autonomy, applying strategies that improve student self-determined motivation as well as feelings of pleasure and satisfaction with physical education.

### AUTHOR CONTRIBUTIONS

JM-M conceptualized the study, supervised and managed the data, drafted and revised the manuscript, approved the manuscript to be published, and agreed to be accountable for all aspects of the work in this manuscript. GF and EB analyzed and managed the data, drafted and revised the manuscript, approved the manuscript to be published, and agreed to be accountable for all aspects of the work in this manuscript. JL and authors conceptualized the study, analyzed and wrote the results, approved the manuscript to be published, and agreed to be accountable for all aspects of the work in this manuscript. RNJ supervised the data, drafted and revised the manuscript, approved the manuscript to be published, and agreed to be accountable for all aspects of the work in this manuscript.

### ACKNOWLEDGMENTS

We gratefully acknowledge Mayara de Souza Arceno Mees and Igor da Cruz, for assistance in data collection.

### REFERENCES

fpsyg-09-02721 January 5, 2019 Time: 19:7 # 8

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Fin, Moreno-Murcia, León, Baretta and Nodari Júnior. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Improvement of Inter-Professional Collaborative Work Abilities in Mexican Medical and Nursing Students: A Longitudinal Study

Guillermo J. Tuirán-Gutiérrez1 , Montserrat San-Martín2 , Roberto Delgado-Bolton3,4 , Blanca Bartolomé3,4 and Luis Vivanco3,4 \*

1 School of Healthcare Sciences, Autonomous University of Coahuila, Piedras Negras, Mexico, 2 Faculty of Social Sciences of Melilla, University of Granada, Melilla, Spain, 3 Center for Biomedical Research of La Rioja (CIBIR), Logroño, Spain, 4 National Centre of Documentation on Bioethics, Logroño, Spain

Background: Inter-professional and interpersonal relationships in collaborative work environments can prove to be critical elements in healthcare practice. When implementers fail to understand the importance of a collaborative perspective, this can lead to communication problems which ultimately harm the users.

Objectives: To improve the inter-professional collaborative work skills of Mexican students in their first year of medical and nursing degrees through the use of a training program geared toward development of interpersonal skills and interdisciplinary work.

#### Edited by:

Camilo Hurtado-Parrado, Troy University, United States

#### Reviewed by:

Monica Novoa, Fundación Universitaria Konrad Lorenz, Colombia Riccardo Sartori, University of Verona, Italy

> \*Correspondence: Luis Vivanco<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Psychology

Received: 19 April 2018 Accepted: 03 January 2019 Published: 15 January 2019

#### Citation:

Tuirán-Gutiérrez GJ, San-Martín M, Delgado-Bolton R, Bartolomé B and Vivanco L (2019) Improvement of Inter-Professional Collaborative Work Abilities in Mexican Medical and Nursing Students: A Longitudinal Study. Front. Psychol. 10:5. doi: 10.3389/fpsyg.2019.00005 Methods: The sample was composed of 162 students (62 males and 99 females) from the School of Healthcare Sciences of the Autonomous University of Coahuila, Mexico. The main measures used were the Jefferson Scale of Empathy (JSE); the Jefferson Scale of Attitudes toward Inter-Professional Collaborative Work between Medical and Nursing Professionals (JSAPNC); and the Jefferson Scale of Lifelong Learning (JeffSPLL). The entire sample was divided into two groups (experimental and control groups). Both groups attended an extra-curricular program using a coaching methodology. In the first case the topic focused on attitudes toward inter-professional collaborative work. In the second case, the program focused on addiction. Both programs ran for 4 months. Psychometric instruments were applied at the beginning and at the end of both programs. After analyzing the reliability of the instruments, an ANOVA test was performed.

Results: The control group of medical students showed a deterioration in the development of collaborative work skills (p < 0.01), whereas in the experimental group this deterioration was not present. In the experimental group of nursing students, a significant increase in the development of collaborative work skills (p < 0.05) was observed. The differences were clearly due to the professional area of study (p < 0.001).

Conclusion: There are differences in collaborative work skill development among different professional areas. These differences can be reduced through the implementation of a program aimed at developing collaborative work and interpersonal skills in the early stages of training.

Keywords: professionalism, empathy, lifelong learning, inter-professional collaborative work, medical students, nursing students, professional roles

## INTRODUCTION

fpsyg-10-00005 January 12, 2019 Time: 17:5 # 2

In physicians and healthcare professionals, professionalism refers to the set of skills and values that characterizes the essence of humanism in professional work (Vivanco and Delgado-Bolton, 2015). This concept comprises an articulated body of professional traits and skills that constitute the essence of their professional work regardless of their geographical, social, or cultural settings. For assessment purposes, a definition from the medical research field suggests that medical professionalism is achieved by mastering three essential areas: clinical skills, communication skills, and an appropriate understanding of the ethical and legal framework of professional behavior (Stern, 2006). Communication skills are not limited to patients and their families, but also relate to other healthcare professionals who are part of the inter-professional team which works at the healthcare institutions. According to the World Health Organization (2010), this inter-professional collaboration, also called teamwork, is described as the ability of "multiple healthcare workers from different professional backgrounds to provide comprehensive services by working with patients, their families, their professional careers, and the community, in order to deliver the highest quality of healthcare across settings." Benefits of this teamwork have been described in recent publications in healthcare institutions (Pedrazza et al., 2017). Some of those benefits are associated with the improvement of certain professional competences that are important in interdisciplinary teams0 performance, such as communication, innovation, and creativity skills or with decision-making abilities (Ceschi et al., 2014). In fact, according to Veloski and Hojat (2006), teamwork, together with empathy and lifelong learning, are the three essential components of professionalism in medical and healthcare sciences.

Despite important social, political, and economical differences among Latin American countries, there are no big differences in terms of medical and nursing educational curricula (Andrade, 1978) nor in the public health policies (Atun et al., 2015) among them. This has been evidenced by findings among Latin American physicians and nurses regarding professionalism, measured first from the scope of empathy (Alcorta-Garza et al., 2016), and most recently from the assessment of attitudes toward inter-professional collaborative work and lifelong learning (San-Martín et al., 2017a). With regard to inter-professional relationships, and specifically to the role that professionals play, two studies in Mexican healthcare institutions were the first to report the prevalence of a "hierarchical" model in physiciannurse work relationships in Latin America (Hojat et al., 2001, 2003). Fifteen years later, two international studies, one carried out among healthcare professionals who worked at public institutions in four Latin American countries (San-Martín et al., 2017a), and the other regarding Latin American medical students enrolled in training programs at Spanish teaching hospitals (San-Martín et al., 2017b), were published. Both studies not only confirmed the pervasiveness of a hierarchical model in the physician-nurse work relationships of new generations of Latin American healthcare professionals, but also remarked on the important role that some cultural stereotypes, mainly associated with professional roles, play in the development of three critical aspects of professionalism: empathy, inter-professional collaborative work, and lifelong learning.

Based on these findings, this study was designed in a Mexican university with two main objectives. The first was to confirm whether differences related to professional roles, previously reported in Mexico between physicians and nurses, are already evident in first-year medical and nursing students at two time-points, during the first month of their studies and upon completion of the first academic semester (4 months). According to this first objective, two research hypotheses were established. The first hypothesis was that first-year Mexican students attending medical and nursing schools develop different attitudes (that can be psychometrically measured) very early on, regarding the dynamics of physician-nurse collaborative work, due to the influence that culture and social stereotypes play on professional roles. The second hypothesis was that the differences observed in the development of attitudes toward physiciannurse collaborative work among students are not related to sex differences.

The second objective of this study was to prove the efficacy of a training program, based on coaching methodology, to improve attitudes toward physician-nurse collaborative work in first-year students enrolled in Mexican medical and nursing schools. A closer look at this second objective led to the creation of two more research hypotheses. The third one was that first-year students who attended a training program on inter-professional collaborative work improved their interprofessional collaboration abilities in physician-nurse teamwork. The fourth hypothesis was that a greater development of these inter-professional work abilities had a positive effect on the development of empathy and lifelong learning, given that these three areas comprise the core elements of professionalism.

### MATERIALS AND METHODS

### Participants

Medical and nursing students from of the Autonomous University of Coahuila, in Piedras Negras, Mexico participated in the study. Two first-year undergraduate student cohorts, one of medical students from the School of Medicine and another of nursing students from the School of Health Sciences, were invited to participate in the study at the beginning of their first academic semester. Of these, 183 students (77 male, 106 female), representing 59% of both cohorts, agreed to participate.

### Design and Procedure

In the first week of class, two cohorts of first-year students (one from the medical school and the other from the nursing school, both from the same university) attended a workshop led by an external researcher. In the workshop, students were informed that this study was performed for the purpose of measuring the development of some competencies associated with professionalism in healthcare workers. Participants were randomly divided into two groups, maintaining a similar distribution according to their degree coursework and

sex, in accordance with an experimental design previously developed. One group (experimental group) received a 4 month training program on inter-professional collaborative work abilities, whereas the other one (control group) received a 4-month training program on drug addiction prevention. A multiple-choice questionnaire including specific psychometric measurements (see below) was administered before and after the training programs. The questionnaires were on paper and were returned in sealed envelopes. The students' participation was anonymous, and this anonymity was maintained through the use of pseudonyms. There was no potential harm to participants. Finally, those participants who completed both groups of questionnaires were included in the statistical analysis. A summary of the study design is shown in **Figure 1**.

An independent ethical committee for clinical research (Comité Ético de Investigación Clínica de La Rioja) approved the study design (Ref. CEICLAR PI 199) in Spain. The study was carried out in accordance with recommendations and authorization of the participating institution's administration (Autonomous University of Coahuila, Piedras Negras). In accordance with the Declaration of Helsinki, all students who agreed to participate in this study brought a written informed consent.

### Main Measures

The Jefferson Scale of Attitudes toward Physician-Nurse Collaboration (JSAPNC) was used. The JSAPNC was developed in response to the need for a valid instrument to measure an important aspect of professionalism, namely teamwork and inter-professional collaboration between physicians and nurses. According to the authors of the JSAPNC (Hojat et al., 1999), physician-nurse collaboration is defined as an ability of nurses and physicians to work together cooperatively, sharing responsibilities for solving problems and making decisions to formulate and carry out plans for patient care. The JSAPNC is composed of 15 Likert scale items that are scored from 1 (strongly disagree) to 4 (strongly agree). This instrument has demonstrated a high reliability and validity among medical and nursing students in different cultural contexts (Ward et al., 2008; Hojat et al., 2015; Zakerimoghadam et al., 2015).

Two versions of the Jefferson Scale of Empathy (JSE) were used to measure students' orientation toward empathetic relationships with patients (Hojat, 2016): the medical student version (JSE-S), for medical students and the healthcare student version (JSE-HPS) for nursing students. The JSE has 20 Likert scale-type items worth seven points, ranging from 1 (strongly disagree) to 7 (strongly agree). The difference between the two versions of the JSE used is in the rewording of those items where the terms "medicine" or "physician" appeared, in order to make it more applicable to students from other healthcare areas. A sample item of the JSE-S is: "Physicians should try to stand in their patient's shoes when providing care to them," while the same item in the JSE-HPS is reworded as follows: "Health care providers should try to stand in their patients' shoes when providing care to them." A higher score means that the student has a greater orientation or behavioral tendency toward empathic engagement in patient care (Hojat, 2016).

Two versions of the Jefferson Scale of Physician Lifelong Learning (JeffSPLL) were used to measure attitudes toward lifelong learning: the medical student version (JeffSPLL-MS), for medical students (Wetzel et al., 2010); and the healthcare profession student version (JeffSPLL-HPS) for nursing students (Novak et al., 2014). The JeffSPLL was originally designed to measure the development of skills related to information gathering, the use of learning opportunities, and self-motivation (Hojat et al., 2009). The JeffSPLL has 14 Likert scale-type items that are scored from 1 (strongly disagree) to 4 (strongly agree). Similarly to the two versions of the empathy scale, the differences between the two versions of the JeffSPLL are in the rewording of those items where the term "medicine" is used, in order to make it more applicable to students from other healthcare areas. The JeffSPLL tool has demonstrated a high reliability and validity in both student groups in different cultural contexts (Muliira et al., 2012; San-Martín et al., 2016).

Information regarding degree coursework, sex, and age were collected through a complementary form.

### Training Programs

Based on coaching methodology, two extra-curricular training programs were specially developed for this study: (i) for the experimental group, a program on inter-professional collaborative work; and (ii) for the control group, a program on drug addiction prevention. Both programs included similar methodologies based on theoretical classes and practical sessions, the same number of lessons (36 lecture hours) distributed over 18 weekly workshops of 2 h each, and were offered in parallel during the first academic semester (4 months). A detailed description of the contents of each program is reported in **Appendix 1**.

## Statistical Analysis

The reliability of the instruments used was measured by Cronbach's alpha coefficient. This assessment was performed twice: before and after the training programs were given.

Once the normality of the inter-professional collaborative work scores was studied, a variance analysis (Three-way ANOVA) based on the following variables was performed, both prior to and at the end of the study: study group, sex, degree coursework, and the interaction between sex and degree coursework. Furthermore, a comparative analysis using Wilcoxon tests, by study groups (experimental vs. control) and by degree coursework (medicine vs. nursing), before and after the study was performed. In order to determine possible associations among inter-professional collaborative work abilities, empathy, and lifelong learning, a correlation analysis using Spearman's coefficient (ρ) among the above-mentioned variables was also performed at the beginning and at the end of the study. Finally, a multiple regression analysis for "study group by degree coursework" variables was performed in order to determine whether the development of inter-professional collaborative work abilities at the end of the course has a different effect on the

other two measured components of professionalism: empathy and lifelong learning.

Data processing was carried out with R software (R Core Team, 2017), version 3.4.1 for Windows, and included the use of nortest (Gross and Ligges, 2015) and multilevel (Bliese, 2016) statistical packages.

### RESULTS

### Participants

From the entire sample of 183 students, 162 students (62 males, 99 females, and one gender not stated) responded to both questionnaires. This sample was used for the statistical analysis and represented 52% of the entire population of first-year students enrolled in medical and nursing programs offered by the School of Medicine and by the School of Healthcare Sciences of the Autonomous University of Coahuila, in Piedras Negras, Mexico.

According to their degree coursework, 84 participants were first-year medical students (45 males, 38 females, and two genders not declared), and 89 were first-year nursing students (26 males and 63 females). The mean age of the entire sample was 19 years, ranging from 18 to 35 years (SD = 2).

## Reliability and Descriptive Statistics

The range observed for Cronbach's alpha coefficients obtained for the five instruments used in this study was between 0.71 and 0.80. The test-retest reliability over a 4-month period yielded values ranging between 0.60 and 0.83. The score distribution summary, and reliability for all instruments used are reported in **Table 1**.

### Measurement of Attitudes Toward Physician-Nurse Collaboration in First-Year Medical and Nursing Students

The first hypothesis which looked at the influential role that culture plays in students' attitudes toward inter-professional work

TABLE 1 | Descriptive analysis, reliability, and reliability of the Jefferson Scales of physician-nurse collaboration, empathy, and lifelong learning.

JSAPNC, Jefferson Scale of Attitudes toward Physician-Nurse Collaboration; JSE-S, Jefferson Scale of Empathy- Medical Student version; JSE-HPS, Jefferson Scale of Empathy-Health Professions Student version; JeffSPLL-MS, Jefferson Scale of Physician Lifelong Learning-Medical Student version; JeffSPLL-HPS, Jefferson Scale of Physician Lifelong Learning-Health Professions Student version; n, sample size; M, mean; SD, standard deviation.

showed no differences in JSAPNC of newly admitted students when compared by degree coursework (p = 0.06). However, an inverse association between JSAPNC and age was confirmed (ρ = −0.17; p = 0.03), but only in newly admitted students. The second time that the JSAPNC was measured, just after the study was finished, this association disappeared (ρ = +0.003; p = 0.97). On the other hand, the second measurement of the JSAPNC showed new differences according to study group (p = 0.0005), and degree coursework (p = 0.00004), as shown in **Table 2**.

Regarding the second research hypothesis, no differences were found when the JSAPNC score was compared by gender, both at beginning (p = 0.16) and at the end (p = 0.35) of the study. A similar situation was also observed for the interaction between sexes and degree coursework, both at beginning (p = 0.805) and at the end (p = 0.841).

### Enhancement of Attitudes Toward Physician-Nurse Collaboration in First-Year Medical and Nursing Students

The third research hypothesis related to the positive effect of a training program on inter-professional collaborative work abilities. A three-way ANOVA, at the beginning and at the end, revealed that students from the experimental group achieved higher scores on the JSAPNC compared to those who participated in the control group, and this difference was statistically significant (p = 0.0005), as shown in **Table 2**. According to degree coursework, the comparative analysis by study groups showed that JSAPNC scores of nursing students from experimental group increased over time (p = 0.02), while JSANPC scores of medical students of this group remained stable (p = 0.57). However, in the control group, JSPANC scores of medical students showed a significant deterioration over the time (p = 0.006), while nursing students of this group remained stable (p = 0.16). The summary of these findings is reported in **Table 3**.

The fourth research hypothesis related to the role that interprofessional abilities play in the development of the other two measured elements of professionalism. Correlation analysis of the entire sample confirmed a positive relationship between empathy and inter-professional collaboration, both at the beginning (ρ = +0.27; p = 0.0008) and at the end (ρ = +0.41; p < 0.0001) of the study. Although no association was observed between lifelong learning and teamwork in newly admitted students (ρ = +0.11; p = 0.17), a significant association between these two elements appeared at the end of the study (ρ = +0.19; p = 0.02). At this point, statistical differences were also observed between interprofessional collaborative work and lifelong learning according to "study group by degree coursework" (p = 0.03). Those differences are shown in **Figure 2**.

### DISCUSSION

The scales showed adequate psychometric properties with Cronbach's alpha coefficients higher than the international recommendation of 0.70 set by the American Educational Research Association. Results observed in the present study support the use of the Spanish versions of the above-mentioned scales as consistent measures of empathy, teamwork, and lifelong learning for medical and nursing students. These findings are also similar to those previously reported for empathy in medical

TABLE 2 | Descriptive statistics and analysis of variance (Three-way ANOVA) of the scores of the inter-professional collaborative work by study group, gender, and degree coursework (n = 162).

n, sample size; M, mean; SD, standard deviation; F, F-value; η 2 , Eta-squared; p, p-value.

JSAPNC, Jefferson Scale of Attitudes toward Physician-Nurse Collaboration; n, sample size; M, mean; SD, standard deviation; p, p-value.

and nursing students (Hojat, 2016), and slightly lower than those reported for teamwork and lifelong learning (Hojat et al., 1999; Wetzel et al., 2010; Novak et al., 2014).

Regarding the first objective, teamwork, and cultural influence, the inverse association found between teamwork and age in newly admitted students suggests that students who start their degree coursework at a later age have a higher risk of presenting certain prejudgments related to professional roles and inter-professional work than in those who begin at a younger age. The most plausible explanation for these findings is the negative effect of dominant cultural stereotypes related to professional roles, a finding that is in accordance with research evidence reported in previous studies of Mexican (Hojat et al., 2001, 2003) and other Latin American healthcare professionals (San-Martín et al., 2017a,b). The findings of this study regarding the increasing difference in the development of this ability between medical and nursing students over time, or the deterioration in the development of this ability in the control group provide new evidence supporting the negative influence of the "hierarchical" dominant model in physician-nurse work relationships. The findings reported in this study clearly confirm that differences reported for inter-professional collaborative work among students are due to their degree coursework and not to their gender differences. This is also consistent with other studies where those differences were directly associated with professional roles (San-Martín et al., 2017a).

On the other hand, the study demonstrates that, similar to previously reported findings for empathy (Cunico et al., 2012; Hojat, 2016), collaborative inter-professional work can be enhanced and sustained in medical and nursing students by targeted educational programs. The significant increase in JSAPNC scores by the nursing students in the experimental group who were attending the training program, or the stability in the JSANPC scores of the medical students who were participating in the experimental group, evidence an antagonist tendency in comparison with those observed in medical and nursing students who were enrolled in the control group. These findings resonate with those reported in a recent study where there was a significant overlap between empathy, teamwork, and integrative patient care. The authors concluded that an improvement in inter-professional work skills is not only possible but can also positively influence the improvement of other components of medical professionalism, such as empathy (Hojat et al., 2015). In accordance with this last issue, the findings of this study confirm the positive influence that improvement of teamwork has on the other two components of professionalism. Along the same lines, a strong relationship among empathy, teamwork,

and lifelong learning are experimentally demonstrated. These findings coincide with others reported in previous studies with medical students (San-Martín et al., 2016) and healthcare professionals (San-Martín et al., 2017a,b; Soler-González et al., 2017). In all these studies, new research evidence supports a multi-score measurement of medical professionalism, initially proposed by Veloski and Hojat (2006) in clinical settings with a cross-cultural scope. This study supplies new evidence in an undergraduate population. Additionally, this study provides new evidence supporting the importance of improving interprofessional skills in the development of attitudes toward lifelong learning in nursing students. This issue is especially remarkable given that the development of lifelong learning skills as a professional competence is an important gap in nurses who are working under the influence of dominant "hierarchical" work models, as has been reported in recent studies in healthcare institutions located in developing countries in Africa (Muliira et al., 2012) and Latin America (San-Martín et al., 2017a).

### AUTHOR CONTRIBUTIONS

LV was in charge of the study's overall design. GT-G oversaw development of the training programs and carried out the study in the participating institutions in Mexico. LV and MS-M statistically processed the data. LV and RD-B prepared the draft manuscript. All authors contributed to the present work,

### REFERENCES

participated in the interpretation and processing of results, and reviewed and approved the final manuscript.

### FUNDING

This study was supported by the Rioja Salud Foundation (FRS), Spain, and by a grant from the Instituto de Salud Carlos III (Pi16/01934), co-financed by Operational Program of the European Regional Development Fund (FEDER-La Rioja) in Spain (6FRS-ABC-012).

### ACKNOWLEDGMENTS

We would like to acknowledge the contribution of the Autonomous University of Coahuila and the Ibero-American University Foundation (FUNIBER), both in Mexico. Also, Christy Esmahan for her contribution to the English revision of this manuscript. Finally, we would like to acknowledge all the students who participated in this study.

### SUPPLEMENTARY MATERIAL

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyg. 2019.00005/full#supplementary-material

nurses on the total and factor scores of the Jefferson scale of attitudes toward physician-nurse collaborative relationships. Int. J. Nurs. Stud. 40, 427–435. doi: 10.1016/S0020-7489(02)00108-6

professionalism in medical students during the clinical rotations in Peru. Int. J. Ethics Educ. 1, 163–172. doi: 10.1007/s40889-016-0017-8

a psychometric evaluation. Acad. Med. 85(10 Suppl.), S41–S44. doi: 10.1097/ ACM.0b013e3181ed1ae9

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Tuirán-Gutiérrez, San-Martín, Delgado-Bolton, Bartolomé and Vivanco. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Avian Emotions: Comparative Perspectives on Fear and Frustration

*Mauricio R. Papini1 \*, Julio C. Penagos-Corzo2 \* and Andrés M. Pérez-Acosta3 \**

*1 Texas Christian University, Fort Worth, TX, United States, 2 Universidad de las Américas Puebla, Cholula, Mexico, 3 Universidad del Rosario, Bogotá, Colombia*

Emotions are complex reactions that allow individuals to cope with significant positive and negative events. Research on emotion was pioneered by Darwin's work on emotional expressions in humans and animals. But Darwin was concerned mainly with facial and bodily expressions of significance for humans, citing mainly examples from mammals (e.g., apes, dogs, and cats). In birds, emotional expressions are less evident for a human observer, so a different approach is needed. Understanding avian emotions will provide key evolutionary information on the evolution of related behaviors and brain circuitry. Birds and mammals are thought to have evolved from different groups of Mesozoic reptiles, theropod dinosaurs and therapsids, respectively, and therefore, their common ancestor is likely to be a basal reptile living about 300 million years ago, during the Carboniferous or Permian period. Yet, birds and mammals exhibit extensive convergence in terms of relative brain size, high levels of activity, sleep/wakefulness cycles, endothermy, and social behavior, among others. This article focuses on two basic emotions with negative valence: fear and frustration. Fear is related to the anticipation of dangerous or threatening stimuli (e.g., predators or aggressive conspecifics). Frustration is related to unexpected reward omissions or devaluations (e.g., loss of food or sexual resources). These results have implications for an understanding of the conditions that promote fear and frustration and for the evolution of supporting brain circuitry.

Keywords: emotion, fear, frustration, birds, aggression, response suppression, conflict, comparative psychology

### INTRODUCTION

In one of his epistles to Lucius, the Roman philosopher Seneca wrote that "No animal, when it enters upon life, is free from the fear of death" (Seneca, Epistle CXXI, 19, trans., Seneca, 1925). This conclusion is consistent with the general observation that most animals seem to have evolved withdrawal, escape, and avoidance behaviors from threatening situations, and seek and approach situations that promote their well-being and fulfill their reproductive potential. For example, in the mollusk *Tritonia diomedea*, tactual stimulation by a starfish (a natural predator) triggers a series of dorsoventral contractions that allow the animal to escape to a distant location (Wyeth and Willows, 2006). In some cases, however, animals seem to be willing to die, especially in defense of their colony. For example, when the colony is attacked by a predator, postreproductive females of the aphid *Quadrartus yoshinomiyai* attach themselves to the predator and secrete a sticky substance that glues the intruder and also themselves to the plant. This kills the predator, but also results in the aphid's death (Uematsu et al., 2010).

#### *Edited by:*

*Camilo Hurtado-Parrado, Fundación Universitaria Konrad Lorenz, Colombia*

#### *Reviewed by:*

*Federico Sanabria, Arizona State University, United States Robert Lickliter, Florida International University, United States*

#### *\*Correspondence:*

*Mauricio R. Papini<EMAIL_ADDRESS>Julio C. Penagos-Corzo<EMAIL_ADDRESS>Andrés M. Pérez-Acosta<EMAIL_ADDRESS>
#### *Specialty section:*

*This article was submitted to Emotion Science, a section of the journal Frontiers in Psychology*

*Received: 29 October 2018 Accepted: 17 December 2018 Published: 17 January 2019*

#### *Citation:*

*Papini MR, Penagos-Corzo JC and Pérez-Acosta AM (2019) Avian Emotions: Comparative Perspectives on Fear and Frustration. Front. Psychol. 9:2707. doi: 10.3389/fpsyg.2018.02707*

**117**

Are these behaviors accompanied by a subjective emotional experience?

Educated people may find it difficult to believe that organisms considered simple, such as a mollusk and an aphid, have the neural sophistication sufficient to produce emotional responses. But what about animals with more complex nervous systems, such as mammals and birds? Comparative psychologists have struggled with the problem of emotion because, as many other important concepts, they cannot be directly observed or measured. Darwin (1871/1965) took a different approach by emphasizing emotional *expressions*, that is, the facial and bodily movements that accompany situations that seemed akin to those that would generate emotional responses in humans. Darwin cited examples mainly drawn from mammals (e.g., apes, dogs, and cats). For example, "animals which live in society often call to each other when separated, and evidently feel much joy at meeting; as we see with a horse, on the return of his companion, for whom he has been neighing" (Darwin, 1871/1965, pp. 84–85). In Darwin's book, descriptions of emotions in birds are reduced to a few examples of negative valence. Birds "ruffled their feathers when angry or frightened" (p. 97) and are "accustomed, when in danger, either to squat on the ground or to sit motionless on a branch, so as to escape detection" (p. 100). Whereas contemporary researchers generally reject such an anthropomorphic language (but see, Safina, 2015), most would still recognize Darwin's two descriptions of avian emotion given above as examples of agonistic behavior and fear-induced freezing. Curiously, Darwin provided no examples of avian emotions that could be related to frustration, disappointment, or loss, although the example with reunited horses also given above points to the effects of social separation in a mammal.

In defining these emotions, we continue to have limits similar to those encountered by Darwin in 1871. First, we still cannot study animal emotions directly because we have no access to an organism's subjective experience. Therefore, we must depend on animal behavior and physiology—emotional expression. Second, we have to use standards developed by those who study emotional behavior in mammals and apply them to other taxonomic groups. Third, the human emotional repertoire seems too broad to start a discussion of animal emotion. It seems more effective to center our attention in a few emotions that have clear protective function. In this review, we focus on two such emotions: fear and frustration.

Animals are said to be under the influence of fear if they "avoid when they can, flee if avoidance has not been successful, and defend themselves, usually aggressively, when flight is impossible or difficult" (Gray and McNaughton, 2000, p. 38). The avoidance of threat may involve response suppression (usually called "freezing"), which, as noted by Darwin, helps the organism to avoid detection. Frustration tends to evoke similar behaviors, but in response to the unexpected failure of rewards to occur. Thus, frustration "is a temporary state that results when a response is nonreinforced (or nonrewarded in more neutral language in the appetitive case) in the presence of a reward expectancy" (Amsel, 1992, p. 1). Amsel characterizes the behavioral consequences of frustration in terms of four descriptive concepts: invigoration, suppression, persistence, and regression. When either fear or frustration is combined with approach responses triggered by the same location or stimulus, they also contribute to anxiety. For example, fear of a location previously paired with pain (electric shock) does not induce anxiety, unless the organism must approach that location for some other reason (e.g., food or social proximity). Traditionally, this is characterized as an approach-avoidance conflict (Miller, 1944). Anxiety is dependent upon the conflict induced by an ambivalent goal.

This article reviews evidence from studies with birds based on situations designed to study fear and frustration, and the ensuing states of anxiety and conflict. But first, we ask what is a bird and what do we know about their evolutionary history?

Although birds and mammals have evolved from different reptilian lineages (theropod dinosaurs and therapsids, respectively; Benton, 2014), there are many phenotypic similarities that originated independently by convergent evolution (Kemp, 1988). Some characters common to birds and mammals that are not found in extant reptiles include relatively large brain size, similar general activity levels, cycles of sleep and wakefulness, endothermy, and complex patterns of reproductive behavior, among others. Convergent evolution in learning has been explicitly postulated for movement mimicry among psittacine birds and primates (Moore, 1992). Is it possible that evolutionary convergence extends to the emotions of fear and frustration?

The fossil record suggests that by the late Jurassic period, about 140–150 million years ago, there were several lineages of theropod dinosaurs with feathers. A specimen from China, *Anchiornis huxleyi*, dated 155 million years old, was a small theropod dinosaur covered with feathers, although not likely capable of flight (Hu et al., 2009). There is increasing evidence that feathers first evolved in dinosaurs for functions unrelated to flight, such as thermoregulation or display. Consistent with their use in communication displays, a study of melanosomes, molecules responsible for feather color in living birds, suggested that *Anchiornis* was brightly colored with gray, reddish-brown, white, and black feathers (Li et al., 2010). Although not considered a bird, *Anchiornis* and a few other fossil specimens, are classified in the same group with *Archaeopteryx lithographica*, a transition form with reptilian and avian characters, and all of them as the immediate ancestors of birds (Benton, 2014).

*Archaeopteryx* was not like any modern bird. It had teeth, a bony and long tail, and claws, all reptilian characters, and although the feathers had the asymmetric morphology typical of modern birds with flight capacity, whether it could take off from the ground up remains controversial (Voeten et al., 2018). Modern birds appeared in the fossil record during the Cenozoic era (Benton, 2014), but mitochondrial DNA studies suggest an origin during the late Cretaceous period, around 100 million years ago (Brown et al., 2008). *Vegavis iaai* is a fossil bird from the Late Cretaceous of Antarctica, dated 66–68 million years ago and considered to be related to modern birds (Clarke et al., 2005). This evidence suggests that the ancestors of modern birds survived the mass extinction event of the Cretaceous-Paleogene boundary, dated to about 65 million years ago, and then underwent an adaptive radiation during the early portion of the Cenozoic era that parallels that of the mammals.

Both birds and mammals show evidence of encephalization, that is, relatively large brain size, compared to reptiles (Striedter, 2005; Northcutt, 2011). *Archaeopteryx* also appears to have had a relatively large brain (Balanoff et al., 2013), although perhaps not as large as that of modern birds (Alonso et al., 2004). Encephalization in birds and mammals has proceeded independently from reptilian ancestors that, as far as we know, have a relative brain size that, on average, was about 10 times smaller (Northcutt, 2011). Avian encephalization may be related to a number of characters, such as endothermy, flight, cognition, and sociality. Daily energy demands that could sustain some of these functions may have triggered a positive feedback loop with brain size favoring the evolution of encephalization. It is clear that the organization of the telencephalon of birds and mammals has proceeded independently to a point that it is difficult to recognize homologies at a macroanatomical level. Equally clearly, however, there are substantial telencephalic homologies that can be appreciated at the level of afferentefferent connections, neurochemical systems, and gene-expression profiles (Montiel and Aboitiz, 2018).

Cognitive skills and social behavior have been connected to brain evolution and high levels of anatomical change in birds (Wyles et al., 1983; Iwaniuk and Wylie, 2017). For example, tool use is correlated with increased size of the avian (nidopallium) and primate (neocortex) brain (Mehlhorn et al., 2010; Lefebvre, 2013). A similar connection between brain and emotion is only beginning to be made (Panksepp, 2017). In part, the relative paucity of results is due to a large extent on the fact that behavioral data on emotion in birds are not clear. One goal of this review is to bring this issue to the foreground.

### AVIAN FEAR

Fear and related emotional states are induced in response to threat, whether actual or potential. The following review is organized into four areas of research involving threatening stimuli: response suppression, active escape/avoidance behavior, aggressive behavior, and conflict.

### Response Suppression

Birds exhibit freezing and corticosterone release in fear-inducing situations. In one study (de Haas et al., 2012), freezing behavior (lack of motion and vocalizations) was assessed in an open field in young chicks and the effects were measured at an adult age. Freezing levels positively correlated with adult corticosterone levels. Moreover, corticosterone increased in birds subjected to a test involving restriction of movement. Interestingly, the presence of a fearful bird increased corticosterone levels in other individuals within the group. Additional research on response suppression in birds relates to tonic immobility (TI). TI involves the suppression of the righting response, reduced vocalizations, and intermittent eye closure (Pusch et al., 2018). In a TI paradigm, the experimenter holds the hen by hand upside down for 15 s and then releases the animal. TI occurs if no movement is detected for 10 s (Hrabcakova et al., 2012; Pusch et al., 2018). According to Gallup (1973), TI is a reliable index of fear in birds. He found that a conditioned stimulus for shock is more effective than a shock to increase the duration of TI. A recent review concluded that TI remains a valid indicator of affective states in the face of threats (Fureix and Meagher, 2015). Quail strains selected for high and low TI show differences in fear responses, with the high TI strain showing more freezing and less exploration than the low TI strain (Jones et al., 1991). These differences were attenuated by exposure to environmental enrichment (colored geometric patterns painted on cards and pinned to the walls and a variety of colored objects).

The posterior arcopallium in birds (corresponding to portions of the mammalian amygdala; Jarvis et al., 2005; Hanics et al., 2017) seems to play an important role in the control of fear responses, as it does in mammals (Ressler and Maren, 2019). In Japanese quail, for example, lesions of the posterior arcopallium increase behavior indicative of fear, whereas lesions of the anterior arcopallium reduce fear responses (Saint-Dizier et al., 2009). Fear responses were tested in four tasks: TI, open-field test, hole-in-the-wall test, and novel-object test. In addition, there is evidence of lateralization of function in the arcopallium. In chickens, unilateral lesions of the right medial arcopallium have a greater effect on fear responses than equivalent lesions in the left hemisphere (Phillips and Youngren, 1986).

Neophobia is the rejection of novel objects. In birds, neophobia is usually measured in terms of the latency to approach, sit on, and finally take a food item from a novel feeder. Blackcapped chickadees (*Poecile atricapillus*) and European starlings (*Sturnus vulgaris*) exhibited longer latency to approach and consume food from a novel feeder than from a familiar feeder (Apfelbeck and Raess, 2008; Roth et al., 2010). Aversive conditioning also potentiates neophobia. In an experiment with chicks (Franchina and Dryer, 1989), novel visual (red water) or taste (vinager) simuli were paired with an injection of lithium chloride, a toxin that induces sickness, or a saline injection. In subsequent neophobia tests, animals were tested with either green water (visual novelty) or saline water (taste novelty). There was evidence of neophobia in both tests, but conditioning with the vinager led to more neophobia than conditioning with red water. Bobwhite quail (*Colinus virginianus*) can be easily conditioned to suppress consumption of food with a novel color after pairings with a toxin (Wilcoxon et al., 1971).

Passive avoidance research with birds has attracted more attention. In one procedure, beads of a specific color are coated with a substance that has an aversive taste to chickens (methyl anthranilate). In later tests, young chicks are given successive tests with beads of the same or different color and the amount of pecking is recorded. Chicks peck more at the novel color than at the color paired with the aversive substance, but the Papini et al. Avian Emotions

order in which the colors are presented and the time between successive tests determine the outcome of the test (Crowe and Hale, 2002).

Several aspects of passive avoidance learning in chicks are controlled by different parts of the telencephalon. For example, extensive lesions of the dorsal, lateral, and anterior hyperpallium (formerly hyperstriatum) impaired the retention and relearning of passive avoidance. Interestingly, whereas lesions of the dorsal hyperpallium disrupted acquisition of passive avoidance, anterior lesions caused no measurable deficits in passive avoidance (Benowitz, 1972). More recent research has focused on neurochemical and cellular effects of passive avoidance learning. The administration of dopamine agonists in one-day-old chicks alters memory consolidation in this passive avoidance task (Hale and Crowe, 2002). In the same species, this passive avoidance procedure resulted in an increase in cellular proliferation in the mesopallium ventrale (formerly hyperstriatum ventrale) and in the olfactory tubercle (Dermon et al., 2002). However, passive avoidance has also been reported to produce a decrease in hippocampal synaptic density in chickens (Nikolakopoulou et al., 2006). This may imply that passive avoidance is a stressful event leading to dendritic atrophy and thus a reduced spine density. Others reported the opposite result, namely, an increase in spine density in the right hemisphere of the hippocampus of male chicks trained in passive avoidance (Unal et al., 2002). The source of these differences is unclear. What is clear is that passive avoidance training in chickens gives rise to an aversive memory that has measurable neural correlates similar to those observed in mammals. For example, the density of hippocampal synaptic spines is also modified by Pavlovian fear conditioning in mice (Abate et al., 2018).

Circadian factors are also important in passive avoidance. Chickens that received a single pairing between a red bead and methyl anthranilate at 16:00 h had poor discrimination (i.e., they generalized responding) in red-versus-blue tests administered 24 h later. By contrast, chicks trained at 08:00 h and 12:00 h discriminated red from blue beads more accurately (Radford et al., 1981). Experiments with mammals show similar sensitivity to circadian factors (Barbachano et al., 2017). Caging conditions also affect passive avoidance learning. Using electric shocks as reinforcer, chickens caged in groups exhibited better learning than animals housed in individual cages (Brown, 1976). Group caging is known to also facilitate aversive learning in mammals (Penagos-Corzo et al., 2015).

### Active Escape/Avoidance Behavior

Active escape/avoidance behavior has been studied in a variety of situations. This section covers research on alarm signals, the flight initiation distance (FID), and active escape/avoidance learning.

The social learning of predator avoidance can be conceptualized in conditioning terms. The predator is the conditioned stimulus and the social alarm of a conspecific is the unconditioned stimulus. Accordingly, stimuli associated to social alarm are tagged as threatening. However, at least in some birds, learned social avoidance is temporarily more flexible than in laboratory paradigms of conditioning in which a stimulus predicts important biological events (Griffin, 2004, 2008). A natural predator not always generates automatic avoidance. For instance, there is evidence from several bird species that some animals are attracted to the predator and the experience derived from these encounters is used by conspecifics. Observing a predator model with a dead gull (*Larus argentatus*) increased the avoidance distance of gulls in a later encounter with the model (Kruuk, 1976). In many species, the encounter with a predator releases specific behaviors that conspecifics perceive and later use to tag the stimulus as threatening. Social alarm in birds is based on calls and also on the noise of the wings at take-off flight (Hingee and Magrath, 2009). Social alarm in birds may be conspecific or heterospecific (Dawson Pell et al., 2018). In the case of Japanese tits (*Parus minor*), birds that have learned to respond to alarm calls become more responsive to objects resembling the predator—a sign of a search image (Suzuki, 2018). It is also worth noting that birds show the acquisition of passive avoidance through observation. For example, chickens can learn about an aversive object observing the responses of another chicken (Johnston et al., 1998). Thus, chicks that observed through a wire mesh the aversive reaction of another chick when pecking at a bead coated with methyl anthranilate later avoided a similar bead. Avoidance learning by observation has also been reported in natural environments in great tits (Landová et al., 2017; Thorogood et al., 2018).

The FID has also been used in natural environments to assess escape/avoidance behavior (Møller, 2010). Increased fear in birds, as in mammals, is associated with adrenal cortex activation (Davis et al., 2008; Tilgar et al., 2010). Neuroendocrine responses involve hypothalamic–pituitary–adrenal (HPA) axis activation and corticosterone secretion (Cockrem, 2007). Even if there is evidence supporting the relationship of both FID and corticosterone with stress and fear, the relationship between these two variables is not always direct. For example, in a comparison between urban and rural birds, despite finding differences in the FID between these two populations, a relationship with corticosterone was not found (Rebolo-Ifrán et al., 2015).

In the traditional instrumental paradigm of active avoidance learning, a signal precedes an aversive stimulus, unless the animal makes a specific response during the signal, in which case, the signal is terminated and the aversive stimulus is prevented. There are two versions of active avoidance learning, one in which the animal moves to a different compartment when the warning signal is presented. This procedure is called two-way avoidance because either compartment is both safe and dangerous depending on the trial. In the second procedure, called one-way avoidance, the animal also moves to another compartment, but each compartment is always either safe or dangerous. Macphail (1968) provided evidence of both types of avoidance learning in pigeons, albeit with only a few animals and with sequential training starting with one-way avoidance and ending with two-way avoidance training. Pigeons also exhibited efficient acquisition of a treadle-pressing response to avoid shock in an unsignaled Sidman avoidance procedure (Smith and Keller, 1970). This procedure is analogous to active avoidance, except that no explicit signal is presented (shocks and response feedback provide signals). Instead, shocks occur at regular intervals and a response postpones the next shock for a period longer than the shock-shock interval. Using a visual + auditory compound and a treadle response, Foree and LoLordo (1975) reported that pigeons learned to respond to prevent punishment for pecking at a key for food. Moreover, post-training tests determined that the treadle response was controlled predominantly by the visual element of the compound. Interestingly, when pigeons pressed a treadle just to avoid shock or just to obtain food, then behavior was controlled predominantly by the auditory or visual component, respectively (Foree and LoLordo, 1973). Lesions of the arcopallium (previously archistriatum) impaired both active two-way avoidance and Sidman avoidance learning in pigeons (Dafters, 1975), just as they also impaired passive avoidance (see above). Two-way active avoidance performance is actually enhanced by section of the olfactory nerves, even though the warning signal was visual (Hutton et al., 1974). Opioid receptors in the olfactory tubercle (previously associated to the lobus paraolfactorius) have also been detected in chicks trained in the passive avoidance paradigm. Post-training infusions of mu and delta opioidreceptor antagonists in the olfactory tubercle impaired retention of the passive avoidance task, but kappa and opioid receptorlike receptor antagonists had no effect (Freeman and Young, 2000).

### Aggressive Behavior

Experiments with several mammalian species have shown that aggressive behavior can be elicited by painful stimuli, such as electric shocks (Ulrich, 1966). Pigeons, which readily peck at other pigeons (or at stuffed pigeons or mirror images) during periods of nonreinforcement (see below), do not seem to respond with aggressive behavior toward a stuffed conspecific when receiving shocks (Rashotte et al., 1974). Such discrepancy can be understood in terms of selective pressures related to foraging and predator avoidance. Pigeons compete with conspecifics for access to food, but they rarely engage in intraspecific aggressive behaviors. Consistent with this characterization, pigeons show aggressive responses to conspecifics in the context of foraging (e.g., appetitive extinction), but not in the context of defensive behavior (e.g., shock-induced pain).

Aggressive behavior occurs in chickens reared in animal production facilities. For a variety of reasons, chickens show an aggressive behavior described as feather pecking that results in bleeding and even cannibalism (Fossum et al., 2009). Feather pecking has been associated to fear responses in the target animal. Although birds show agitation after the initial attacks, this behavior is followed by periods of freezing. Physiological changes suggest that feather pecking induces pain (Gentle and Hunter, 1991). In a line of birds selected for low mortality from feather pecking, Kops et al. (2013) reported lower levels of noradrenaline activity in the arcopallium relative to a control line and, consistent with this finding, lower levels of fear in the manual restriction test (in this test, the animal is tilted on its right side while its legs are gently pulled; response movements are recorded).

### Conflict

Conflict situations usually involve a convergence of two contingencies of opposite hedonic value (Miller, 1944). This can be implemented in a Skinner box situation, where pigeons trained in a fixed-ratio or variable-ratio schedule for food reinforcement (which generates high rates of key pecking) are concurrently exposed to punishment with shock contingent on some key-pecking responses. A number of studies show that while punishment leads to behavioral suppression, there is also a tendency for recovery of response levels within the session (Dardano, 1970; Powell, 1970).

There is extensive psychopharmacological research on the effects of a variety of drugs on punished responding in birds. Chlordiazepoxide (CDP), an anxiolytic drug that binds to the benzodiazepine site in the GABAA receptor and facilitates the influx of chloride ions induced by GABA (Meyer and Quenzer, 2018), increases punished responding in rats, monkeys, and pigeons (Barrett and Gleeson, 1991). McMillan (1973) reported that CDP and diazepam (also a benzodiazepine anxiolytic) increase response rate under punished responding relative to matched unpunished behavior. Punished responding is also increased by bretazenil (a partial agonist of the benzodiazepine site of GABA receptors), which does not affect matched nonpunished responding, and the effect was eliminated by the benzodiazepine antagonist flumazenil (Witkin et al., 1997).

A variety of serotoninergic drugs have been used in the treatment of anxiety and depression in humans, but have little or no effects in established animal models of either disorders with nonhuman mammals. Yet, pigeons respond to these drugs much like human patients (Barrett et al., 1994). One example is buspirone (a serotonin-1A-receptor agonist with anxiolytic effects). Buspirone does not affect punished responding in rats (Howard and Pollard, 1990) or squirrel monkeys (Wettstein, 1988), and it also fails to eliminate consummatory negative contrast in rats (Flaherty et al., 1990). However, buspirone eliminates the suppressive effects of punishment in pigeons (Barrett and Witkin, 1991). The mechanism of action is benzodiazepine-independent (Barrett et al., 1986), but serotonindependent (Witkin et al., 1987; Gleeson et al., 1989).

The evidence reviewed in this section is consistent with the hypothesis that at least some avian species experience emotional states akin to mammalian fear. The behavioral, neurobiological, and psychopharmacological results reviewed above suggest interesting parallels between mammals and birds in emotional behavior.

### AVIAN FRUSTRATION

Following the lead from experiments with rodents, this review focuses on a variety of procedures involving some form of surprising reward omission (SRO; Papini and Dudley, 1997). Such events are labeled as "surprising" because they occur in the presence of signals or in situations previously associated with reward presentation. SROs can lead to a variety of behavioral and physiological effects in rodents, including response invigoration, response elicitation, escape, response suppression, and persistence. In this section, we ask whether similar behavioral effects have been observed in analogous experiments with birds. This evidence will provide a base to test the hypothesis that birds experience emotions akin to frustration when exposed to SROs. The following sections are organized along six areas of research: response invigoration, response suppression, escape/ avoidance behavior, persistence, aggressive behavior, and conflict.

### Response Invigoration

Amsel and Roussel (1952) reported that rats ran faster after a surprising nonreward event than after a surprising reward event in a double-runway apparatus. Their interpretation is that a SRO induces an emotional state (called primary frustration), which energizes behavior by increasing motivation to engage the dominant response. A major problem of interpretation has been to distinguish between response invigoration after the SRO (frustration) versus response suppression after the reward presentation (transient demotivation). Subsequent research has shown that both processes, response invigoration and suppression, actually occur in situations involving food omission (Stout et al., 2003). The emotional interpretation is consistent also with the elimination of response invigoration during early appetitive extinction trials in rats deprived of adrenal glands and, therefore, of reduced circulating glucocorticoids (Thomas and Papini, 2001). Adrenalectomized rats learn and extinguish lever-pressing behavior, but lack the response invigoration that usually occurs when food is initially withheld in extinction.

Response invigoration (increased key-pecking behavior) after occasional food omissions has produced inconsistent results in pigeons. In free-operant procedures, Staddon and Innis (1969) reported response invigoration when rewards were omitted in fixed-interval schedules. However, Wilton et al. (1969) reported inconsistent results also using free-operant schedules in pigeons, although the same procedures produced response invigoration in rats. Interestingly, Wilton et al. (1969) observed pacing behavior following reward omission, which they interpreted as indicating an emotional effect of the omission. In an additional phase, Wilton et al. (1969) introduced a limited hold on responding, such that pigeons had to peck the target key within a 0.9-s period to be able to obtain a reward. This limited hold was introduced to reduce the influence of pacing. Under these conditions, the results were again inconclusive. One pigeon showed response invigoration, one showed response suppression, and a third one exhibited no clear effect of reward omissions. However, even if a consistent difference in responding after reward versus after nonreward would have been obtained, the issue of whether it was invigoration driven by frustration or suppression caused by transient demotivation would have remained unresolved.

More conclusive effects of reward omissions were reported in discrete-trial experiments. For example, during fixed-interval performance, pigeons exhibited higher key-pecking rates after reward omission than after reward presentations. However, increasing the intertrial interval from 2 to 12 s eliminated the effect by increasing performance after both outcomes relative to a group receiving continuous reinforcement (Papini and Hollingsworth, 1998). A frustration account would have been supported by a reduction in responding during the long intertrial interval, but, instead, performance was increased, a result pointing to transient demotivation. Thus, pigeons responded less after food presentation, rather than more after food omission. Similar results were obtained in a runway experiment. Pigeons could anticipate rewarded and nonrewarded trials when discriminative stimuli were used, but not when outcomes were unpredictable. However, they ran faster after nonreward than after reward whether the outcomes were expected or unexpected (Stout et al., 2002). Again, this effect seems to have been caused by transient demotivation to respond for food. Furthermore, key pecking was higher immediately after nonreward than after reward, a difference that was eliminated by introducing a delay for responding. However, the difference was eliminated by response recovery after surprising reward, rather than by response decay after surprising nonreward, as a frustrative interpretation anticipated (Stout et al., 2002). Thus, there is no evidence that pigeons (or any other avian species, as far as the authors are aware) exhibit response invigoration after a SRO event that is attributable to primary frustration.

### Response Suppression

As with rats (Logan, 1960; Rashotte and Amsel, 1968), pigeons find it difficult to inhibit responding for food when response spacing is required for reinforcement. For example, pigeons have difficulty adjusting to a schedule of differential reinforcement of low rates in which key pecking needs to occur once every 20 s for reward to be delivered (McMillan and Campbell, 1970). In addition, both d-amphetamine (a drug with multiple synaptic effects, including inhibition of monoamine uptake; Meyer and Quenzer, 2018) and CDP had a small, disrupting effect on key pecking during such schedules. These drug effects are difficult to interpret due to the small number of birds used with each dose (*n* = 2) and to the inconsistency of performance. Perhaps, the most parsimonious explanation for the small disrupting effect of both drugs would appeal to a motor impairment, rather than an emotional component. In general, animals find it difficult to inhibit responding for food, even when a cost (i.e., reward loss) is inflicted upon such impulsive behavior. In pigeons, however, key pecking shows greater impulsivity than treadle pressing (pushing a lever with a foot) in a differential reinforcement of low rates situation (Hemmes, 1975). Whereas pigeons continued operating a treadle with up to a 35-s delay requirement for responding, key pecking was disrupted by a delay requirement as short as 14 s. Nonetheless, examples of response suppression in appetitive situations are especially interesting because the behavior has to overcome the tendency toward impulsive responding.

Response suppression is expected if an animal learns to anticipate, based on prior experience, either the forthcoming omission of a reward or the frustrative response such an omission caused, both having negative emotional connotations. Such learning occurs, for example, during appetitive extinction when approach responses decrease to a minimum, as has been shown since Pavlov (1927) in a variety of situations. Emotional responses can modulate the speed of the response decrement occurring during appetitive extinction. So, rats trained with large versus small rewards and shifted to extinction exhibit differential rates of response decrement—faster after large reward than after small reward acquisition (Wagner, 1961; Papini et al., 2001). One explanation of this phenomenon, called the magnitude of reinforcement extinction effect (MREE), is that the omission of a large reward conditions a stronger anticipation of frustration than the omission of a small reward and, therefore, a stronger

avoidance of the goal that accelerates extinction (Amsel, 1992). Pigeons trained to traverse a runway or to peck at a key in a spaced-trial situation (one trial per day) showed evidence of a reversed MREE, that is, faster extinction after training with a small reward than with a large reward (Papini, 1997; Papini and Thomas, 1997; Thomas and Papini, 2003). Interestingly, pigeons trained with different cues signaling the large and small magnitudes in a successive, spaced-trial discrimination procedure (a spacedtrial version of a simultaneous contrast paradigm) also extinguished more slowly after acquisition with the large-reward cue, than with the small-reward cue (Papini, 1997). A similar reversed MREE was reported in Japanese quail (*Coturnix japonica*) trained in a sexual conditioning paradigm (Baquero et al., 2009). In this case, groups of quail received access to either eight females or one receptive female in the goal box of a runway. When shifted to extinction (no females present at the goal), large reward led to slower extinction than small reward.

In the MREE paradigm, groups trained with different reward magnitudes (large vs. small) are shifted to the same magnitude in extinction (no reward). This is formally similar to the successive negative contrast (SNC) paradigm in which the shift is to a lower, but nonzero, magnitude. In this situation, a reward downshift from a large to a small magnitude has a transient suppressive effect on approach behavior relative to an unshifted control always trained with the small reward (Flaherty, 1996). SNC is related to a variety of indicators suggesting that the effect is accompanied by an aversive emotional reaction—primary and anticipatory frustration, depending on the situation (Papini et al., 2015).

Pigeons trained to peck at an illuminated key for 15 food pellets showed shorter response latencies than pigeons trained to peck for one food pellet, a fact indicating that the magnitudes were discriminable. However, a 15-to-1 pellet downshift caused only a gradual adjustment of response latencies without any signs of SNC (Papini, 1997). This result is consistent with the reversed MREE effects described above; in both cases, pigeons exhibited difficulty to suppress behavior following SROs. An experiment with chickens using a wet cereal reward (more preferred) versus the same cereal mixed with orange oil (less preferred) in a runway also produced evidence of a reversed SNC effect (Davis et al., 2015). These results may not be general across avian species since evidence of SNC was reported in starlings (*S. vulgaris*). In one experiment (Freidin et al., 2009), two groups of starlings received either mealworms (a preferred food) or turkey crumbs during preshift trials. Mealworm animals consumed more food than turkey-crumb animals, thus showing that the rewards affected consummatory behavior. Then, the mealworm animals were downshifted to turkey crumbs, whereas the unshifted controls continued to receive turkey crumbs. Now, downshifted animals consumed less than unshifted controls, thus showing evidence for SNC. In addition, probing the target (i.e., inserting its beak into the bowl that used to contain mealworms in preshift trials) produced no evidence of SNC. Such probing behavior is likely to have been instrumentally reinforced by access to mealworms, in which case, a failure to observe SNC in probing is potentially significant.

Instrumental behavior provided evidence of reversed MREE and SNC in three avian species (pigeons, quail, and starlings), but a regular SNC effect was observed in terms of consummatory behavior in starlings. In rats, consummatory SNC effects tend to be more robust than instrumental SNC effects. For example, several experiments in which rats were trained with sucrose solutions of different concentrations in the goal box of a runway have produced evidence of consummatory, but not instrumental, SNC in the same animals (Flaherty and Caprio, 1976; Sastre et al., 2005). Similarly, the same rats that produced no evidence of SNC in single-option training yielded evidence of SNC in free-choice trials in which they could respond to two levers previously paired with large and small rewards (Conrad and Papini, 2018). In free-choice trials, rats preferred the lever associated to the large reward during preshift, but switched preference to the lever associated with the unshifted reward during postshift trials. As a result, it seems plausible that implementing a SNC manipulation using either consummatory or free-choice procedures would yield evidence of frustration in birds.

### Escape Behavior

The previous section provided scanty evidence for response suppression in situations involving reward downshifts. Suppression of approach behavior is akin to goal avoidance. One can also ask whether birds demonstrate evidence of escaping from a stimulus situation in which they are either exposed to surprising nonreward (escape from frustration effect; Norris et al., 2009) or to a signal for nonreward (escape from the S− effect; Terrace, 1971). To our knowledge, there are no demonstrations of the escape from frustration effect in birds, but there is evidence for the escape from the S− effect in pigeons.

The aversive properties of surprising nonreinforcement can be assessed directly. For example, pigeons learn a key-pecking response when its only consequence is to terminate the stimulus predicting no reward (S−), in an S+/S− successive discrimination (Rilling et al., 1969; Terrace, 1971; Gonzalez and Champlin, 1974). For example, Rilling et al. (1969) trained pigeons in a multiple schedule in which one component delivered rewards at a high rate, whereas the other component was either extinction or had a lower reward rate. Pigeons had the opportunity to peck at a second key at any time during the session and turn off all the lights in the conditioning box. Under these conditions, pigeons pecked more frequently at this second key when they were either in extinction or reinforced at a lower rate. Since the escape key did not affect the rate of reinforcement, these results cannot be interpreted in terms of reducing the delay to the next reward. Terrace (1971) also showed that the removal of the escape contingency results in the extinction of this response. Interestingly, Terrace (1971) reported that escape from the S− does not develop following errorless discrimination training in which the pigeon has never experienced nonreinforcement for responding to the S− and, therefore, should not have experienced frustration. These results were interpreted uniformly as suggesting that the S− has aversive emotional properties.

Pigeons also learn to postpone a time-out period from reinforcement much as they learn to postpone an electric shock. Such responses may be considered to be examples of active avoidance, as different from the response suppression examples discussed in the previous section. Pigeons were trained under a concurrent schedule in which pecking at one key produced rewards on a variable interval schedule while pecking at a second key postponed a time-out period during which all lights were turned off. Consistent with previous studies with rats and chimpanzees, DeFulio and Hackenberg (2007) reported that pigeons acquired responding to the second key that postponed a time-out period, but reduced such responses when they did not affect the occurrence of the time-out period. In addition, they showed that stimuli paired with effective postponement of the time out induced higher response rates than stimuli that were less effective in postponing the time out. The authors view this procedure in terms of the aversive properties of the time-out period. To the extent that a time out postpones food delivery, it has the potential to induce frustration. Escape from S− and time-out avoidance offer interesting parallels with fear in the shock escape/avoidance situation and are thus potentially useful to shed light on emotional processes related to frustration in pigeons.

### Persistence

The partial reinforcement extinction effect (PREE) is defined as greater resistance to extinction after acquisition with partial rather than continuous reinforcement. It is a ubiquitous effect that occurs under a wide range of conditions and in many species (Papini, 2014). In birds, the PREE has been found also under a variety of conditions, including massed and spaced training trials. Under massed conditions, training is administered either in a free-operant situation with the key light continuously available for responding (e.g., Nevin, 1988) or with intervals between trials in the order of seconds to a few minutes in discrete-trial situations (Jenkins, 1962; Gonzalez and Champlin, 1974). Massed training conditions can produce behavioral effects that are consistent with emotional memory, but based upon nonemotional mechanisms. When trials are administered in closed temporal proximity, the carry-over sensory or shortterm memory traces of rewards consumed in one trial can acquire control over responding during the following trial. Partially reinforced animals can learn to respond in the presence of carry-over events involving both reward and nonreward, whereas continuously reinforced animals only learn to respond in the presence of reward carry-over events. As a result, a shift to extinction is less disruptive to partially reinforced animals than it is to continuously reinforced animals—a matter of stimulus generalization decrement (Hull, 1952).

There is evidence that short intertrial intervals promote the PREE under conditions that do not yield this effect when training is widely spaced. For example, toads (*Rhinella arenarum*, formerly *Bufo*) trained to traverse a runway for water reinforcement exhibit the PREE with 15-s intertrial intervals, but not with 300-s intertrial intervals (Muzio et al., 1992). Couvillon et al. (1980) tested the carry-over hypothesis by interpolating nontarget stimuli between trials with the target stimulus. Pigeons showed single alternation behavior when reinforced and nonreinforced trials alternated, but behavioral patterning was eliminated when the interpolated task was introduced. However, the same interpolated task still yielded a PREE to the target stimulus in pigeons, suggesting that carry-over stimuli may not prevent pigeons from reactivating the memory of previous trial outcomes, a mechanism that is assumed to explain the spaced-trial PREE.

This result is consistent with reports of the spaced-trial PREE in pigeons under both key-pecking (Papini et al., 2002) and runway situations (Roberts et al., 1963; Thomas and Papini, 2003). A similar runway effect was observed in quail with food reward (Buriticá et al., 2013). In these experiments, 50% partial reinforcement training administered at a rate of one trial per day led to greater resistance to extinction than continuous reinforcement. Thomas and Papini (2003) extended these findings in three directions. First, they reported that training with variable reward magnitudes (large and small reward intermixed during acquisition) also yield increased resistance to extinction in widely spaced training in pigeons. Second, they asked whether the pigeon PREE was based on the same mechanisms underlying the PREE in rats, and provided evidence based on drug effects. Three drugs were selected based on published research with rats (see Thomas and Papini, 2003, for references): CDP (a GABAergic drug that eliminates the PREE), haloperidol (a dopamine D2 receptor antagonist that has no significant effect on the PREE), and nicotine (an agonist at acetylcholine receptors that enhances the PREE). Although these drugs affected the spacedtrial runway PREE in pigeons, their effects differed relative to those described in rats. For example, CDP retarded the emergence of the PREE in pigeons, but it did not eliminate it; haloperidol and nicotine, however, did eliminate the PREE in pigeons. Thus, although behaviorally analogous, it is plausible that the PREE is based upon different neurotransmitter systems in rats and pigeons, a possibility that remains to be fully analyzed.

Finally, Thomas and Papini (2003) tested the co-variation of the PREE and MREE within the same experiment, using widely spaced training in the runway situation. Four groups of pigeons received acquisition training with large, partial reinforcement (L/P); large, continuous reinforcement matched for reinforcement (L/Cr); large, continuous reinforcement matched for trials (L/Ct); and small, continuous reinforcement (S/C). L/P acquisition yielded increased resistance to extinction compared to L/Cr and L/Ct acquisition and thus yielded evidence of a PREE. However, S/C extinguished faster than L/Ct, thus revealing a reversed MREE. Such a dissociation between these two effects involving SROs is consistent with the hypothesis that these behavioral phenomena are based upon different mechanisms.

It seems clear that birds show a PREE behaviorally similar to that observed in rats. However, whether the effect is dependent upon the emotional memory of frustrating events remains to be determined. The information available thus far suggests that the mechanisms underlying the PREE in pigeons and rats are different.

### Aggressive Behavior

Evidence consistent with an emotional component stemming from SROs comes from experiments assessing aggressive behavior in birds. Azrin et al. (1966) provided a detailed description of the aggressive responses of pigeons trained in intermittent schedules of food reinforcement, during period of food omission:

Visual observation revealed that attack consisted of strong pecks at the throat and head of the target bird, especially around the eyes. The feathers of the target bird were often pulled out and the skin bruised. The attack was often preceded by a brief period of pacing in front of the wall on which the response key was mounted. Occasionally, the pecking attack was preceded by striking movements of the wing or by a slow swaying approach to the target bird with the head lowered. Frequently, the attack was preceded and accompanied by a deep-throated sound (pp. 194–195).

During appetitive extinction, aggressive responses are directed at another pigeon present in the conditioning box (Azrin et al., 1966; Gentry, 1968; Rilling and Caplan, 1973), at a picture of a pigeon projected on a screen (Yoburn and Cohen, 1979), and at a mirror reflecting the image of the pigeon (Cohen and Looney, 1973). Chickens also behave aggressively when food is present, but made inaccessible (Duncan and Wood-Gush, 1971), and also during appetitive extinction (Kuhne et al., 2011). Azrin et al. (1966) found that the intensity of aggressive responses in pigeons increased with the number of rewards in acquisition and decreased when pigeons were given free access to food. Pigeons also display aggressive behavior in the presence of cues that predict reward omissions (Terrace, 1972). Interestingly, wing flapping, an aggressive response, is prominent during the early stages of visual discrimination training when differential responding has not yet emerged. However, as the pigeon learns the discrimination, wing flapping is reduced and eliminated. This result is consistent with the hypothesis that frustration mediates the relationship between unexpected reward failures and aggressive behavior (Amsel, 1992). Frustration is expected to weaken when the discrimination develops and, as a result, nonreward becomes expected in the presence of the S− and so wing flapping is reduced.

There is also evidence that the vocalizations described by Azrin et al. (1966) in the previous quotation reflect emotional arousal. Rashotte et al. (1975) reported evidence of a vocalization (called "Vocal A" by the authors) that usually accompanied aggressive responses, such as during episodes of wing-striking and pecking at the head region of the target pigeon. These were low-frequency vocalizations in the range between 0 and 2 kHz.

In poultry, it has been observed that preventing the occurrence of natural behaviors may redirect behavior. For example, chickens in animal production settings can peck and remove feathers from other birds to the point of causing bleeding. Feather pecking is usually interpreted as resulting from prevented natural behaviors, including foraging or dust bathing behavior. Careful monitoring of the topography of feather pecks in hens concluded that such pecks resemble those involved in foraging, rather than sand bathing, and thus likely develop out of frustrated opportunities for displaying feeding behaviors (Dixon et al., 2008).

Aggressive responses are routinely induced by periods and signals of nonreinforcement, thus offering a potentially fertile ground to test the notion that birds experience frustration. What is missing in this research is a systematic analysis of the nonreward-induced aggressive behavior in terms of underlying neurobiological factors. Drug and brain manipulations are needed to determine the extent to which these behavioral effects can be attributed to emotional activation.

### Conflict

A number of experiments with pigeons have looked at choice in situation involving different reward magnitudes or probabilities. For example, McDiarmid and Rilling (1965) first reported that pigeons prefer a schedule associated to a less frequent reward to one associated to a more frequent reward if the former involves a shorter delay than the latter. Similarly, pigeons trained to withhold key pecking to earn a large reward, rather than pecking immediately to earn a small reward, failed to inhibit responding and, thus, lost a substantial amount of food (Ainslie, 1974). Such performance can typically be reversed by manipulating either the length of the delay or the magnitude of the reward. Variations of this procedure, known as delay discounting, suggest that pigeons, like other animals, are sensitive to temporal delays such that a large reward is reduced in value if it is presented only after a temporal delay. Although delay discounting is usually interpreted in cognitive terms, it is potentially amenable to an interpretation based on frustration (Amsel, 1992). When there is an immediate reward available in the same session, delayed reward options become less acceptable if they induce frustration.

Research on delay discounting has shown that individual neurons located in the pigeon's nidopallium caudolaterale, a region considered homologous to the mammalian prefrontal cortex, encode information of both reward delay and magnitude (Kalenscher et al., 2005). Additional unit recordings from the nidopallium caudolaterale indicate that some neurons are maximally active during the interval between the onset of the stimulus and the production of the response. A task that required rapid responding to one stimulus (as fast as possible), but postponing responding to a second stimulus for 1.5 s from stimulus onset, revealed units that increased activity with a rate appropriate to these two types of trials—steeper for the rapid-response stimulus than for the waiting-response stimulus (Kalenscher et al., 2006). Moreover, trials in which the pigeon responded too early or too late, thus missing a reward, were characterized by the unit responding at the incorrect rate. Delay discounting offers an interesting arena to test emotional learning in pigeons.

### FURTHER COMMENTS

These additional comments on avian emotions are organized according to three significant questions suggested by the present review. First, *what is the strength of the evidence for fear and*  *frustration in birds*? The evidence for fear in birds seems stronger than the evidence for frustration. As noted in the introduction, the search and evaluation of evidence for fear in birds starts from a comparison with analogous experiments with mammals, since mammalian fear is perhaps the emotion that is best understood. Like mammals, birds show evidence of similar responses to analogous stimulus conditions. TI, freezing, the acquisition of passive and active avoidance responses, and the effects of benzodiazepine anxiolytics in punished responding all point to substantial similarities with mammalian fear. However, these are just a few pieces in a large puzzle. To have a comprehensive view of the degree of similarity in underlying mechanisms, a starting approach would be to systematically compare two species, under analogous conditions, and in terms of at least four levels of analysis: behavioral, neurobiological (neural circuitry), neurochemical (synaptic transmission), and cell-molecular (synaptic plasticity) (Papini, 2002, 2003, 2008). A case for homology of emotional behavior would be strengthened if the same mechanisms were uncovered at all levels of analysis. By contrast, similarity in behavior supported by different underlying mechanisms would be consistent with independent evolution by homoplasy. Of all the preparations for studying emotional behavior reviewed above, the one that seems most suitable for such a detailed analysis would seem to be punished responding. There are extensive behavioral and neurochemical (psychopharmacological) data in both pigeons and rats that set the basis for a systematic evolutionary approach to a search for avian/mammalian homology in fear.

Although the term "frustration" has been used extensively in the literature, especially in applied research with birds, the evidence for extensive similarities in the adjustment to SROs in birds and mammals is not as strong as it is in the case of fear. In some cases, the behavioral effects are simply different (e.g., SNC and related effects). In other cases, pigeons and rats produce similar behavioral effects, but apparently based upon different mechanisms, as revealed by drug manipulations (PREE). A promising phenomenon that would need to be rescued for analysis with current behavioral and neural techniques is that of extinction-induced aggressive behavior in pigeons (Papini and Dudley, 1997; Papini, 2014).

Another result in need for further analysis is the behavioral dissociation between PREE and MREE in pigeons. For example, a comparison between pigeons and rats in their adjustment to SROs suggests different scaling properties of their behavior. For rats, reward downshift is controlled by the ratio of the reward magnitudes obtained in postshift and those expected from preshift experience, rather than by their absolute magnitude (Papini and Pellegrini, 2006; Pellegrini and Papini, 2007; Pellegrini et al., 2008). For pigeons, however, postshift key-pecking performance is controlled by the absolute magnitude of the preshift reward (Pellegrini et al., 2008). The behavior of pigeons in reward devaluation tasks seems to be under tight control by the long-term memory of the preshift reward magnitudes. Instead, rats exhibit behavioral flexibility in analogous situations. Negative emotions such as anticipatory frustration induced by prior experience with SROs may accelerate the detachment from a signal or location previously paired with reward, a process known as incentive disengagement (Papini, 2003). One function of negative emotions may be to facilitate the disengagement from incentives that are no longer yielding sufficient rewards to support survival and reproductive success.

Second, *what is needed to characterize the relationship between brain and emotion in birds*? A model to imitate is provided by research on spatial learning and hippocampal size and function in birds and mammals (Sherry et al., 1992; Broglio et al., 2015). In this case, there is a fit between structure and function that permits a direct visualization of brain-behavior relationships. This is facilitated by the development of analogous training techniques that can be applied to different species (teleost fish, amphibians, birds, and mammals) and a structure that can be clearly identified in terms of homology across a wide range of species. A similar argument could be made for the relationship between active and passive avoidance learning (in relation to fear), or runway performance under conditions of reward devaluation and extinction (in relation to frustration). Such training techniques have been applied to a variety of vertebrate species using analogous procedures. The key structure in this case would be the amygdala, which, like the hippocampus, has identifiable homologs across vertebrates (Moreno and González, 2007; Vargas et al., 2012). For example, studies with Roman high- and low-avoidance rat strains, selected for active avoidance learning since the 1960s (Driscoll and Bättig, 1982), provide a potential approach to follow. In addition to differences in avoidance learning, these strains also exhibit differences in a variety of emotional behaviors, including other types of avoidance learning, anxiety situations, and also situations involving SROs (Torres and Sabariego, 2014). In general, Roman low-avoidance rats show higher levels of anxiety, SNC, PREE, and the emotional self-medication effect involving increased voluntary consumption of anxiolytics immediately after episodes involving SROs, compared to Roman high-avoidance rats. Gómez et al. (2008) reported that poor one-way avoidance in Roman low-avoidance rats was correlated with low cell density in the basolateral amygdala. Similar selective breeding protocols could be applied to some avian species, including Japanese quail and chickens, for which there are also behavioral techniques to study analogous phenomena (e.g., selective breeding for TI; Jones et al., 1991).

Third, *what information do these studies in birds offer to understand the origin of negative emotions in vertebrates*? The study of fear and frustration in mammals has suggested that the underlying neural mechanisms overlap extensively—the fear = frustration hypothesis (Wagner, 1969; Gray, 1987; Gray and McNaughton, 2000). Comparative research in other vertebrates can help determine whether brain circuits dedicated to these emotions arose simultaneously or sequentially during vertebrate evolution. Based on data on avoidance learning in goldfish (*Carassius auratus*), Papini (2003) suggested that the brain mechanisms underlying frustration could have evolved from fear mechanisms in early mammalian ancestors by a combination of gene duplication and co-option. This is based on several sources of evidence, including the comparative distribution of the SNC effect in vertebrates. SNC has been described in several mammalian species and it has been shown to correlate with behavioral and physiological indicators of emotional activation (Papini et al., 2015). Analogous experiments with teleost fish, amphibians, turtles, and birds have provided no evidence of SNC, with the exception of a single experiment with starlings (Papini, 2014). However, some of the same species that show no evidence of SNC readily provided evidence for avoidance learning: goldfish (Portavella et al., 2003), amphibians (Daneri et al., 2007), and birds (see above). Interestingly, avoidance behavior in goldfish (Portavella et al., 2004) and in toads (Puddington et al., 2016) depends on activity in brain areas homologous to the mammalian amygdala.

Much remains to be done to develop lab techniques to study behaviors clearly related to emotion, following the lead of such studies with mammals. This review suggests several potential candidates for behavioral targets. But, in addition, behavioral effects need to be analyzed at lower mechanistic levels from a comparative perspective to uncover clues on the evolution of vertebrate emotional systems. These goals could

### REFERENCES

be best approached at this time using fear and frustration as model emotions.

### AUTHOR CONTRIBUTIONS

All the three authors contributed, to varying degrees, in each of the following items: conception of the article, writing of the manuscript, review of form and content, and approval of the final manuscript.

### FUNDING

AP-A was supported by funds from COLCIENCIAS, Universidad del Rosario, and Fundación para el Avance de la Psicología, Colombia. *Resultados de la Convocatoria 395-2007, "Programa de Movilidad Internacional de Investigadores e innovadores a Eventos y Estancias de Corta Duración"*. To: AP-A, PhD, postdoctoral fellow at the Department of Psychology, Texas Christian University, Fort Worth, TX, USA.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

*Copyright © 2019 Papini, Penagos-Corzo and Pérez-Acosta. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.*

# Hands Up! Atypical Defensive Reactions in Heavy Players of Violent Video Games When Exposed to Gun-Attack Pictures