In model 2, regarding school context, the explained variance is 20.4% and the model correctly classifies 16.8% of the vulnerable adolescents. Specifically, those adolescents who perceive lower teacher support, with an OR of 0.87 (95% CI = 0.83–0.91), have less positive feelings toward school (OR = 0.77, 95% CI = 0.65– 0.91) and worse academic achievement (OR = 0.60, 95% CI = 0.50–0.72) have a higher likelihood of being vulnerable.

Model 3, which includes the variables of peer context, shows a predictive capacity of 16.8%, with 10% of the vulnerable adolescents being correctly classified. Those adolescents who report lower perceived social support (OR = 0.97, 95% CI = 0.94– 0.99) and less satisfaction with friendships (OR = 0.76, 95% CI = 0.69–0.83) are more likely to be vulnerable.

Model 4 is devoted to variables related to lifestyles and its explained variance level is 21%, with 17.5% of the vulnerable adolescents being correctly classified. In this model, alcohol use stands out, showing that those adolescents who show a higher frequency of alcohol use in the last 30 days are 1.45 times more likely to be vulnerable (95% CI = 1.20–1.76). Likewise, the adolescents who do less moderate to vigorous physical activity (OR = 0.74, 95% CI = 0.67–0.80) and less vigorous physical activity (OR = 0.89, 95% CI = 0.80–0.99) have a higher likelihood of being vulnerable.

The group of psychological variables, analyzed in model 5, has the highest level of explained variance among the six specific models. Specifically, the level of explained variance in model 5 is 44.7% and 52% of the vulnerable adolescents are correctly classified. As in the previous section regarding resilient adolescents, the significant variables in this model are: sense of

TABLE

8

Logistic

regression

models

on

resilience

by

demographic,

school

context,

peer

context,

lifestyle,

psychological

and

socioeconomic

variables.

 <

 <

 variables.

TABLE

 p <

 p <

coherence, curiosity and exploration and satisfaction with body image. The likelihood of being vulnerable is higher in those adolescents with a lower score in sense of coherence (OR = 0.27, 95% CI = 0.18–0.40), less satisfaction with their body image (OR = 0.46, 95% CI = 0.35–0.69) and a lower score in curiosity and exploration (OR = 0.95, 95% CI = 0.92–0.98).

Model 6, examining the socioeconomic variables, again shows a lower predictive capacity than previous models (11.2%), with only 2.7% of the vulnerable adolescents being correctly-classified. The mother's educational level is the only significant variable in this model, revealing that those adolescents whose mothers have a lower educational level exhibit a higher probability of being vulnerable (OR = 0.66, 95% CI = 0.51-0.87).

Lastly, in model 7 or the global model (in which only the significant variables from previous models have been included), the following six variables were significant: perceived academic achievement, perceived teacher support, moderate to vigorous physical activity, sense of coherence, curiosity and exploration and satisfaction with body image. The predictive capacity of this model is very high, with an explained variance level of 56.5%. This model was also able to correctly classify a high proportion of vulnerable adolescents, specifically 62.9%.

The independent variables which stand out in this model due to their ORs being closer to zero, and therefore their higher predictive capacity, are sense of coherence (OR = 0.30, 95% CI = 0.19-0.45), academic achievement (OR = 0.49, 95% CI = 0.32–0.76) and satisfaction with body image (OR = 0.50, 95% CI = 0.35–0.71). Moderate to vigorous physical activity (OR = 0.70, 95% CI = 0.58–0.86) and teacher support (OR = 0.85, 95% CI = 0.75–0.95) appear on an intermediate level. Lastly, the level of curiosity and exploration (OR = 0.96, 95% CI = 0.93– 0.99) made the most modest contribution. Higher levels of the aforementioned variables are associated with a lower likelihood of belonging to the group of vulnerable adolescents.

## DISCUSSION

The aim of this study was to characterize resilience and vulnerability in a large and representative sample of adolescents. This objective was first addressed separately on a number of potential levels of influence (demographic, school, peer, lifestyle, psychological, and socioeconomic variables) and later, in a more holistic approach, by integrating the factors in all the aforementioned levels.

A separate analysis of each of the two phenomena showed, first at all, that although there was a higher representation of boys and younger adolescents in the resilient group, and of girls and older adolescents in the vulnerable group, the variables sex and age were not sufficient to accurately predict adolescent adaptation. Previous research has found differences in wellbeing and adjustment between boys and girls, as well as according to age (Cavallo et al., 2006; Ramos et al., 2010), but at the same time there is notable diversity amongst adolescents of the same sex and age. This diversity tends to be related to the combination of life experiences and psychological characteristics of these adolescents. Hence the demographic variables (that were included in all the regression models) were insufficient to characterize such complex phenomena such as resilience and vulnerability and their significant effects disappeared when they were entered along with the rest of the variables in the final model. In fact, sex and age already lost their significant effects in previous models, specifically in those evaluating the contributions of psychological and socioeconomic variables. This is probably owing to that those models incorporated variables such as satisfaction with body image, which tends to be lower and more strongly associated with girls' wellbeing (Knauss et al., 2007; Mond et al., 2011), or family wealth, which tends to be assessed more negatively by older adolescents (Goodman et al., 2001). Therefore, it could be understood that these predictor variables (such as body image or family wealth) explain the predictive capacity of the variables sex and age on the phenomena resilience and vulnerability.

Beyond demographic variables, a look to the separate models for each set of predictors shows that a hierarchy based on the predictive capacity of each set of variables would be very similar for resilience and vulnerability: psychological variables in the first place, along with contextual and lifestyle variables, and more modest contributions of demographic and socioeconomic variables.

In addition, the final models for resilience and vulnerability also revealed a number of common factors for the explanation of both phenomena. In other words, these analyses also helped identify several factors that contributed significantly to explaining both resilience and vulnerability.

First at all, sense of coherence was one of the most important factors for both resilience and vulnerability. This construct, coming from the salutogenic model in the field of public health, has to do with a person's ability to interpret their social environments as predictable and ordered, their confidence that any life demand can be successfully dealt with as well as a motivational-emotional component that helps one to see difficult situations as challenges and facilitates an active engagement in problem-solving (Antonovsky, 1987). Therefore, the important contribution of sense of coherence to resilience and vulnerability should come as no surprise. On one hand, its links to some factors associated with successful adaptation in classic resilience studies, such as analytical skills, motivation to engage in the environment, self-efficacy and self-esteem (Masten, 2001; Hamill, 2003), are apparent in the prior description. In addition, research on sense of coherence indicates that its relationship with health and wellbeing is rooted in helping people mobilize other useful coping resources in stressful situations (Lindström and Eriksson, 2010), which has led to its inclusion in the health assets model as a supra-order asset for wellbeing (Morgan and Hernán, 2013). In this sense, one line of research that arises from the results obtained in the current study is the study on the processes that explain why a high sense of coherence would help resilient adolescents take full advantage of available resources, whereas low levels of the same would hamper the effective use of the apparently more abundant resources in the case of vulnerable adolescents.

Satisfaction with body image and perceived academic achievement also appeared as important explanatory variables in the analysis of both resilience and vulnerability. The significant contribution of satisfaction with body image is probably related to the importance of physical appearance for adolescents' positive self-perception. In this regard, numerous studies have found a significant relation between satisfaction with body image and self-esteem in adolescence (Tiggemann, 2005), this latter being a factor traditionally connected to successful adaptation (e.g., Dumont and Provost, 1999). Something similar can be said of the relationship between perceived academic achievement and self-efficacy (Danielsen et al., 2009), another fundamental protective factor in resilience research (Hamill, 2003). Additionally, previous research indicates that feeling competent in daily life is very important for the adaptation of individuals suffering adversity (Masten and Coatsworth, 1998). Therefore, it is likely that behind an adolescent who thinks that their teachers consider their academic achievement as good, there are various underlying beneficial elements for adaptation and wellbeing, such as experiences of competence in the school context, higher school connectedness or even a higher intellectual capacity (Masten et al., 1999; Blum, 2005).

In addition to perceived academic achievement, higher levels of teacher support increased the likelihood of showing resilience and diminished that of being part of the group of vulnerable adolescents. Studies about teachers' contribution to adolescent wellbeing also suggest that, regardless of the level of academic achievement, teacher support acts as an asset associated with wellbeing for all adolescents (e.g., García-Moya et al., 2015), which makes it fundamental to favor close and supportive teacher-student relationships.

Moderate to vigorous physical activity was also amongst the significant factors associated with resilience and vulnerability. Physical activity has been found to have protective effects in stressful situations (Gerber and Pühse, 2009; Silverman and Deuster, 2014), as well as it tends to reduce the likelihood of engaging in risk behaviors (Pate et al., 1996), therefore serving as a clear example of the importance of taking into account lifestyles' contributions to explaining resilience and vulnerability.

Finally, higher levels of curiosity and exploration increased the likelihood of being resilient and diminished that of being part of the vulnerable group. The curiosity and exploration construct reflects openness and interest in learning, good management of the uncertainty associated with new or unknown situations (Kashdan et al., 2009) and is associated with psychological and contextual variables significantly linked to adaptation and resilience. Specifically, high levels of curiosity and exploration are related to an active response in unfamiliar and challenging environments (Kashdan and Roberts, 2004) and have been linked to psychological variables such as intrinsic motivation and selfefficacy (Kashdan et al., 2004). Additionally, curiosity was also significantly associated with more positive social interactions (Kashdan and Roberts, 2004). Specifically, people with higher levels of curiosity and exploration generated more positive responses from strangers, who tended to be more responsive, participative, and interested in social exchanges with people with high curiosity. Despite this, the contribution of curiosity and exploration to the final model was relatively modest, probably due to its connections with other constructs, such as sense of coherence. The conceptual delimitation of curiosity and exploration is still under study (Kashdan et al., 2009), and with regards to sense of coherence one focus of analysis and debate is precisely its connection to other constructs in positive psychology (Lindström and Eriksson, 2010). Consequently, advancing in the conceptual delimitation of these constructs, identifying common elements and differences between them, is an important line of research (García-Moya and Morgan, 2016) that could contribute to a better understanding of resilience and vulnerability and, in general, of their role in promoting adolescent wellbeing and adjustment.

As explained in the previous lines, the vast majority of the examined factors operated by increasing the likelihood of good adaptation in resilient adolescents and diminishing it in vulnerable ones. Overall, this suggests more similarities than differences in the factors contributing to explaining resilience and vulnerability. These findings coincide with previous research pointing out that factors associated with resilience are not specific to this phenomenon, but that they are the manifestation of basic systems of human adaptation and, therefore, are influential in both adversity and non-adversity situations (Masten and Coatsworth, 1998; Masten, 2001). Additionally, some scholars have noted that protective factors identified in resilience and vulnerability studies frequently correspond to the positive pole of risk factors for maladaptation or, in other words, that in this type of research it is possible to identify factors in which one of their extremes facilitates successful adaptation while the opposite hampers it (Sameroff and Fiese, 2000; Fergus and Zimmerman, 2005; Luthar et al., 2015), which seems to coincide with findings in the present study.

Despite the predominant similarities described so far, results also revealed some differential aspects between the resilience and vulnerability phenomena. First, the psychological variables showed a larger explicative capacity in vulnerable adolescents than in resilient ones (R 2= 0.447 and 0.370, respectively), whereas factors related to school and peer contexts, especially the second, showed a stronger association with resilience than with vulnerability (R 2= 0.228 and 0.238 respectively in the models on resilience vs. 0.204 and 0.168 for vulnerability). Some research suggests that certain protective factors such as temperament (e.g., Werner and Smith, 1982) or intellectual capacity (Masten and Coatsworth, 1998), to name some classic examples, have a multiplier effect, i.e., they can contribute to a higher likelihood of encountering other positive events in life, giving rise to chain reactions favoring positive adaptation or, conversely, they can initiate cascading effects in which new risk factors are more probable. Applying a similar logic, it can be hypothesized that certain psychological variables, such as a low sense of coherence, a lower tendency toward curiosity and exploration, or a higher dissatisfaction with body image, could be preventing vulnerable adolescents from taking advantage of potential resources in extrafamily environments (school and peer contexts), whereas resilient adolescents, despite their more unfavorable family context (which was the indicator used for the definition of adversity in the current study), would be more likely to find and benefit from resources available in extrafamily environments thanks to their more positive profile in psychological variables.

Along these lines, prior research has documented the existence of compensatory effects from other contexts in only part of the adolescents exposed to low-quality family contexts (e.g., García-Moya et al., 2013b).

Second, three of the examined factors, specifically perceived family wealth, satisfaction with friendships and breakfast frequency, were only significant in the analysis of resilience. This means that these variables made a difference for adolescents exposed to adversity in the family context (resilient vs. maladaptative adolescents) but did not contribute to explain differences in adaptation between vulnerable and competent adolescents. Scientific literature has extensively documented that resilience has among its defining attributes an ability, despite adversity, to find and take advantage of any resources and opportunities in proximal environments.

In this sense, it is not surprising that being raised in a family environment with good socioeconomic resources opens a horizon of possibilities to resilient adolescents that they seem to know how to take advantage of. What is interesting in the findings of the present study is that although vulnerable and resilience adolescents reported similar levels of perceived family wealth, this factor made one of the most noticeable contributions in analyzing resilience (but not vulnerability). A reflection on the nature of the indicator used may help understand this finding. On the one hand, research suggests that perceived family wealth includes some of the elements which are common to objective indicators such as family affluence, and therefore, it can arguably be interpreted as indicative to some extent of the wider access to external resources and opportunities for development that families' socioeconomic level relates to Bornstein and Bradley (2003). However, research also indicates that subjective and objective measures are not assessing exactly the same content (Hartley et al., 2015; Elgar et al., 2016), since unlike objective indicators perceived family wealth may also incorporate a comparative assessment of the socioeconomic position of the adolescent's family in comparisons with that of others they related with (Moreno-Maldonado et al., under review). The levels of wealth perceived by resilient adolescents may therefore represent a relative socioeconomic advantage for these adolescents compared to their peers also exposed to adversity in family relationships (the maladaptative group).

Results on satisfaction with friendships can also be interpreted in a similar sense, i.e., that resilient individuals are able to take advantage of the potential resources they find. Peer support tends to be considered a protective factor in adversity situations (Olsson et al., 2003) and resilient adolescents in the present study probably illustrate very well the compensatory effects which are frequently mentioned in this field (e.g., Luthar et al., 2015): they belong to a group who, despite coming from families in which parent-child relationships are not good, is able to build positive relationships with their peer group and benefit from them (Lansford et al., 2003; Rubin et al., 2004). In a similar vein, Luthar et al. (2015) state that relationships with peers can become a "remedial" socializing context for children who grow up exposed to family adversity. In addition, positive peer relationships are indicative of good social competence, a fundamental skill in which resilient adolescents usually show positive results, which are comparable to those of competent adolescents and clearly more favorable than the social competence levels exhibited by maladaptative adolescents (Masten et al., 1999).

Finally, the fact that breakfast frequency was significant only in the analysis of resilience may be related to the fact that, as children gain more independence during adolescence, the importance of parental supervision in this behavior decreases while internalization of the habit and other personal characteristics, such as constancy and self-regulation, gain prominence (Kalavana et al., 2010). Given that breakfast frequency is also believed to act as a proxy for diverse socioeconomic and family aspects this is an issue which, in particular, would benefit from further research.

In any case, the comments that have been made throughout this discussion about a higher ability of resilient individuals to take advantage of potential resources in proximal contexts or the important role of psychological factors for explaining the resilience and vulnerability phenomena should not be interpreted as evidence that they are characteristics unrelated to the contextual experiences associated with resilience and vulnerability. As rightly pointed out by Luthar et al. (2015), contextual experiences indeed give shape, from the beginning and in a continuous transactional dynamic, to said skills or psychological resources.

This study has some limitations that should be taken into consideration in the interpretation of its findings. Firstly, its cross-sectional design means that the results must be interpreted on an associative level, not being possible to draw conclusions about the directionality of the relationships found. Secondly, adversity was defined using quality of parent-child relationships as a sole criterion. Although, as explained in the introduction, this is an well-informed decision, which draws on scientific literature that highlights the role of family as a basic system for human adaptation (Masten, 2001; Fergus and Zimmerman, 2005), previous research also shows the wide variety of life circumstances that can constitute adversity in childhood and adolescence (Luthar et al., 2000); consequently, it would be inappropriate to generalize these findings to other adverse circumstances. Finally, this study used a factorial health score as its measure of adaptation. Although, this measure is a sound and validated global health indicator (Ramos et al., 2010) whose characteristics fit well with key measurement issues in the empirical definition of positive adaptation (Luthar and Cushing, 1999), there is substantial evidence on the multi-dimensional nature of human adaptation, which makes individuals show dissimilar levels of adaptation in different areas (Luthar et al., 1993). Therefore, future research should complement the present study by conducting separated analyses of the contributions of the factors analyzed here to distinct areas of adaptation, mainly the following: academic, behavioral, social and emotional (Masten et al., 1999; Luthar et al., 2000).

Despite the aforementioned limitations, this study also has important strengths. In line with recommendations from some of the seminal reviews in this research field (Luthar et al., 2000, 2015; Masten, 2014), the elements of adversity and adaptation were clearly operationalized for the definition of resilience and vulnerability in the present study, which is fundamental for an adequate interpretation of its findings and its comparability with other studies. The criteria used for making the distinction and comparisons among the four adaptation groups (competent, vulnerable, resilient, and maladaptative) were also based in previous research (Tiêt and Huizinga, 2002). Additionally, this research adheres to the methodological rigor characteristics of the HBSC survey (Roberts et al., 2009), as well as it stands out for its large sample size, which allowed for a characterization of resilience and vulnerability phenomena in a representative and notably large sample. Although, the four groups may appear unbalanced in size, the representativeness of the initial sample is a guarantee that this is a relatively realistic reflection of the population. In addition, using effect size tests in all of the analyses minimizes the potential bias that such differences in the subgroups' size could case from a methodological point of view. The high predictive capacity of the models of resilience and vulnerability obtained, which reached levels of explained variance higher than 50%, is also outstanding. These values are considered high in the field of behavioral science (Cohen, 1988), being notably above the 10– 20% that is usual for associations between protective factors and adaptation outcomes in resilience studies (Luthar et al., 2000). Finally, this study has three elements that are, to some extent, innovative. Firstly, factors traditionally receiving little attention as referred to in the introduction, such as lifestyles, satisfaction with body image, sense of coherence, curiosity and exploration and perceived family wealth, were analyzed in the present study of resilience and vulnerability. Secondly, this study included vulnerable adolescents, a population subgroup that had rarely been studied in previous research due to its limited sample size (Masten et al., 1999). Additionally, this work makes a valuable contribution regarding the prevalence of vulnerability and resilience in the general population. Given the difficulties associated with defining resilience and vulnerability and the limited methodological consensus with regards to the measures to use and how to apply them to a representative sample, it is understandable that prevalence studies are not available. In this regard, the present study found that vulnerable adolescents made up 3.9% of the global sample, representing 11.9% of the group that reported high-quality parent-child relationships. The resilience group represented 4.5% of the global sample, corresponding to 13.4% of the participants with low-quality parent-child relationships, which is in line with the findings of some longitudinal studies that have found a very low prevalence and stability in resilient coping (Cicchetti and Rogosch, 1997). Specifically, Bolger and Patterson (2003) found that between 6 and 21% of abused children were functioning competently during at least one of the temporal points examined in their longitudinal follow up from middle childhood to early adolescence, but less than 5% consistently maintained that competent functioning over time.

In addition to its strengths from a research perspective, which have just been highlighted, the fact that the present study provides valuable implications for the improvement of the methodological quality of interventions with resilient and vulnerable populations, which was one its guiding principles, should also be noted amongst its strong points.

Throughout these pages a number of important factors for adolescents' successful adaptation have been underlined. These include certain personal characteristics (such as sense of coherence, satisfaction with body image and curiosity and exploration), as well as some that characterize their lifestyles (regularity in healthy eating habits and physical activity) or that refer to the quality of their developmental contexts (such as satisfaction with peer relationships, academic achievement and teacher support). Therefore, all of these are elements to bear in mind in interventions aimed at promoting successful adaptation and wellbeing in adolescence (Olsson et al., 2003). Likewise, this study highlights the need to conduct further research devoted to developing reliable and valid indicators for the assessment of all these factors, both those that characterize the individual person and the ones that characterize their developmental contexts. These indicators will serve the double function of detecting subjects with different profiles of adaptation as well as of monitoring their evolution and evaluating the implemented interventions.

On a separate issue, it should be noted that some studies have advocated that interventions should be adjusted to the distinct developmental needs of adolescence (Kim et al., 2015). What the present research adds is that detecting different adaptation profiles would also serve to adjust interventions to every person's specific needs. On the one hand, some could argue that allocating powerful and costly resources to detect and intervene in vulnerable individuals, which represents 3.9% of adolescents, would not be an efficient strategy. However, it must be noted that this study used very demanding criteria to define the categories of vulnerability/resilience, and hence may have underestimated the prevalence figures. Additionally, it is well known from the accumulated evidence in previous research that life paths of vulnerable people will be full of difficulties in very different areas (this paper has provided some good examples of this). From an economic perspective, those adverse life paths will lead to a lot of public spending in the education, health, legal and judicial, and labor systems, amongst others (see Khan et al., 2015), if the direct and indirect costs to which these difficulties will give rise are taken into consideration; consequently, they should be detected as soon as possible. On the other hand, one should not forget that amongst the adaptation profiles considered in this paper, there were also a 18.88% of maladaptative adolescents with clear intervention needs, and in the remaining 77.3% of adolescents there will most likely always be areas of improvement and optimization in need of reinforcement. Similarly, it could also be thought that the resilient adolescents, for which our study shows a prevalence only slightly higher than 4%, would not need any intervention because they seem to resist adversity without help. It is true that these adolescents seem to have an admirable capacity to deal with adversity, but their resilience is not without limits. As can be seen when comparing them to the competent adolescents (please compare values of the resilient column with values of the competent column in **Tables 4**, **5**), resilient adolescents scored lower in an important number of variables. In other words, even though adolescents in the resilient group showed very high levels of adjustment despite coming from adverse family environments, their adjustment levels could still be higher if they were aided in taking full advantage of their skills and if interventions were implemented at the source of adversity. Needless to say, reducing adversity in their family environment should also be a top priority. Additionally, certain studies have already warned on the risks of underestimating resilient adolescents' needs for support, since some of these adolescents, despite being classified as resilient for showing excellent competence levels according to external and behavioral indicators, can nonetheless suffer from elevated levels of emotional distress (Luthar et al., 1993).

A final more general consideration should probably be added. In the dynamic relationship between research and intervention underlined in this paper, it should be emphasized that interventions should not work with models that explain development and change from a lineal or even interactive perspective, since empirical evidence shows data in favor of transactional models that involve much more complex multilevel dynamic systems (Sameroff, 2010). Therefore, even though all recent school intervention efforts aimed at strengthening life skills to optimize development and along the way prevent risk behaviors deserve our most sincere recognition and applause (Springer et al., 2004), the intervention that we defend here should go further. This guiding conceptual framework leads to the claim that intervention in adolescence should be preceded by an ambitious systematic and multi-sector intervention starting at the beginning of life. In this vein, as already noted by Luthar and Cicchetti (2000), interventions should take into consideration and simultaneously work on different levels of influence (individual, family and extrafamily) and should begin as early as possible. Community work with families and current steps toward promoting positive parenting very early in the baby's life are good points of reference in this direction (Rodrigo et al., 2012).

In summary, this study emphasizes the enormous potential of research on resilient and vulnerable individuals, both for creating scientific knowledge and for designing intervention guidelines. For a long time psychology overlooked both phenomena (vulnerability and resilience), due to the predominant scientific

## REFERENCES

interest in central trends, i.e., toward what happened to the majority of people. Research was focused, on one hand, on those individuals that succumbed to adversity, and on the other, on those that showed strength as the result of having grown up surrounded by quality relationships. However, psychology must acknowledge the great deal that has been learnt since then by studying the limited percentage of people whose developmental trajectories apparently challenged the centraltendency hypotheses of that time: individuals who appeared to be strong and healthy despite adversity, as well as those who, despite growing up surrounded by strengths, seemed to be weak. Analysing the life trajectories of the first helps us to clarify what is desirable that all people have in their lives and the analysis of the life paths of the second, teaches us what is necessary to eradicate in all of them.

## ETHICS STATEMENT

The study was approved by the ethics committee of Comité Ético de Experimentación de la Universidad de Sevilla. Parents of adolescents participating in the study received a letter with information about the study and informed consent model.

## AUTHOR CONTRIBUTIONS

All authors conceived of the study, participated in its design and helped to draft the manuscript. Introduction was drafted by IG, methods and results were drafted by PR, FR and the discussion draft was done by IG, CM. All authors made suggestions and critical reviews to the initial draft and contributed to its improvement until reaching the final manuscript, which was read and approved by all authors.

## ACKNOWLEDGMENTS

This study has been funded by the Ministerio de Sanidad, Servicios Sociales e Igualdad [Ministry of Health, Social Services and Equality] of Spain.

processes. Dev. Psychopathol. 15, 139–162. doi: 10.1017.S0954579403 000087

Crockett, and R. K. Silbereisen (Cambridge: Cambridge University Press), 201–223.

Young People's Health and Well-being, Vol. 7. Health behaviour in schoolaged children: International report from the 2013/2014 survey. Health policy for children and adolescents, World Health Organization Regional Office for Europe, Copenhagen.

of 13- and 15-year old adolescents. Sch. Psychol. Int. 21, 195–212. doi: 10.1177/0143034300212006

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Moreno, García-Moya, Rivera and Ramos. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Animal Models of Maladaptive Traits: Disorders in Sensorimotor Gating and Attentional Quantifiable Responses as Possible Endophenotypes

Juan P. Vargas, Estrella Díaz, Manuel Portavella and Juan C. López\*

Animal Behavior and Neuroscience Lab, Department of Experimental Psychology, Universidad de Sevilla, Seville, Spain

#### Edited by:

Pietro Cipresso, IRCCS Istituto Auxologico Italiano, Italy

## Reviewed by:

Eddy A. Van Der Zee, University of Groningen, Netherlands Kenn Konstabel, National Institute for Health Development, Estonia

\*Correspondence:

Juan C. López<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Quantitative Psychology and Measurement, a section of the journal Frontiers in Psychology

Received: 07 September 2015 Accepted: 03 February 2016 Published: 19 February 2016

#### Citation:

Vargas JP, Díaz E, Portavella M and López JC (2016) Animal Models of Maladaptive Traits: Disorders in Sensorimotor Gating and Attentional Quantifiable Responses as Possible Endophenotypes. Front. Psychol. 7:206. doi: 10.3389/fpsyg.2016.00206 Traditional diagnostic scales are based on a number of symptoms to evaluate and classify mental diseases. In many cases, this process becomes subjective, since the patient must calibrate the magnitude of his/her symptoms and therefore the severity of his/her disorder. A completely different approach is based on the study of the more vulnerable traits of cognitive disorders. In this regard, animal models of mental illness could be a useful tool to characterize indicators of possible cognitive dysfunctions in humans. Specifically, several cognitive disorders such as schizophrenia involve a dysfunction in the mesocorticolimbic dopaminergic system during development. These variations in dopamine levels or dopamine receptor sensibility correlate with many behavioral disturbances. These behaviors may be included in a specific phenotype and may be analyzed under controlled conditions in the laboratory. The present study provides an introductory overview of different quantitative traits that could be used as a possible risk indicator for different mental disorders, helping to define a specific endophenotype. Specifically, we examine different experimental procedures to measure impaired response in attention linked to sensorimotor gating as a possible personality trait involved in maladaptive behaviors.

Keywords: dopamine, endophenotype, latent inhibition, mental disorder, prepulse inhibition

## INTRODUCTION

The criteria used by current diagnostic scales are based on the analysis of external symptoms of the patient. Disorders such as attention deficit with hyperactivity or mental disorders such as schizophrenia are diagnosed based on symptoms that, in many cases, require the patient to evaluate their intensity. This situation creates a serious problem for the diagnosis, given the large amount of subjective information handled by the psychologist or the psychiatrist (Robbins et al., 2012).

The problem of subjectivity and comorbidity in diagnostic errors are, in part, a consequence of the absence of biological markers to facilitate proper classification of the disorder. With relative ease, the diagnostic manuals such as the DSM or ICD propose a continuous change in the criteria for inclusion or exclusion of a disorder due largely to the heterogeneity and complexity of symptoms that define that disorder. These are so complex that patients with different symptoms might have the same diagnosis, a fact that significantly increases the difficulty of providing

proper treatment. This high comorbidity between various diseases indicates a clear deficiency in the classification system of mental disorders, preventing the identification of valid pathologies (Hyman, 2010). It is possible that the psychotherapeutic and pharmacological failures are largely due to this fact. Note for example that the therapeutic effectiveness of pharmacological treatments reaches approximately 50% (Wong et al., 2010).

Using a diverse group of pharmacological treatments to relieve disorders such as depression is also an indicator of the disparity of its diagnosis. For example, the use of inhibitors of serotonin reuptake is applied for a specific type of depressive symptoms, which differs from those used under MAO inhibitors or under tricyclics. The differential response of each patient to treatment indicates that disorders included in the same category should be treated with different principles. Alternatively, this phenomenon could be indirectly indicating that different types of disorders within a category may have a different biological basis.

An alternative to this traditional view is the characterisation of endophenotypes. An endophenotype is a quantitative measurable trait associated with a genetic predisposition (Gottesman and Shields, 1972, 1973). In contrast to the symptomatic view of psychopathology, the endophenotype analyses the characteristics that show possible brain vulnerability to suffer a specific type of disorder. The objective is the study and quantification of specific features that reflect a mental disorder associated with a biochemical sign (Hasler et al., 2006; Turetsky et al., 2007). Throughout its long history, the functional study of behavior in the laboratory has provided a number of indicators that could serve as markers for selective expression of the maladaptive behaviors. Applying this model to the field of psychopathology, mental disorders could be considered as extremes at one or both tails of these normal distributions (Miller and Rockstroh, 2013). From this point of view, psychopathology would view disorders as dimensional notions, and not as categories under a binary diagnosis (Hyman, 2010; Frances and Widiger, 2012; Morris and Cuthbert, 2012).

Here, we provide a set of measurable procedures sensitive enough to be used to identify possible endophenotypes developed from animal models. These endophenotypes are based on the correlation between brain processes and measurable responses of a subject that enable us to discriminate between different sets of symptoms, and facilitate new specific therapies. In addition, the evaluation of these traits could facilitate a more objective classification system of psychopathologies.

## HOW DOES THE USE OF AN ANIMAL MODEL CONTRIBUTE TO PSYCHOPATHOLOGY CLASSIFICATION?

The recent developments in genetics and epigenetics allow us to better approach understanding behavior and facilitate the understanding of mental disorders. The fact that some behaviors have a Mendelian basis, suggests the possibility of finding simple mutations that affect behavior in a relatively specific manner. However, there are only a small group of features known as Mendelian traits (or traits 1:1) in relation to genotype. Mental disorders such as depression or schizophrenia are clearly polygenic, or may also be generated by various mutant alleles of the same gene and specific environmental conditions, making the analysis of their causes a complex procedure (Zahn-Waxler et al., 1988; Winokur and Kadrmas, 1989; Kidd, 1997; Moldin, 1997; Owen, 2000; Torrey and Yolken, 2000; Goldman, 2012). Moreover, these illnesses are the result of the interactions of both genetic and epigenetic factors. And although we now have suitable tools for genotype analysis, the fact that these etiological factors -genes and environment- interact to produce similar phenotypes, significantly increases the difficulty to precisely define the specific weight of each one in the generation of behavior (Plomin and Rende, 1991). Identifying what groups of genes may contribute to the expression of a disorder is a long process of molecular genetics. However, the identification of relating groups of genes with specific traits is currently a more achievable goal.

The use of animal models for the study of personality traits, vulnerability to certain disorders or substance abuse dependence is an interesting strategy for developing behavioral protocols in the laboratory. Although in some cases these models could show poor face and predictive validity, the construct validity associated with the etiology or mechanism of the underlying disorder is usually high (O'Donnell, 2011). For example, animal models of schizophrenia have been successful in evaluating risk factors (see **Table 1**). This fact is crucial in order to develop new pharmacological treatments or genetic therapies. However, the reduced face validity is often a problem when applying to human models.

The development of endophenotypes is one alternative to try to improve this model. Taking advantage of high construct validity, we can develop sensitive tests for quantifying specific traits. Measures such as latent inhibition (LI) or prepulse inhibition (PPI) are, among others, easily quantifiable under controlled conditions in the laboratory. In addition, we can use the advantage of these procedures in a similar way in both animals and humans, and the results are easily extrapolated from an animal model to a human model (Le Pen et al., 2011). While PPI is a very simple procedure seeking to analyze early attentional gating mechanisms, the LI is a learning process related to selective attention and habituation to irrelevant information (Lubow and Gewirtz, 1995; Swerdlow et al., 1996; Braff and Swerdlow, 1997). Animal models indicate that problems in the expression of PPI or LI correlate with cognitive deficits such as working memory or alternation behavior, locomotion activity such as hyperactivity induced by a dopamine receptor agonist, and some negative symptoms also described in pathologies such as schizophrenia (Flagstad et al., 2004; Le Pen et al., 2006; Moore et al., 2006; Hazane et al., 2009). For example, patients with schizophrenia show these symptoms associated with a dysfunctional prefrontal cortex (PfC; Manoach, 2003; Silver et al.,

#### TABLE 1 | Several animal models have studied schizophrenia.

fpsyg-07-00206 February 17, 2016 Time: 20:28 # 3

Pharmacological models have used amphetamine, PCP or NMDA to simulate some of the symptoms. However, only two models have shown the illness as a developmental process. The neonatal ventral hippocampal lesion (NVHL) and MAM models showed marked maladaptive behavior when animals reached adulthood. Le Pen et al. (2011) and O'Donnell (2011) have described several behavioral procedures where we can find similar results with different techniques aimed at developing a dysfunctional PfC.

2003; Godsil et al., 2013), therefore, a behavioral test aimed to evaluate PfC function is a useful tool for an accurate differential diagnostic. The knowledge acquired in recent years on the use of a quantifiable measurement of these traits is boosting the development of unified models of diagnosis that include data from all levels, that is: genetics, biochemical, and behavioral levels.

But the question is, how can we contribute to this proposal? Consider, for instance, one of the most complex disorders, schizophrenia. Currently, schizophrenia is an umbrella term for a diverse group of disorders with possibly different etiologies. Focusing on PfC dysfunction, animal research has provided explanatory models to understand the possible development of this mental disturbance (O'Donnell, 2011; Godsil et al., 2013). Procedures aimed to alter gestation and fetal development such as the MAM model (Methylazoxymethanol), or techniques affecting the maturation process of PfC such as ventral hippocampus lesion in neonates, allow us to experimentally analyze this disorder (Waddington et al., 1999; Bramon et al., 2005; Chambers and Lipska, 2011). Both procedures show a clear PfC dysfunction (Tseng and O'Donnell, 2004, 2007). Cells unit recording studies indicate the possibility of a deficit in inhibitory GABAergic cells. This could be the cause of an excessive release of dopamine cells in the mesocortical system (Tseng and O'Donnell, 2004, 2007; O'Donnell, 2011; Godsil et al., 2013). This could be the reason that PPI or LI could be affected in these animal models and in schizophrenia. Both behavioral processes require an operative PfC for a normal expression. PPI and LI are very sensitive to disturbances in this structure. In this regard, a deficit in one or both processes could be a risk factor. On the whole, the characteristics of a dysfunctional PfC and the impairment in LI and PPI expression could be signs of a specific type of mental disorder, apart from the current model of mental illness where the disorder and its severity are expressed in terms of a scale filled out by the patient or a close family member.

## DOPAMINERGIC SYSTEM AS A SIGN OF A POSSIBLE RISK FACTOR

The function of the dopamine neurotransmitter has attracted great interest because of its relationship with the processes of learning and with several mental disorders such as schizophrenia, depression, ADHT or addiction to a substance of abuse (Robbins, 1992; Feldman et al., 1997; Weiner, 2003; Grace and Sesack, 2010; Simpson et al., 2010; Wise, 2010; Milad and Rauch, 2012; Díaz et al., 2015). The distribution of dopaminergic neurons is abundant in the central nervous system. The midbrain neurons and their efferences to the ventral striatum and PfC play a special role in the learning process (Robbins and Everitt, 1996). Dopaminergic pathways of the ventral tegmental area (VTA) toward the nucleus accumbens (NAc) are closely linked to the motivational processes of learning (Berridge and Robinson, 1998; Berridge, 2007). Many stimulant drugs, such as cocaine or amphetamine, operate in this place, and their function significantly increases the release or reduces the reuptake of dopamine in the system.

Dopamine receptors belong to the G-protein coupled receptors family. All these receptors possess seven transmembrane domains and five subtypes of dopamine receptors according to their molecular characteristics. These have been grouped into two pharmacological families according to the effect produced by agonists and antagonists. D1 family includes the subtypes D1 and D5 receptors. Both stimulate adenylyl cyclase, producing cAMP. On the other hand, D2 receptor family includes the subtypes D2, D3, and D4. These receptors inhibit the formation of cAMP. The D1 receptor is the most abundant in the central nervous system (Missale et al., 1998). The greatest concentration of this receptor is found in the neostriatum, NAc, amygdala, and substantia nigra. However, its affinity for dopamine is relatively low. The D2 receptor is found in high concentrations in the neostriatum (GABAergic neurons) in the NAc and hippocampus, and with a moderate density in the substantia nigra, cerebral cortex, globus pallidus,

thalamus, and hypothalamus. These data make D1 and D2 receptors specific targets for the study of cognitive, emotional, and motivational disorders. Electrophysiological studies have made important contributions concerning their functional activity in the mesolimbic system (O'Donnell and Grace, 1998; Moore et al., 1999; Grace, 2000; Floresco et al., 2001; Goto and Grace, 2008). These studies are of great relevance given the importance of these receptors in the processes of associative learning. Recent studies have shown that the D2 receptor is located in the projections of both the PfC and the amygdala in the form of autoreceptors (O'Donnell and Grace, 1995; Groenewegen et al., 1999; Goto and Grace, 2008). Specifically, D2 receptors are located in the presynaptic areas with the function of modulating the dopaminergic activity of the VTA over NAc through excitatory projections. That is why this receptor has been linked to the goal directed processes or controlled processes that require high attentional activity (O'Donnell and Grace, 1995; Goto and Grace, 2008). In contrast, the activity of D1 receptors in the mesolimbic system is different than the one described for D2. These are located in the post-synaptic cells of the NAc that receive glutamatergic afferences from the hippocampus and dopaminergic afferences from the VTA (O'Donnell and Grace, 1995; Groenewegen et al., 1999; Goto and Grace, 2008).

Disturbances in this system increase the risk of developing serious mental illness (O'Donnell, 2011; Godsil et al., 2013). Disorders such as schizophrenia have been linked directly to disturbances during brain development associated with the second trimester of pregnancy (Waddington et al., 1999; Bramon et al., 2005). Changes in the dopaminergic sensitivity and in the levels of dopamine or dopamine receptors volume could be the result of this process. Specifically, the family of D2 receptors seems to be more related to the disease process (Grace and Sesack, 2010; Simpson et al., 2010; Wise, 2010; Milad and Rauch, 2012), since the antagonists of these receptors such as haloperidol are effective in reducing symptoms (Lubow and Weiner, 2010). This is the reason why it was suggested a substantial increase of this type of receptors underlies this disorder as shown, for instance, in the studies of post-mortem tissue (Seeman and Nizkik, 1990).

Currently the drug treatment of disorders such as schizophrenia or ADHT act directly on the dopaminergic modulation in the brain. The changes that cause the blockade or stimulation of receptors of this neurotransmitter can be studied in the laboratory. Changes in the sensorimotor gating or selection of relevant stimulus of the environment can be considered as possible quantifiable traits directly related with the level of dopamine or dopaminergic receptors in the mesocortical and mesolimbic system. It is important to emphasize that injuries to PfC produce dopamine dysregulation and deficits in PPI and LI expression.

## PPI OF STARTLE RESPONSE. A SENSORIMOTOR GATING MEASURE

The startle response to an intense stimulus is a reflex behavior that has been described in all mammals studied. This is a fast-twitch of the skeletal muscle that leads to processing environmental stimuli and guiding the attention of the subject to a possible threat. This type of response is interesting because it has been associated with specific genes that appear in schizophrenia and as a possible trait with endophenotypical characteristics. For example, Vaidyanathan et al. (2014) studied the startle blink reflex using a very large human sample. Analyzing the startle response, they found a heritable specific pattern of behavior in the sample. In addition, this trait was associated with candidate genes in the endophenotype of schizophrenia. However, although it is an automatic reaction, the outcome can be modulated by the previous presence of a stimulus of lower intensity, therefore PPI is defined as the attenuation of the startle response to an intense pulse when it is preceded by a lower-intensity prepulse stimulus. When the prepulse is perceived, the mechanism of startle is inhibited and the animal displays a lower response (Graham, 1975; Lüthy et al., 2003; Larrauri and Schmajuk, 2006).

The problems with sensorimotor gating have been linked with the levels of dopamine in the NAc. The NAc integrates information from different structures, and even though dopamine modulation in NAc is dependent on mesocortical and mesolimbic systems (Ellenbroek et al., 1996; Larrauri and Schmajuk, 2006), the selective modulation of PfC afferent transmission is especially relevant. PfC afferences could facilitate behaviors oriented to specific goals, and a dopamine deficit could be involved in the incapacity to control the behavior (Goto and Grace, 2008).

It should be noted that the dopaminergic innervation of the PfC increases progressively through adolescence until adulthood. In this period, we can find modifications in density, shape and organization of the circuits (Kalsbeek et al., 1988; Benes et al., 2000; Seamans and Yang, 2004; Segalowitz and Davies, 2004; Manitt et al., 2011; Naneix et al., 2012). A mature circuit allows the dopaminergic neurons to fit their responses in an adaptive way, modulating their response in correlation with environmental changes (Spear, 2000; Tseng and O'Donnell, 2004, 2007; O'Donnell, 2011; Cass et al., 2013; Godsil et al., 2013). Currently, it is estimated that delays or alterations in the maturation process of the PfC dopamine system could be the cause of a large number of mental disorders (O'Donnell, 2011; Godsil et al., 2013). Specifically, a poor inhibitory capacity of the PfC over the NAc may be the major etiological factor in severe disorders such as schizophrenia. In fact, a deficit in the response to the pulse has been observed in different types of cognitive disorders, and it is specifically relevant in patients with schizophrenia (Braff et al., 1992, 2001a,b). Thus, a reduced PPI could be used as a trait for attentional deficit, besides being included as a schizotypy personality trait or a possible endophenotype of schizophrenia (Cadenhead et al., 2000; Braff, 2010; O'Donnell, 2011).

However, this trait is not specific for patients with schizophrenia but indicates a trait of vulnerability, and it is very clear in patients with schizophrenia. In this regard, the PPI deficit could be a necessary condition as a risk factor of schizophrenia, but it could not be sufficient by itself. The PPI deficit might be found in several disorders, and a pathological process such as schizophrenia needs other indicators.

## PPI, DOPAMINE AND IMPULSIVITY: A TRAIT, A NEUROTRANSMITTER AND A QUANTIFIABLE MEASURE NOT ASSOCIATED EXCLUSIVELY WITH SCHIZOPHRENIA

PPI is an easy system to measure in animals including humans. It has been used in animal models of schizophrenia, even though there are several studies where this procedure has been correlated with impulsivity traits. López et al. (2015) analyzed the PPI in rats classified as impulsive by an autoshaping procedure. Animals designated as sign trackers showed approach behavior to a conditional stimulus before delivery of unconditional stimulus. Specifically, for sign tracker animals (STa) the conditional stimulus could be a surrogate of the unconditional stimulus (Flagel et al., 2007; Robinson and Flagel, 2009). These kind of animals showed high levels of dopamine in NAc, but only in the presence of a conditional stimulus (Flagel et al., 2011). These data were consistent with the results of López et al. (2015) using a PPI procedure. In fact, the STa showed a lower PPI response to stimuli of low intensity. This reduced inhibitory ability of the STa showed a difference in the behavioral pattern in normal animals. Furthermore, these data may indicate that ST subjects may be more vulnerable to cognitive disorders in which dopamine is involved.

An important question about the vulnerability of STa to an impulsive behavior comes from specific activity of D2 subtype dopamine receptor. This receptor is located presynaptically on PfC terminals, and has been related with a selective modulation of the NAc to facilitate goal-directed behaviors (Goto and Grace, 2008). In addition, several psychopathologies associated with PfC have shown a deficit between this structure and the projections to NAc (O'Donnell, 2011). López et al. (2015) found a possible vulnerability from STa, since these animals showed a large sensibility of D2 receptor to the administration of an agonist such as quinpirole. This drug affected only STa performance, indicating that this type of trait differs from that observed in schizophrenia. It would be appropriate at this stage to point out the difference between an animal model of impulsivity and an animal model of schizophrenia regarding a dysfunction in PfC. These models have developed several protocols to evaluate attentional processes, and LI is a perfect candidate to discriminate between impulsivity and schizophrenia, because it allows for evaluating attention and executive functions, both specific to PfC function. Impulsive models of animals have found differences in incentive salience of the conditional stimulus, but not in attentional problems (Berridge and Robinson, 1998; Berridge, 2007) such as in schizophrenia models.

## LI, DOPAMINE AND ATTENTIONAL DEFICITS

LI is a learning process observed when the acquisition of a conditional response to a conditioned stimulus paired with a reinforcer is retarded if the same stimulus has previously been pre-exposed in the absence of the reinforcer. LI pharmacology has been associated almost exclusively with the use of an animal model of schizophrenia, and is therefore largely consistent with the pharmacology of schizophrenia (Lubow and Kaplan, 2010; Lubow and Weiner, 2010; Díaz et al., 2015). Specifically, because some of the symptoms of schizophrenia are characterized by an inability to filter, or ignore irrelevant or unimportant stimuli, an anomalous LI was proposed as a tool for the study of possible deficits of attention (Lubow and Weiner, 2010).

Again, dopaminergic activity of the NAc is the essential neural substrate for its expression. Animal models have shown that the primary role of the NAc is to restrict the expression of LI under certain conditions, and thus ensure that the LI is flexible and sensitive to environmental demands. It is important to highlight that, in the absence of modulator mechanisms responsible for restricting the expression of LI to the specific conditions, the effects of an irrelevant stimulus would be extremely robust and maladaptive. In this regard, LI might reflect the psychological processes that are impaired in schizophrenia, since most of the patients showed a reduced expression of this phenomenon. The identification of brain regions whose damage leads to disrupt the LI, joined with the studies of different parameters of expression in animal models, can provide important information on the dysfunctional brain circuits in schizophrenia. In previous decades it was suggested that some kind of hyperactivity of the dopaminergic systems represent a primary biochemical alteration in schizophrenia, which apparently constituted at least a plausible justification for biochemical alteration in this disorder (Iversen, 1976).

To gain insight into quantifiable attentional processes in LI, Díaz et al. (2014) analyzed the effect of various types of pre-exposure to a stimulus. The results indicated that there is a transfer from the ventral to the dorsal striatum in the processing of environmental information. In addition, the dorsomedial striatum is key to encode stimuli when these become irrelevant due to the lack of consequences after their presentation. A deficit in PfC could be the cause of a loss of transfer from ventral to dorsal striatum. Currently there are some laboratories working on this possibility. The inability to modulate dopamine in NAc does not allow for attentional disengagement, showing a persistent state of continuous attention.

The inability of encoding irrelevant information is one of the clearest deficits observed in patients with schizophrenia. Many modern learning theories assume that the amount of attention to a signal depends on how well the signal predicts the significant event of the past. Schizophrenia is associated with attention deficit and recent theories of psychosis have argued that positive symptoms such as delusions and hallucinations are related to a lack of selective attention. Patients with schizophrenia, who had severe positive symptoms, showed a clear difficulty in discriminating between predictive and non-predictive cues when compared to healthy adults. In addition, the rate of learning about non-predictive signals correlated with more severe positive symptoms in schizophrenia. These results suggest that the positive symptoms of schizophrenia were associated with increased attention, both to signals that are likely to be predictive and to those that are not predictive for causal learning. This

selective attention deficit was the result of learning irrelevant causal associations (Morris et al., 2013). In this regard, the development of specific protocols to differentiate the expression of LI could be used as a possible risk factor in the population.

However, the complexity of this disorder suggests the possibility of different etiological factors may underlie the disease. At present there are many contradictory results regarding whether IL is affected in schizophrenia. Lubow and Kaplan (2010) addressed this issue in a recent review. They emphasize the difference between positive and negative symptoms in relation to the expression of IL. For instance, patients with high levels of negative symptoms and low of positive showed a potentiated LI. This data is relevant, because they could be observing different symptoms of the illness or different illness.

## CONCLUDING REMARKS

The mesocortical input of dopamine and the PfC play a critical role in normal cognitive processes and in several neuropsychiatric diseases. This dopamine input regulates aspects of working memory, planning and attention, among others. Similarly, some disturbances may be the basis for a variety of positive and negative symptoms, and therefore of many of the cognitive deficits associated with mental illness. Despite intensive research, we still have a lack of understanding of the basic principles of dopamine activity in the PfC and all the mesolimbic system. In recent years, there has been considerable effort to understand the cellular mechanisms of modulation of dopamine neurons in the PfC and its relationship with behavior. However, the results of these efforts have often led to contradictions and disputes (Nieoullon, 2002). Given the complexity of the

## REFERENCES

function of the mesolimbic and the dopaminergic systems, the development of new tools will be necessary to facilitate discrimination of diagnostics and to provide a more objective assessment of the current classification systems. Namely, we suggest a shift or reconsideration in diagnostic scales adding other indicators. Clinical psychology has many tools to evaluate PfC dysfunction (for a review see Gruszka et al., 2010). We propose that PPI and LI could help to develop a new classification system, where we could distinguish between a psychotic illness such as schizophrenia by a dysfunction in PfC dopamine from other types of schizophrenia included in current scales. As we indicated above, current classification systems could be considering a diverse group of disorders under the same term of schizophrenia illness, and the different combination of positive and negative symptoms could indicate the severity of the disorder. The in depth analysis of these mechanisms, combined with genetic factors, is a new view that could facilitate the development of diagnostic categories in a more specific way and, therefore, a new therapeutic perspective in the future.

## AUTHOR CONTRIBUTIONS

All authors contributed similarly in the theoretical development of the manuscript.

## FUNDING

This research was supported by Ministerio de Economía y Competitividad (PSI2012- 32445 grant).

latent inhibition. Psychopharmacology 232, 4337–4346. doi: 10.1007/s00213- 015-4063-2

medial prefrontal cortex of juvenile rats. Neurobiol. Learn. Mem. 90, 339–346. doi: 10.1016/j.nlm.2008.04.005

Zahn-Waxler, C., Mayfíeld, A., Radke-Yarrow, M., McKnew, D. H., Cytryn, L., and Davenport, Y. B. (1988). A follow-up investigation of offspríng of parents with bipolar disorder. Am. J. Psychiatry 145, 506–509. doi: 10.1176/ajp.145.4.506

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Vargas, Díaz, Portavella and López. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Reading Ability Development from Kindergarten to Junior Secondary: Latent Transition Analyses with Growth Mixture Modeling

Yuan Liu1, 2, Hongyun Liu3, 4 \* and Kit-tai Hau5

1 Faculty of Psychology, Southwest University, Chongqing, China, 2 Key Laboratory of Cognition and Personality, Southwest University, Ministry of Education, Chongqing, China, 3 Beijing Key Laboratory of Applied Experimental Psychology, Beijing Normal University, Beijing, China, 4 School of Psychology, Beijing Normal University, Beijing, China, 5 Department of Psychology, Chinese University of Hong Kong, Hong Kong, Hong Kong

The present study examined the reading ability development of children in the large scale Early Childhood Longitudinal Study (Kindergarten Class of 1998-99 data; Tourangeau et al., 2009) under the dynamic systems. To depict children's growth pattern, we extended the measurement part of latent transition analysis to the growth mixture model and found that the new model fitted the data well. Results also revealed that most of the children stayed in the same ability group with few cross-level changes in their classes. After adding the environmental factors as predictors, analyses showed that children receiving higher teachers' ratings, with higher socioeconomic status, and of above average poverty status, would have higher probability to transit into the higher ability group.

#### Edited by:

Salvador Chacón-Moscoso, University of Seville, Spain

#### Reviewed by:

Ratna Nandakumar, University of Delaware, USA Jocelyn Holden Bolin, Ball State University, USA

> \*Correspondence: Hongyun Liu<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Quantitative Psychology and Measurement, a section of the journal Frontiers in Psychology

Received: 09 April 2016 Accepted: 11 October 2016 Published: 25 October 2016

#### Citation:

Liu Y, Liu H and Hau K-t (2016) Reading Ability Development from Kindergarten to Junior Secondary: Latent Transition Analyses with Growth Mixture Modeling. Front. Psychol. 7:1659. doi: 10.3389/fpsyg.2016.01659 Keywords: reading development, latent transition analysis, growth mixture model, dynamical systems, social rating

## INTRODUCTION

Reading is an important activity composing of various sub-skills which grow at different speed. In reality, students are nurtured in a dynamic system where they are not only self-organizing, but also interacting and being substantially affected by the psychosocial environment (Votruba-Drzal et al., 2008; Ding et al., 2013; Iruka et al., 2014). In such a system, one under-researched area is the effect of young students' social environment at home and at school on their learning to read behavior. The purpose of the present study, therefore, was to explain the pattern of reading development and to depict the relations between the developmental pattern and children's behavior as perceived by their parents and teachers. We applied and explored with the application of the latest appropriate statistical method—the latent transition analysis with growth mixture model on a large scale longitudinal survey (Early Childhood Longitudinal Study, Kindergarten Class of 1998-99, ECLS-K, Tourangeau et al., 2009).

## READING DEVELOPMENT: NON-CONTINUOUS PATTERN AND GROUPING

Reading can be seen as a way of meaning extraction which requires the working of different subskills on the text (Stahl, 1997; Clay, 2001; Rodgers, 2004). Recent research has highlighted the need to look more closely at the different skills. Word reading, therefore, might have to be separated from reading comprehension because the former includes some of the basic phonological abilities, letter knowledge, and shortterm memory (Muter et al., 2004; Kendeou et al., 2009), whereas the latter may need inference, monitoring, and knowledge of the story structure (Vellutino et al., 2007; Kendeou et al., 2009).

The mastery process of the language is, however, quite different for different subskills, such as for the constrained and unconstrained skills (e.g., Paris, 2005, 2009; Paris et al., 2005). Children's reading ability grows irregularly with spurts and stops (de Weerth et al., 1999). For example, with substantial individual differences, children's language competence may grow extremely rapidly before Spring Grade 1 but may decline thereafter (Palardy, 2010; Kieffer, 2012). Verhoeven et al. (2011) showed that the different patterns of the reading development were distinct from those around Grade 2.

Since the reading development pattern may differ from phase to phase, researchers are very interested in tracing and examining the growth trajectories. Paris (2005) suggested that when calibrating the unconstrained skills to the constrained skills, reading development follows a non-continuous growing pattern. This may not be easily detected when a simple linear growth modeling is used. Thus, for example, Quinn et al. (2015) have to use a two-part model to depict separately the developmental trajectories of the vocabulary knowledge and the reading comprehension through Grade 1 to Grade 4. Their bivariate model showed that vocabulary knowledge acted as a causal indicator of the subsequent reading comprehension growth. In summary, if researchers intend to depict the full picture of the reading developmental trajectory, a continuous growth model may not be suitable. Students stay at different "stages" with adaption to the new context using different reading skills.

A more sophisticated issue is that not all students share the same growing pattern across stages (Kaplan, 2002; Pianta et al., 2008). Empirically, these differential patterns in growth can be analyzed by (i) differentiating children into language ability groups and (ii) tracing their changes in groups as they progress in schools. For example, while most students develop rapidly before Spring Grade 1 and then slow down afterwards, some children may have a consistently slow growth rate (Kaplan, 2005; Kapland, 2008; Palardy, 2010).

The variation in growth rate is more likely to occur in the lower grades—as early as first grade (Ferrer et al., 2015), or around age of eight (Stanovich, 1986). Studies also showed that the dyslexic reader would probably grow at a slow pace that hardly enables the children to catch up with other typical readers (Grimm et al., 2010; Ferrer et al., 2015). The grouping phenomenon among slow developers is potentially harmful to them, since this low-ability-group students may have lower selfefficacy or motivation to learn let alone their ability shortage. Thus, it is important to find the conducive factors to facilitate these low ability students to "transit" into the higher competence group.

To solve the above challenging questions, we need a combined model to depict the various developing patterns with spurs and spots. Furthermore, as students' growth is determined by their current pre-exiting ability as well as by other influential factors in the environment, a dynamic systems model was adopted to analyze the interplay of these factors.

## READING DEVELOPMENT IN DYNAMIC SYSTEMS

To depict and explore the reading development, two issues should be noticed. Firstly, the sub-skills are correlated among each other. For example, Verhoeven et al. (2011) showed that the vocabulary at the beginning phase could predict word decoding and reading comprehension at the early stages of development. From Grade 2 onwards, word decoding competence in turn predicted later vocabulary development. Reading ability develops under the effects of the formal skills (Oakhill and Cain, 2012). Secondly, children live in a complicated environment where many of the external factors may influence the reading development. Thus, a dynamic systems view should be introduced when describing such a development.

The dynamic systems theory originates from natural science studies (for a review, see van Geert, 2003). According to this perspective, individual development is a consequence of the dynamic interactions within an individual and between an individual and the environment. In the last two decades, the dynamic systems view has been intensively discussed and widely applied, especially in language development research (Robinson and Mervis, 1999; van Geert and Steenbeek, 2005; Hollenstein, 2011; van Geert, 2011).

According to the dynamic systems, reading development can be described in terms of the change, interactions, and conjoint analysis of the individual and environment systems (Clay, 1977, 2001). For example, Clay (2001) believed that individuals would be able to construct and self-organize with their potential ability. They will push through the boundaries and improve their knowledge with their skills already mastered. So, proficient readers are able to mobilize the processing systems to fit the challenges of different texts by using environmental cues such as visual and motor stimulants. Kainz and Vernon-Feagans (2007) showed that the acquisition of reading ability was not isolated from the outside world. Kainz and Veron-Feagans worked with their colleague and developed a system of the dynamic circles involving the individuals, families, classrooms and school systems. This would be helpful to children's reading development and possibly help their transitions into higher ability groups (Kainz and Vernon-Feagans, 2007; Vernon-Feagans et al., 2008).

Among various factors in the social environment, teachers and parents' perception and attitude on students' study behaviors play important roles. These factors and their interplay vary from one individual to another and crucially affect students' academic outcomes. Ladd et al. (1999) Child × Environment model provides further explanation on how the quality of children's relationships can directly and indirectly influence school achievement from a dynamic system perspective. In the model, they show that children's initial behavior or the background factors influence their relationships with peers and teachers. Peer and teacher relationships in the school environment enhance or sometimes adversely affect student's achievement. For example, it is likely the students from lower socioeconomic backgrounds would be benefitted more by teachers who employed a more interpersonal approach of instruction, such as incorporating mixed group work, using peer tutoring, and solving problems with partners (Jung, 2014). Other studies have also consistently shown that high quality teacher-child relationship is conducive to high achievement (Davis, 2003; Pianta and Stuhlman, 2004; Hughes and Kwok, 2007; O'Connor and McCartney, 2007; Hughes et al., 2008). This relationship is also influenced by children's social behavior, such as their classroom engagement, which in turn affects children's achievement and academic outcomes (Cohen, 1997; Hughes and Kwok, 2007; O'Connor and McCartney, 2007).

From a dynamic systems perspective, teachers and parents could offer help to speed up children's transition into higher ability groups (Cho et al., 2013; Eyden et al., 2014). For example, teachers and parents' perceptions of students' ability and effort are closely related to children's academic achievement (Rytkönen et al., 2007; Natale et al., 2009; Longobardi et al., 2011). Particularly, since highly motivated children are perceived as talented and effortful (Upadyaya et al., 2012), parents and teachers' positive perceptions on children would be conducive to children's development. Upadyaya and Eccles (2015) showed that teachers' perceptions on ability and effort could predict the subsequent reading ability in a longitudinal study. It is thus quite important how teachers and parents perceive and show to the students their positive evaluation. This is because at the early elementary school years, children often assimilate teachers' perceptions in formulation the judgment of their own ability (Rosenholtz and Simpson, 1984; Tiedemann, 2000). From another perspective, children's educational aspiration partially reflected their parents and teachers' expectation on them as well, thus highlighting the importance of setting an appropriate but sufficiently high educational aspiration (Kuklinski and Weinstein, 2001; Herbert and Stipek, 2005).

## THE PRESENT STUDY

Two important issues would be addressed in the present study. Firstly, we were interested in the transition showing students' potency to develop their abilities. There are patterns shared by children in the same group in that they improved in their mastery of different reading skills, and thus grew together from one stage (lower ability groups) to the next (higher ability groups). Secondly and more importantly, we are interested in those environmental variables, especially the parents and teachers' perception on the children, that might facilitate such a transition.

Driven by the research questions, we had several research questions to examine under the dynamic systems theory. First, according to the integrated view of dynamic systems theory, a self-organizing process reflected an auto-regression development. We would examine whether and how extensive the subsequent ability status was determined by the previous status. Second, we would examine how much individual differences existed in students' growth trajectories. Finally, the contribution

of parents' and teachers' perception on students' growth would be examined.

## METHODS

## Participants

We used the publicly available data in the Early Childhood Longitudinal Study, Kindergarten Class of 1998-99 (ECLS-K) (Tourangeau et al., 2009) 1 to examine our research questions. This data set was developed under the National Center for Education Statistics (NCES). We chose the ECLS-K because it focused on children's early school experiences from kindergarten to Grade 8, and the longitudinal data displayed students' long-term trajectory development. Furthermore, ECLS-K adopted a multi-source, multi-method approach, which included interviews with parents, data from principals and teachers, information from student records, and direct assessment on children (including reading, mathematics and science cognitive items). The study was in alignment with the dynamic systems theory, in which various environmental variables were considered.

In total, seven waves of measures of reading assessment were available in the data set (C1R4RSCL–C7R4RSCL). As the data at Fall Grade 1 (C3R4RSCL) contained only 30% of the total sample, without jeopardizing the generalization of our conclusion, it was not included in our study. The remaining data points were from Fall Kindergarten, Spring Kindergarten, Spring Grade 1, Spring Grade 3, Spring Grade 5 and Spring Grade 8 (y1 − y6 in **Figure 1**).

1Retrieved from http://nces.ed.gov/ecls/kindergarten.asp.

Together with the parents' and teacher's questionnaires, 7803 children's questionnaires were available in our analyses.

There were 456 individuals with missing covariate values, and totally 1033 individuals with missing values on one or more of the covariates or indicators. For the missing rate of each variable, other than the slightly higher rate at y1 (7.0%), all other ranged from 0.5 to 4.8% only, with an overall average missing rate of 2.6%. Generally, the missing pattern of the present dataset could be treated as missing at random, so that the multiple imputation method by Mplus 7.0 (Muthén and Muthén, 2012) could be appropriately used. We generated 10 datasets, and the sample size 7803 was applied to the analyses with either the null model or with covariates being included. Basic information among the variables is shown in **Table 1**.

## Measures Reading Ability

The reading items were drawn from assessments used in other large-scale studies of similar-aged youth, including the National Assessment of Educational Progress (NAEP), the National Education Longitudinal Study of 1988 (NELS:88), the Education Longitudinal Study of 2002 (ELS:2002), the Texas Assessment of Knowledge and Skills (TAKS), and previous rounds of the ECLS-K. The reading items in ECLS-K were repeatedly measured with ten levels of the reading ability (see **Figure 2**). Each new wave was recalibrated to the former one and tests at each wave included some identical items so that the instruments at different waves could be linked on the same IRT scales (represented on the same unit of measurement). Specifically, in the collection of the Grade 8 data which was used in the present analysis, all the proficiency scores for the former levels were re-estimated to be pooled with the latest wave (see Tourangeau et al., 2009 for details).

### Social Rating

The social rating was the evaluation of the children's behavior by parents and teachers. The items were obtained from the Social Rating Scale (SRS) Approaches to Learning scales of the ECLS-K Parent and Teacher questionnaires. The SRS survey items comprised of parents' and teachers' ratings on how frequent and whether students had those study-related behaviors or not. The scale contained items such as intrinsic motivation, persistence/attention, and study habits. These ratings by teachers and parents, rather not self-reported by children, reflected students' social behaviors as perceived by the others, thus shows the interaction between students and their guardians.

A four-point scale was used, with "1 = never" and "4 = very often." Parents' SRS was collected annually except in the third, fifth, and eighth grades, while teachers' SRS was not collected at the eighth grade. In the study, the SRS in Fall Kindergarten was used to predict the transition of latent class. All these items were used as continuous variables in the present analyses (see Tourangeau et al., 2009).

## Background Information

While many studies had investigated the relationships among Socio-economic status (SES), poverty, race, minority and achievement, which were generally used as the background variables (e.g., Hattie, 2009; OECD, 2013). Specifically, SES referred to students' relative position in the social hierarchy, directly reflected the resources at home, and was often used as an important controlling variable. Both SES and poverty status measured important characteristics of the background family information and were thus chosen in our analysis (see Tourangeau et al., 2009).

## Analyses Procedure

### Model Definition

The latent transition analysis (LTA) (Prochaska and Velicer, 1997) was used to analyze the longitudinal transitions. The autoregression part of the LTA model described appropriately the self-organizing process under the dynamic systems. LTA also allowed us to add environmental covariates to moderate the autoregression process. With the LTA model, the measurement part could be further replaced according to different contexts and situations.

y1 − y6 = reading ability indicators from Wave 1 to Wave 6.

identifying upper- and lower-case letters of the alphabet; (2) Beginning Sounds, associating letters with sounds at the beginning of words; (3) Ending Sounds, associating letters with sounds at the end of words; (4) Sight Words, recognizing common "sight" words; (5) Words in Context, reading words in context; (6) Literal Inference, making inferences using cues that were directly stated with key words in text; (7) Extrapolation, identifying clues used to make inferences; (8) Evaluation, demonstrating understanding of author's craft and making connections between a problem in the narrative and similar life problems; (9) Evaluating Nonfiction, comprehension of biographical and expository text; and (10) Evaluating Complex Syntax, evaluating complex syntax and understanding high-level vocabulary (Tourangeau et al., 2009).

As an extension, we took advantage of the growth mixture model (GMM) to replace the measurement part of the original LTA (see Muthén et al., 2012). The GMM model could detect the growth of the reading skills by allowing individual differences in growth rate within each group, in contrast to the more stringent requirement with little individual differences allowed at each point of time.

According to earlier studies (Votruba-Drzal et al., 2008; Kieffer, 2012), the Spring Grade 1 (y3) was chosen to be the cut point of two stages. Thus, (y1 − y3) were the indicators of Stage 1 (kindergarten stage) with latent growth factors INT 1 and SLP 1 (intercept/initial state and slope) classified into latent groups (G1); whereas (y3 − y6) were the indicators of Stage 2 (primary to junior high school) with latent growth factors INT2 and SLP2 classified into groups (G2, see **Figure 1**).

#### Implementing the 3-Step Analysis

Specifically, in testing the effects due to environmental facilitating factors, covariates have to be introduced into the LTA. When adding these covariates, it is necessary to find appropriate ways to control for the characteristics that predict the membership in the different latent classes. Therefore, a 3-step Maximum Likelihood Method (referred to the 3-step approach in subsequent discussion) was used (see Collins and Lanza, 2010; Vermunt, 2010; Asparouhov and Muthén, 2014, see also Liu and Liu, 2015 for details).

In the first step in the 3-step LTA, GMM was used to get the classification of latent class for each stage, using the indicators at their respective stage only. For example, when estimating GMM at Stage 1, y1 to y3 were used as indicators, with y4 to y6 and the covariates serving as auxiliary variables; the proportions of each latent class were recorded. Similarly, GMM was conducted at Stage 2. In the second step, using the classification outcomes and the proportions given by Mplus, the classification error was computed for each latent class. With the odds ratio computed by the second step as the starting value of each latent class, LTA (G2 was regressed on G1) with the covariates (G1, G2, and the transition from G1 to G2, respectively, regressed on covariates) was applied (for detailed syntax, see Asparouhov and Muthén, 2014).

#### Model Selection Indices

The selection of the number of the latent classes has been a topic of much discussion (e.g., Nylund et al., 2007; Tofighi and Enders, 2008; Peugh and Fan, 2012). Most studies suggested that the BIC (Bayesian information criterion) value should be the best choice because it was a sample based index which also penalized sophisticated model. Tofighi and Enders (2008) in their simulation study showed that a sample size adjusted BIC (aBIC) was an even better index, and thus was used in our study. A smaller BIC/aBIC value indicated better model fit for nesting models. Besides, the entropy value was to measure how well a mixture model separated the classes. An entropy value close to 1 indicated good classification certainty. Asparouhov and Muthén (2014) suggested that an entropy level of 0.6 or higher might provide sufficient good classification for the 3-step method.

## RESULTS

## Selection of the Proper Model

As LTA was used in combination with GMM, the original GMM analyses were examined first. The piecewise GMM (y1 − y3 as Piece 1 and y3 − y6 as Piece 2) null model was chosen. We conducted the exploration analyses from 2 to 4 classes (see **Table 2**). The model fit indices, −2LL, BIC, and aBIC,

TABLE 2 | Model comparison and selection.

2c, 2 classes; 3c, 3 classes; 4c, 4 classes; 3-step, 3 step method.

consistently supported a 3-class model. Then we checked the class proportion to ensure the empirical significance. For the 2-class model, the proportion was 0.95 and 0.05 for each class; for the 3-class model, the proportion was 0.93, 0.05, and 0.02; for the 4 class model, two groups contained 0 individuals. It was evident that the third group in a 3-class model was so tiny (less than 5%) and would not contribute substantially and empirically to the model, so the 2-class model was retained.

We then conducted the GMM-LTA null model, using two stages of growth but without any covariate. Results showed that the 2-class model was the best according to the selection criteria (BIC and aBIC), with slightly worse but acceptable entropy value (see **Table 2**).

Finally, we conducted the 3-step GMM-LTA. BIC was 8099 with an entropy value of 0.92. The information criteria and entropy value indicated that the 3-step model was the best. The final model consisted of two groups at two stages, respectively (**Figure 3**).

## Grouping Membership

The classification results are shown in **Table 3**, and the parameter estimates for the growth factors are shown in **Table 4**. At Stage 1, most of the students were classified into the high ability group (90.9%, with initial ability of −1.10). The other 9.1% were in the low ability group with a lower initial status (−1.56). The growing rate (slope) of the high ability group (0.93) was slightly faster than that of the low ability group (0.89), but with quite similar pattern

High, High Ability Group; Low, Low Ability Group.

TABLE 4 | Parameter estimates of the growth factors for the final model solution.

seen from the trajectory in **Figure 3**. For Stage 2, from Spring Grade 1 to Grade 8, children in different classes had different growing rates. There were 95.2% in the high ability group with an initial ability of 0.41 and a growth rate of 0.12, while 4.8% in the low ability group had an initial ability of −0.48 and a growth rate of 0.19.

After grouping, there were two groups in each stage; so four possible combinations of sub-groups were formed (**Table 3**). Combination 1 (90.9%), which contains individuals classified in the high ability groups at both Stages 1 and 2, had the largest proportion. Combination 4 referred to individuals classified as low ability at both stages contained 4.8% of the population. This showed that most students' growth was stable (totally 95.7% of the population). There were about 4.3% of students being classified as Combination 3, who moved from the low ability group to the high ability group across time. No individual was in Combination 2, indicating that there was no reversed pattern (changed from high ability group to low ability group).

A transition probability showed that, once classified into the high group, students would have a 100% probability staying in the high ability group thereafter. In contrast, children starting in the low group would likely be in the low ability group at Stage 2 but had a considerably high probability to transit into the high ability group at Stage 2.

## Effect of the Environmental Factors

We set the significant level at p <.001 for this study with a large sample size. Results (see **Table 5**) showed that the covariates could predict the Stage 1's classification. Other than the background variables, both parents and teachers' higher ratings were associated with children's higher reading ability (with the lower ability group as the reference) at Stage 1. The Stage 2's classification could be predicted positively only by the parents' rating and SES level, with higher parents' rating and SES related to better children's performance (i.e., classified in the higher ability group). In contrast, higher teachers' rating was related to lower students' performance (being classified in the lower ability group; β = −6.02, odds ratio = 0.00).

Interactive effects with grouping transition were examined. It was found that when teachers' ratings (β = 5.66, odds ratio = 288) were more positive, then the children had a higher chance to transit from the lower to the higher ability group. Specifically, when the teachers' ratings were one unit higher, the low ability children at Stage 1 would have 288 times higher probability in transiting to the high ability group at Stage 2. However, the effects due to parents' ratings (β = −2.77, odds ratio = 0.06) and SES (β = −4.14, odds ratio = 0.02) were negligible.

aClassification was regressed on the covariates.

bStage 2 High ability group (cf. low ability group) was regressed on the covariates in Stage 1 low ability group.

## DISCUSSION

## Developing Patterns

The present study showed the advanced 3-step GMM-LTA model well described the complex longitudinal ECLS-K database set in the dynamic systems model. The developmental trend showed a fast grow from kindergarten to Spring Grade 1 and then a slowing down to a plateau on time beyond. A closer examination of the reading ability scores (**Figure 2**) showed that the formal five levels of reading proficiency were more related to Paris's constrained skills which were close perfection after Spring Grade 1. After this time spot, students continuously learned unconstrained skills. From **Table 4**, statistical evidence showed that the variances were much smaller at Stage 2 than those at Stage 1, especially for their growth rates which had little variance at Stage 2. This indicates the non-normal distribution across the development from kindergarten to Grade 8. It is necessary, therefore, to analyze the reading skills separately at different stage, where sub-skills developed with quite different speeds and patterns.

The grouping results were consistent with the literature (Grimm et al., 2010; Ferrer et al., 2015) in that two groups with different ability levels could be differentiated. The classification indicates that most of the students were classified in high ability group, either at Stage 1 or Stage 2. We can thus treat the high ability group as the reference "normal" developing pattern, since it contained more than 90% of the population. So, students classified in the lower ability group were those likely to have reading problems. According to Ferrer et al. (2015), the grouping differentiation could emerge as early as Grade 1; our study indicates that the grouping may emerge even earlier. However, students still had a considerable chance to transit into the higher ability group through the self-organizing progress (conditional probability was 0.47). Educators should pay more attention to children's early reading problems as early as possible before they develop into more serious language learning problems.

## Environmental Facilitators

We found that all the factors being examined had substantial effects on the grouping at Stage 1. Contradictory results were found, however, in the prediction of Stage 2 grouping/transition. The results showed that, parents' rating and SES positively predicted Stage 2 grouping, whereas they negatively predicted the transition. Vice versa, teachers' rating negatively predicted classification, but positively predicted the transition. These contradictions may reflect problems in the long-term prediction efficiency. When we took the transition prediction terms out of our model, all predictions on Stage 2 grouping showed negative estimates (ranged from −0.48 to −0.10), but with quite small or non-significant effects (odds ratios ranged from 0.70 to 0.91). So the social environmental variables collected at Wave 1 may have less predictive power to the subsequent ability, especially for a long-term growth (8.5 years). This is somehow similar to the previous study (Upadyaya and Eccles, 2015) which showed that teachers' perception of the effort of students could predict the subsequent reading ability with a small interval (1 year) only. Further investigations on the prediction power in long term studies would be useful.

As for the transition, the results showed that teachers' ratings had larger effects in predicting the transition probability than that of the parents'. This reveals that the teachers' ratings are probably more accurate as compared to those of the parents', which might be explained by the Child × Environment model (Ladd et al., 1999; Pianta and Stuhlman, 2004). To illustrate, the teacher-student relationship is a mediator influenced by the effect of school behavior and other background or cognitive variables on children's achievement. With the accurate perceptions, teachers may adopt more efficient approaches on students' learning. Teachers' interaction with students is thus playing as a proximal factor influencing the achievement influencing academic achievement more directly, while school entries (family variables) are distal factors. On the other hand, longitudinal studies show that teachers' perception of the students (either ability or effort) can predict subsequent children's self-concept (Natale et al., 2009); teachers are significant socialization agents whose perception greatly impact children's self-concept formation (Madon et al., 2001), and thus have a great impact on students ability. To summarize, we are alerted again of the important role of the teacher-student relationships, since students spend more time in school with their teachers when they progress in schools. In contrast, their after-class activities with parents may reduce so that the parents' evaluations become less accurate and predictive of children's reading performance.

As for background variables, SES is a potentially useful predictor of children's reading performance, particularly on grouping but not on transition. Meta-analysis (e.g., Hattie, 2009) showed that SES has a moderate impact (d = 0.57) on academic achievement. In the present research, we took SES as one of the important home background variables, used it as a controlling covariate, and showed that it had influence on grouping. It is logical, therefore, to pay greater attention to the reading development of students from lower SES background (e.g., Ladd et al., 1999; Jung, 2014).

## LIMITATIONS AND FUTURE DIRECTIONS

One possible limitation is that we used a two-stage model to analyze the data. This was mainly decided from the general trajectory of the reading growth of the data and findings from earlier studies (Kaplan, 2005; Kapland, 2008; Palardy, 2010; Kieffer, 2012). However, the problem is that the interval of the stage (especially at Stage 2) is quite large with the time points of data collection being several years apart. There is a possibility, therefore, that students grow in discernible stages crossing a long period of time. If the intervals of the data collection had been much smaller, we would have been more confident to use the growth modeling within each stage. An alternative is to use the non-linear model to build the GMM (e.g. Grimm et al., 2010). But it requires demanding measures. Future studies could further explore the possible trajectories of reading development, identify the proper cutoff for each stage, and describe the most suitable trend within each stage.

We also notice that long-term effects and growth patterns are less well predicted by the social environmental covariates. These covariates may include the home and teachers' social environmental factors which generally have smaller effects than those of more direct variables such as teaching and school (for meta-analysis, see Hattie, 2009). One possible direction of the future study is, therefore, to focus on the short-term prediction of a set of more comprehensive social environmental factors from schools (teachers, peers, etc.) and families (parents, etc.). Another possibility is to treat the covariate as a time-varying variable in multilevel structure (Vermunt et al., 1999; Bartolucci et al., 2011). That means, in our analyses, the social rating recorded at Kindergarten, Spring Grade 1 Spring and Fall Grade 5 can all be treated as multiple indicators affecting the transition at different time points. Especially under the condition with a large interval of measuring time, time-varying measures would then produce more accurate prediction.

## CONCLUSIONS

In summary, the study contributes in showing that: (i) the LTA-GMM fitted the data well; (ii) most of the children stayed in the same ability group with practically few cross-level class changes in the transition; (iii) children receiving higher teachers' ratings and with higher SES, and of above average poverty status, would have higher chance to transit into the higher ability level group. The findings supported the importance of the moderating effects of these social environmental facilitators on the patterns of children's reading development.

## AUTHOR CONTRIBUTIONS

YL contributes the most to the article. HL is the corresponding author who organizes and helps conducting the analysis. KH helps a lot providing useful suggestions on modeling and revising the article.

## FUNDING

The manuscript is supported by the National Natural Science Foundation of China (No. 31571152).

## ACKNOWLEDGMENTS

We acknowledge the financial support from the Collaborative Innovation Center of Assessment toward Basic Education Quality at Beijing Normal University.

## REFERENCES

of ability in math and reading? Educ. Psychol. 35, 110–127. doi: 10.1080/01443410.2014.915927

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2016 Liu, Liu and Hau. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Does Exercise Improve Cognitive Performance? A Conservative Message from Lord's Paradox

Sicong Liu\*, Jean-Charles Lebeau and Gershon Tenenbaum

*Department of Educational Psychology and Learning System, Florida State University, Tallahassee, FL, USA*

Although extant meta-analyses support the notion that exercise results in cognitive performance enhancement, methodology shortcomings are noted among primary evidence. The present study examined relevant randomized controlled trials (RCTs) published in the past 20 years (1996–2015) for methodological concerns arise from Lord's paradox. Our analysis revealed that RCTs supporting the positive effect of exercise on cognition are likely to include Type I Error(s). This result can be attributed to the use of gain score analysis on pretest-posttest data as well as the presence of control group superiority over the exercise group on baseline cognitive measures. To improve accuracy of causal inferences in this area, analysis of covariance on pretest-posttest data is recommended under the assumption of group equivalence. Important experimental procedures are discussed to maintain group equivalence.

#### Edited by:

*Jason C. Immekus, University of Louisville, USA*

#### Reviewed by:

*Evgueni Borokhovski, Concordia University, Canada Daniel Saverio John Costa, University of Sydney, Australia*

> \*Correspondence: *Sicong Liu<EMAIL_ADDRESS>
#### Specialty section:

*This article was submitted to Quantitative Psychology and Measurement, a section of the journal Frontiers in Psychology*

Received: *26 October 2015* Accepted: *06 July 2016* Published: *21 July 2016*

#### Citation:

*Liu S, Lebeau J-C and Tenenbaum G (2016) Does Exercise Improve Cognitive Performance? A Conservative Message from Lord's Paradox. Front. Psychol. 7:1092. doi: 10.3389/fpsyg.2016.01092* Keywords: exercise intervention, cognition, gain score analysis, ANCOVA, experimental group equivalence, false positive error, review

## INTRODUCTION

Does exercise enhance cognitive functioning in human beings? Meta-analyses have provided support for the beneficial effect of exercise on cognitive performance with effect sizes (g) ranging from 0.097 for acute exercise (Chang et al., 2012) to 0.158 for chronic exercise (Smith et al., 2010). Additionally, some authors have reported on several underlying mechanisms by considering evidence from behavioral and psychophysiological studies (for a review, see Hillman et al., 2008). These arguments seem to offer convincing evidence that exercise results in cognitive performance enhancement. The present study takes a critical perspective on this conclusion by assessing methodological characteristics of relevant evidence.

The most relevant evidence comes from exercise-cognition randomized controlled trials (RCT). First, these RCTs are considered clinical trials. According to World Health Organization (2015, para. 3) and the International Committee of Medical Journal Editors (Laine et al., 2007, p. 275), a clinical trial "is any research study that prospectively assigns human participants or groups of humans to one or more health-related interventions to evaluate the effects on health outcomes." Second, RCT is generally regarded as the best design for testing causal relationship because it makes group equivalence likely on all covariates (Freedman et al., 2007; Torgerson, 2009).

Several Exercise-cognition RCTs' findings support the causal relationship between exercise and cognition. For example, Chang et al. (2012) reported a larger effect size from RCTs (d = 0.19) compared to those from either quasi-experimental or observational designs (d = −0.02 and d = −0.14, respectively). These results have led some authors to conclude that exercise benefits cognition in a population ranging from children to older adults. Although such message is exciting, as Rubin (1974) cautioned, the relevance of evidence to answering research questions is not solely determined by the choice of research design but many other factors. Guided by this message, we examined exercise-cognition RCTs published in the past 20 years for potential methodological shortcomings.

## Why are Errors Possible

When analyzing pretest-posttest data from RCTs, researchers typically apply two group-comparison strategies to draw causal inferences: analysis of covariance and gain score analysis (Vickers and Altman, 2001; Van Breukelen, 2006). Analysis of Covariance (ANCOVA)1 refers to the approach where posttest scores are compared between groups, adjusting for baseline scores (as covariates in the linear model). Assuming baseline group equivalence, Analysis of Partial Variance is a parallel of this strategy (Cohen et al., 2013). The alternative approach, Gain Score Analysis (GSA), considers the gain score (i.e., posttest minus pretest) as the criterion for group comparison. Forms of GSA include repeated-measures analysis of variance (RM ANOVA), gain score t-test, and ANOVA of gain score, among others. Researchers' choice between ANCOVA and GSA often leads to disparate conclusions, an inconsistency historically termed "Lord's Paradox" (Lord, 1967).

Lord's paradox generated a lasting research effort and a consensus was reached among methodologists. The consensus is that, as long as baseline group equivalence is likely by randomization (such as in a RCT design), investigators should choose ANCOVA in drawing causal conclusions, because ANCOVA has a higher testing power and unbiased effect estimate compared to GSA (Cronbach and Furby, 1970; Huck and McLean, 1975; Holland and Rubin, 1983; Miller and Chapman, 2001; Senn, 2006; Van Breukelen, 2006). However, when baseline group equivalence is unlikely (such as in a quasi-experimental design), none of the statistical procedures enables to "control for" such a flaw, and thus no causal inferences should be attempted (Campbell and Stanley, 1963; Lord, 1967; Cronbach and Furby, 1970; Meehl, 1970; Senn, 2006; Van Breukelen, 2006). To reiterate previous points with an analogy, perfect dishes ("causal inferences") come from fresh raw food ("baseline group equivalence") and skillful cooking ("ANCOVA"), whereas no perfect dishes can be made from non-fresh food ("baseline group non-equivalence") irrespective of how skillful the cook is.

Given Lord's paradox conclusion, strong evidence for causal inferences can be obtained only if (a) baseline group equivalence is likely, and (b) pretest-posttest data are analyzed using ANCOVA. In practice, researchers never know with certainty that a given RCT has baseline group equivalence, but they can ascertain baseline group non-equivalence when group baseline measures show statistical differences. Assuming that baseline group equivalence is achieved by identifying no baseline group differences on any baseline measures (which is a likely portrait of a given RCT, at least on baseline measures statistically tested), researchers should choose ANCOVA over GSA when comparing groups.

One advantage of ANCOVA over GSA is an increased power. Originally, ANCOVA was not developed to "control" for anything but to enhance the testing power of independent variables (Miller and Chapman, 2001). For instance, assuming identical within-group variance between pretest and posttest, Van Breukelen (2006) quantified that ANCOVA requires only 75% of the sample size of ANOVA of gain score (i.e., one form of GSA) to detect the same effect when the pretest-posttest correlation is 0.50. The other advantage of ANCOVA over GSA has to do with effect estimate accuracy. Specifically, ANCOVA produces the unbiased effect estimate, whereas GSA can generate under- or over- estimated effect size depending on the situation of baseline group imbalance (Vickers and Altman, 2001).

Baseline group imbalance is the descriptive difference between groups on baseline measures. If an exercise-cognition RCT has only two groups (i.e., one control and one exercise group), the control group and the exercise group have an equal chance to perform better than the other descriptively on a cognitive task at baseline. The interpretation of "better" is task specific. For instance, a shorter reaction time (RT) is better in simple reaction time tasks (e.g., Stroop Color), whereas a larger value is better in time-limited memory tasks (e.g., Digit Symbol). If the control group has baseline superiority (control-BS) by having, for instance, a shorter RT than that of the exercise group on the Stroop Color task, the adoption of GSA will lead to an overestimate of exercise's benefits on cognition. Conversely, baseline exercise group superiority (exercise-BS) will generate an underestimated effect with the GSA method (Vickers and Altman, 2001).

Baseline measures are usually negatively correlated with gain scores (Cronbach and Furby, 1970; Knapp and Schafer, 2009), a phenomenon known as "regression to the mean" (Galton, 1886; Bland and Altman, 1994). In such instances, the bias due to GSA's failure to account for baseline group imbalance can be larger. As a consequence, the Type I error (i.e., false positive) from control-BS and Type II error (i.e., false negative) from exercise-BS are likely to happen when using GSA. For example, Bland and Altman (2011) reported that comparing a baseline with a followup separately in each group by using t-test (i.e., one form of GSA) could raise the actual alpha level to be as high as 0.50 when comparing two groups and 0.75 when comparing three groups, depending on the power of a specific test. To make things worse, Bland and Altman's results were based on one outcome measure. When an exercise-cognition RCT assesses the effect of exercise on multiple cognitive measures (which is often the case), the practice of having a presumable false positive threshold (e.g., α = 0.05) could turn meaningless.

## How to Test for Possible Errors

Rather than assessing the effect of exercise on cognition by considering potential moderators, a procedure common to metaanalytic studies, the focus of the present study was to determine whether exercise-cognition RCTs published in the past 20 years (1996–2015) involve false positives or false negatives due to GSA application in pretest-posttest data analysis. We provided

1 In this paper, the key distinction between ANCOVA and GSA is how researchers use the baseline measure. Although researchers can choose variables (e.g., age) as covariates in testing group difference on gain scores, these analyses are not what we mean by ANCOVA here.

a simple test to achieve this goal. Because group assignment was random, one would expect an equal chance for control-BS and exercise-BS on a certain cognitive measure. In other words, across all RCTs in our review, we expect half RCTs to show control-BS and the other half to have exercise-BS. In terms of a probability distribution, if we assume that X represents the number of RCTs showing control-BS, we would expect the probability of observing X, P (X), to follow a binomial distribution:

## P(X) ∼ Binomial(n, k)

where n represents the total number of RCTs examined and k symbolizes the expected probability (k = 0.5) of getting control-BS in a given exercise-cognition RCT2 . Similarly, if researchers select randomly between GSA and ANCOVA, we should expect the group comparison strategy to follow the same binomial distribution with the only difference being that X is representing the number of RCTs employing GSA.

In order to detect possible false positive and/or negative errors among exercise-cognition RCTs using GSA, we must check for independence between baseline group imbalance (i.e., control-BS vs. exercise-BS) an statistical significance test result (i.e., significant vs. non-significant). If baseline group imbalance were independent to statistical significance test result, we would expect X, representing the number of RCTs using GSA that showed control-BS, to continue following the binomial distribution when conditioned on statistical test result. Assuming that Y stands for the statistical test result that has two possible outcomes (i.e., significant or non-significant), we will have the following conditional binomial distribution:

$$\mathcal{P}(X|Y) \sim \text{Binomial}(n|Y,k).$$

where n is the total number of RCTs using GSA method and k still takes the value of 0.5.

To summarize, we had three hypotheses in the present study. First, we hypothesized that, among all the RCTs, half of them should demonstrate control-BS and the other half should show exercise-BS due to randomization. Second, we hypothesized that researchers, as a group, selected between GSA and ANCOVA without preference, and therefore half of the RCTs should employ GSA and the other half should use ANCOVA as a groupcomparison strategy. Lastly, we hypothesized that, when GSA-RCTs are counted separately based on whether they are positive (i.e., include at least one significant finding) or negative (i.e., include no significant findings), more control-BS (than exercise-BS) GSA-RCTs should be found in positive GSA-RCTs, whereas more exercise-BS (than control-BS) GSA-RCTs should be found in negative GSA-RCTs.

## METHODS

## Literature Search and Inclusion Criteria

The second author (J.-C. L.) conducted a literature search in April and May 2015 using SPORTDiscus, Web of Science, and Google

2We chose k instead of p to avoid confusion later when reporting the probability of our hypothesis testing.

Scholar databases. The search strategy utilized the following key words within full documents: (exercise OR physical activity) AND (cognition OR cognitive performance) AND randomized controlled trial. A manual search of reference list from key studies (e.g., meta-analysis) was also performed. The first author (S. L.) screened studies by title and abstract, then by full documentation. Trial authors were contacted when required information was missing. In total, 38 RCTs were considered for coding. However, five articles were excluded because they were missing information and corresponding authors were unable to respond to our request by July 1, 2015. The final set of studies consisted of 33 exercisecognition RCTs.

The following inclusion criteria were applied to the exercisecognition RCTs: (a) studies were published between January1996 and May 2015, (b) randomization is evident at the individual level, (c) the design included pre- and post-intervention measures on cognitive tasks such as perception, intelligence, academic achievement, memory, executive function, and cognitive impairment, (d) exercise intervention focused on aerobic, resistance training, or a combination of both, (e) studies included a passive control (e.g., waiting list), an active control (that can have a cognitive, physical, or social focus), or a combination of both (see Scherder et al., 2005), and (f) group differences were tested on cognitive measures. If multiple exercise intensities were used within an RCT, we regarded the group receiving the highest intensity as the exercise group and compared it to the control group. For example, if an RCT has two exercise groups (e.g., participants exercising at 60 and 70% of their VO2max) and a reading control group, the group exercising at 70% VO2max was selected as the treatment group and was compared to the control group. In addition, if the two exercise groups differed in exercise modality (i.e., aerobic training and resistance training), we compared each of these exercise groups to the control group, respectively, and the results were coded under a given RCT. Furthermore, if multiple interventions were included and at least one of the groups received an intervention focusing on elements other than exercise (e.g., cognitive training), only the exercise group was considered as a treatment group and was compared to the control group. Finally, if multiple follow-up measurements were available after the intervention period, we chose the immediate post-intervention measurement as the post-test measure. Details of the literature search and study selection were shown in a flowchart (**Figure 1**).

## Coding and Reliability

The first two authors discussed and settled coding variables to be included in the coding sheet. One author (S. L.) independently coded all the studies. The coded variables focused on the information relevant to the focus of the study, which is to check potential Type I and Type II errors in exercisecognition RCTs. Therefore, for every cognitive task, we coded the targeted cognitive process (e.g., executive functioning), baseline group imbalance (control-BS vs. exercise-BS), and statistical test result (significant vs. non-significant). Other key methodological information were also coded including (a) group-comparison strategy in pretest-posttest data analysis (ANCOVA vs. GSA), (b) the form of control (passive vs. active), (c) the presence or

absence of randomization procedure, (d) testing baseline group equivalence on cognitive measure(s), (e) the use of blinding procedures (i.e., single-, double-, or triple-blind), (f) explicit inclusion of intention-to-treat (ITT) analysis, (g) presence of a priori power analysis, (h) total participant number and number of groups (enabling participant number per group to be calculated), and (i) the presence or absence of pre-registering the trial. **Table 1** displays the coded information for each study included.

Eleven articles (33.3% of total) were randomly selected and separately coded to produce inter-coder reliability. A research assistant blinded to the study purposes completed the coding. Inter-rater reliability was calculated using Cohen's Kappa coefficient for each coding variable (**Table 2**). Following Landis and Koch's (1977) recommendations, we considered Kappa values between 0.61 and 0.80 as substantial and above 0.80 as very good. All the coded variables in the present study showed very good reliability. Coding discrepancies were resolved by re-visiting studies and discussion.

## RCT Count and Statistical Analysis

We categorized and counted all the RCTs regarding their groupcomparison strategy and baseline group imbalance. For groupcomparison strategy, we categorized a given RCT into GSA-RCT if it used gain scores as the criterion in comparing groups. We classified an RCT as ANCOVA-RCT if the outcome variable was the post-test score while controlling for baseline score as covariate, or if analysis of partial variance was used.

Although we coded baseline group imbalance for every cognitive task within an RCT, we later counted the number of RCT regarding their baseline group imbalance favorableness (control-BS vs. exercise-BS). This ensured an equal weight for every RCT given their varying number of cognitive measures. For example, one RCT reported 42 cognitive measures but several RCTs reported only one cognitive measure. In this case, the 42-task RCT would be over-weighted if the count were made at the task level. We applied the "dominance rule" in judging whether a given RCT favors control-BS or exercise-BS. For example, if an RCT used four cognitive measures, we coded it as favoring control-BS if three of the four measures had better performing control group at baseline. Due to within-study measurement dependence, multiple cognitive measures tended to show homogeneous results with respect to baseline group imbalance. Among 33 RCTs, we applied the dominance rule to 14 RCTs. Two RCTs showed equal number of cognitive measures between control-BS and exercise-BS, and thus were dropped from the final count on baseline group imbalance.

We also made "conditional count" among GSA-RCTs. First, all the RCTs were screened for GSA employment. Then, GSA-RCTs were categorized as either positive (i.e., having at least one significant finding) or negative (i.e., having no significant TABLE 1 | Study coding sequenced by group comparison strategy and study positivity.

*Year, Year of publication; Grp, (T/C), Baseline group imbalance (total count/conditional count); Sig., Study positivity (at least one significant test result identified by corresponding RCT); Anal., Group comparison strategy in pretest-posttest data analysis; Control, Form of control group; Random, Described random allocation procedures; Test Base, Tested baseline group equivalence on cognitive measures; Blind, Blinding procedures reported; ITT, Explicitly mentioned following intention-to-treat principle; Power, Performed a priori power analysis; N (Grp.), Total sample size (number of groups); Prereg., Pre-registered the trial. Liu-Ambrose et al. (2012) reported data dependence with Liu-Ambrose et al. (2010); E, Exercise-BS; C, Control-BS; Y, Yes; N, No; GSA, Gain score analysis; ANCOVA, Analysis of covariance; A-Cog., Active control with a cognitive focus; A-Phy., Active control with a physical focus; A-Soc., Active control with a social focus; A-Mix, Active control with more than one focus (e.g., cognitive and social); P, Passive control, Both, A control group consisting both actively and passively controlled participants; Single, Single blinding procedure (i.e., cognitive task assessors).*

findings). The "conditional count" process was very similar to the previous count except that a RCT's baseline group imbalance was decided only on those cognitive measures fitting the positive/negative category. Specifically, if a GSA-RCT had at least one significant result (i.e., positive study), its baseline group imbalance was determined on all significant cognitive measures. If a GSA-RCT had no significant results (i.e., negative study), all its cognitive measures were included to determine its baseline group imbalance. These decisions were made for two reasons. First, some positive RCTs employed only one cognitive task (which reached statistical significance). Second, we could bias the negative RCT count regarding baseline group imbalance if we retained the non-significant measures from positive RCTs and recycled them in the negative RCT count.

During the "conditional count," we applied the dominance rule to only one GSA-RCT because it included one cognitive measure supporting control-BS and one cognitive measure with description-wise equal baseline between the control and exercise group; and thus it was counted as control-BS. In addition, one positive GSA-RCT reported a control-BS on one cognitive measure and exercise-BS on the other cognitive measure. This RCT was subsequently classified as neutral and was dropped from the final conditional count. We used the R version 3.2.0 (R Core Team, 2015) to estimate the probability of obtaining those counts based on continuity-corrected binomial distributions. Whereas the first two hypotheses had two-sided tests, the third hypothesis had one-sided test. The alpha level was set at 0.05.

## RESULTS