HOW CAN EDUCATION BETTER SUPPORT THE MENTAL HEALTH & WELLBEING OF YOUNG PEOPLE? CONTRIBUTIONS FROM DEVELOPMENTAL PSYCHOPATHOLOGY & EDUCATIONAL EFFECTIVENESS RESEARCH

EDITED BY : James Elliot Hall and Jana Marinka Kreppner PUBLISHED IN : Frontiers in Education, Frontiers in Psychology and Frontiers in Psychiatry

#### Frontiers eBook Copyright Statement

The copyright in the text of individual articles in this eBook is the property of their respective authors or their respective institutions or funders. The copyright in graphics and images within each article may be subject to copyright of other parties. In both cases this is subject to a license granted to Frontiers. The compilation of articles constituting this eBook is the property of Frontiers.

Each article within this eBook, and the eBook itself, are published under the most recent version of the Creative Commons CC-BY licence. The version current at the date of publication of this eBook is CC-BY 4.0. If the CC-BY licence is updated, the licence granted by Frontiers is automatically updated to the new version.

When exercising any right under the CC-BY licence, Frontiers must be attributed as the original publisher of the article or eBook, as applicable.

Authors have the responsibility of ensuring that any graphics or other materials which are the property of others may be included in the CC-BY licence, but this should be checked before relying on the CC-BY licence to reproduce those materials. Any copyright notices relating to those materials must be complied with.

Copyright and source acknowledgement notices may not be removed and must be displayed in any copy, derivative work or partial copy which includes the elements in question.

All copyright, and all rights therein, are protected by national and international copyright laws. The above represents a summary only. For further information please read Frontiers' Conditions for Website Use and Copyright Statement, and the applicable CC-BY licence.

ISSN 1664-8714 ISBN 978-2-88963-146-9 DOI 10.3389/978-2-88963-146-9

#### About Frontiers

Frontiers is more than just an open-access publisher of scholarly articles: it is a pioneering approach to the world of academia, radically improving the way scholarly research is managed. The grand vision of Frontiers is a world where all people have an equal opportunity to seek, share and generate knowledge. Frontiers provides immediate and permanent online open access to all its publications, but this alone is not enough to realize our grand goals.

#### Frontiers Journal Series

The Frontiers Journal Series is a multi-tier and interdisciplinary set of open-access, online journals, promising a paradigm shift from the current review, selection and dissemination processes in academic publishing. All Frontiers journals are driven by researchers for researchers; therefore, they constitute a service to the scholarly community. At the same time, the Frontiers Journal Series operates on a revolutionary invention, the tiered publishing system, initially addressing specific communities of scholars, and gradually climbing up to broader public understanding, thus serving the interests of the lay society, too.

#### Dedication to Quality

Each Frontiers article is a landmark of the highest quality, thanks to genuinely collaborative interactions between authors and review editors, who include some of the world's best academicians. Research must be certified by peers before entering a stream of knowledge that may eventually reach the public - and shape society; therefore, Frontiers only applies the most rigorous and unbiased reviews.

Frontiers revolutionizes research publishing by freely delivering the most outstanding research, evaluated with no bias from both the academic and social point of view. By applying the most advanced information technologies, Frontiers is catapulting scholarly publishing into a new generation.

#### What are Frontiers Research Topics?

Frontiers Research Topics are very popular trademarks of the Frontiers Journals Series: they are collections of at least ten articles, all centered on a particular subject. With their unique mix of varied contributions from Original Research to Review Articles, Frontiers Research Topics unify the most influential researchers, the latest key findings and historical advances in a hot research area! Find out more on how to host your own Frontiers Research Topic or contribute to one as an author by contacting the Frontiers Editorial Office<EMAIL_ADDRESS>
# HOW CAN EDUCATION BETTER SUPPORT THE MENTAL HEALTH & WELLBEING OF YOUNG PEOPLE? CONTRIBUTIONS FROM DEVELOPMENTAL PSYCHOPATHOLOGY & EDUCATIONAL EFFECTIVENESS RESEARCH

Topic Editors:

James Elliot Hall, University of Southampton, United Kingdom Jana Marinka Kreppner, University of Southampton, United Kingdom

Image: Shutterstock.com/Konstantin Chagin

How can education better support the mental health & wellbeing of young people? Research in the 1970s that addressed this question has since proven seminal to the development of two co-existing fields of research that continue to offer mutually informative insights: Developmental Psychopathology (DP) and Educational Effectiveness Research (EER). DP and EER share the common agenda of understanding factors that relate to individuals' learning and development: DP focuses on the individual learning and developing in context, EER investigates the educational systems, structures, and processes that shape how individuals learn and develop.

Given the complementarity of DP and EER, it is somewhat surprising that they have rarely joined forces and synthesised knowledge to develop a fuller understanding of the roles educational contexts play in the mental health and wellbeing of students. This Research Topic aims to stimulate such collaboration.

Citation: Hall, J. E., Kreppner, J. M., eds. (2019). How Can Education Better Support the Mental Health & Wellbeing of Young People? Contributions From Developmental Psychopathology & Educational Effectiveness Research. Lausanne: Frontiers Media SA. doi: 10.3389/978-2-88963-146-9

# Table of Contents

Catherine Malboeuf-Hurtubise, Geneviève Taylor, Linda Paquette and Eric Lacourse

Danielle Evans, Giulia A. Borriello and Andy P. Field

*77 School-Related Social Support is Associated With School Engagement, Self-Competence and Health-Related Quality of Life (HRQoL) in Turkish Immigrant Students*

Meryem Demir and Birgit Leyendecker

*87 An Evaluation of a Mental Health Promotion Programme to Improve Emotional, Social and Coping Skills in Children and Young People Attending Special Schools*

Gemma L. Unwin, Biza Stenfert Kroese and Jessica Blumson

*102 School Peer Non-academic Skills and Academic Performance in High School*

Gregory J. Palardy

*118 Bad Boys and Mean Girls: Callous-Unemotional Traits, Management of Disruptive Behavior in School, the Teacher-Student Relationship and Academic Motivation*

Jennifer L. Allen, Elisabeth Bird and Celine Y. Chhoa

*134 Shyness and Learning Adjustment in Senior High School Students: Mediating Roles of Goal Orientation and Academic Help Seeking*

Yingmin Chen, Liang Li, Xiaoyi Wang, Yingli Li and Fengqiang Gao

*147 Beyond Academics: A Model for Simultaneously Advancing Campus-Based Supports for Learning Disabilities, STEM Students' Skills for Self-Regulation, and Mentors' Knowledge for Co-regulating and Guiding*

Consuelo M. Kreider, Sharon Medina, Mei-Fang Lan, Chang-Yu Wu, Susan S. Percival, Charles E. Byrd, Anthony Delislie, Donna Schoenfelder and William C. Mann

*164 Frontal Alpha EEG Asymmetry Before and After Positive Psychological Interventions for Medical Students*

Yuan-Yuan Xu, Zheng-Quan Feng, Yuan-Jun Xie, Jin Zhang, Shu-Hao Peng, Yong-Ju Yu and Min Li

# Editorial: How Can Education Better Support the Mental Health and Wellbeing of Young People? Contributions From Developmental Psychopathology and Educational Effectiveness Research

James Elliot Hall 1 \* and Jana Marinka Kreppner 2

*1 Southampton Education School, University of Southampton, Southampton, United Kingdom, 2 School of Psychology, University of Southampton, Southampton, United Kingdom*

Keywords: mental health, well-being, education, developmental psychopathology, educational effectiveness research

#### **Editorial on the Research Topic**

#### **How Can Education Better Support the Mental Health and Wellbeing of Young People? Contributions from Developmental Psychopathology and Educational Effectiveness Research**

Research in the 1970s that addressed this question (e.g., Rutter et al., 1970, 1976, 1979) has since proven seminal to the development of two distinct fields of research that continue to offer mutually informative insights: Developmental Psychopathology (DP) and Educational Effectiveness Research (EER).

Influenced by a modern medical biopsychosocial model (Engel, 1977), Developmental Psychopathology is concerned with the study of developmental processes in biological and psychological systems to inform understanding of adaptation and maladaptation - knowledge used to develop better prevention and treatment of psychopathologies (Rutter and Sroufe, 2000; Toth and Cicchetti, 2010). This focus on the drivers of development is shared by EER as it investigates, "all the factors within schools in particular, and the educational system in general, that might affect the learning outcomes of students in both their academic and social development" (p. 197; Reynolds et al., 2014). DP and EER share the common agenda of understanding factors that relate to individuals' learning and development: DP focuses on the individual learning and developing in context, EER investigates the educational systems, structures, and processes that shape how individuals learn and develop.

Given the complementarity of DP and EER, it is therefore somewhat surprising that they have rarely joined forces and synthesized knowledge to develop a fuller understanding of the roles educational contexts play in the mental health and well-being of students. This Research Topic aims to stimulate such collaboration.

Together, the 11 contributions in this Research Topic demonstrate the complementarity of DP and EER via their focus on how individuals learn and develop in context (educational and otherwise). Most contributions focus on individual learners and consider differences over time, and concentrate more upon the immediate environments around the learner than those more distant (e.g., broader society and culture). **Table 1** maps the 11 studies onto the ecologies and systems that surround the learner. Two theoretical frameworks are used for this mapping to further illustrate the complementarity of the fields—one prominent in DP (Bronfenbrenner, 1986) and one in EER (Creemers and Kyriakides, 2006).

#### Edited and reviewed by:

*Douglas F. Kauffman, Medical University of the Americas – Nevis, United States*

> \*Correspondence: *James Elliot Hall<EMAIL_ADDRESS>
#### Specialty section:

*This article was submitted to Educational Psychology, a section of the journal Frontiers in Education*

Received: *30 July 2019* Accepted: *19 August 2019* Published: *29 August 2019*

#### Citation:

*Hall JE and Kreppner JM (2019) Editorial: How Can Education Better Support the Mental Health and Wellbeing of Young People? Contributions From Developmental Psychopathology and Educational Effectiveness Research. Front. Educ. 4:93. doi: 10.3389/feduc.2019.00093*

TABLE 1 | The focus of the eleven studies in this Research Topic mapped on to the ecologies and systems understood to surround the learner.

We organize the contributions in this Research Topic by adopting a prospective developmental perspective and according to age-related levels of education. We begin with a contribution by Melhuish and Gardiner who analyse longitudinal data from 600 contemporary Early Childhood Education and Care (ECEC) settings, and draw on results spanning 20 years, to argue that educational policy change may improve child development via changes to the quality of ECEC settings.

Three papers are concerned with primary school-age children. Orylska et al. investigate the learning trajectories during an 8 week working memory training intervention of 126 children (aged between 5.9 and 6.5 years) of which 65 met diagnostic criteria for ADHD. Findings demonstrated that learning curves for most children (both ADHD and non-ADHD) followed an inverted U shape suggesting that young children showed initial learning but did not benefit from extended training. Malboeuf-Hurtubise et al. present results from a series of 8-week Mindfulness-Based Interventions (MBIs) carried out with 13 primary school students taught in special education classrooms due to learning needs arising from clinically diagnosed psychiatric disorders. No firm conclusions regarding benefits from the MBIs could be drawn. By consequence, this study reflects the need for caution regarding the use of MBIs in such settings until further data is available. In the third paper, Evans et al. review literature on the impact of transitioning from primary to secondary education on student attainment and wellbeing. They identify a number of putative risk and protective factors to help inform further research on how education can contribute to mental health and well-being during primary-tosecondary transitions.

Two papers consider both primary and secondary school students. Demir and Leyendecker analyse data from 161 primary and secondary school-aged Turkish immigrants in Germany. Results highlight that across primary and secondary school years, children are differentially responsive to the potential supportive roles of immediate educational environments, peers, and teachers. Unwin et al. evaluate the effectiveness, acceptability, and feasibility of a school-based mental health programme delivered in 8 special schools to 53 students aged 5–15 years using a within-group repeated measures design. While teacher—but not parent—ratings supported the feasibility of the programme, further evidence is required from studies that include control groups. Still, the study provides important pointers for future research into how schools can support students' mental health and well-being.

Three papers focus on secondary school students. In a study of 2,541 high school students, Palardy demonstrates that the nonacademic attributes of participants' peers, including behavioral engagement, cognitive engagement, conscientiousness, selfefficacy, and hope, were important for academic achievement beyond participants' own non-academic attributes. Importantly, this paper shifts attention to the influences of peers on young people's educational experiences. Allen et al. recruited 437 secondary school students and 12 teachers for a mixedmethod cross-sectional study investigating difficulties faced by students high in Callous-Unemotional (CU) traits. Their findings highlight the need for adaptation of school-based interventions to lessen the risk of disruptive behavior and poor motivation in this high-risk subgroup of antisocial children. In a crosssectional study of 677 senior high school students, Chen et al. demonstrate that the association between student shyness and reduced learning adjustment may be mediated via students' goal orientation and academic help seeking. Findings indicate possible targets for interventions designed to benefit the well-being and achievement of shy students.

Finally, two contributions concern students in higher education. Using a mixed-method within-group evaluation design, Kreider et al. report results from a multilevel intervention to support educational opportunities for 173 undergraduate Science, Technology, Engineering, and Math (STEM) students with learning disabilities. Their findings demonstrate possible benefits from such multilevel interventions for students' educational progress and well-being. Xu et al. report results from 55 undergraduate medical students who participated

### REFERENCES

in a randomized controlled trial investigating the impact of positive psychological interventions (PPIs) on subjective well-being, depression, anxiety, and neural correlates assessed via electroencephalography (EEG). Results suggest PPIs may improve subjective well-being and reduce emotional distress by promoting adaptive emotional regulation that was associated with change in frontal alpha EEG asymmetry. This paper demonstrates one potential neurological mechanism underpinning the effects of PPIs—a form of intervention common in educational settings (see, Shankland and Rosset, 2017)—on mental health.

With this collection of papers in this Research Topic we hope to stimulate further research collaborations across DP and EER that will improve understanding of the roles educational contexts play in the mental health and well-being of students.

# AUTHOR CONTRIBUTIONS

All authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

#### ACKNOWLEDGMENTS

Our thanks to Dr. Chris Downey<EMAIL_ADDRESS>and Dr. Ariel Lindorff<EMAIL_ADDRESS>for commenting on draft versions of this editorial.

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Hall and Kreppner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# Structural Factors and Policy Change as Related to the Quality of Early Childhood Education and Care for 3–4 Year Olds in the UK

Edward Melhuish1,2 \* and Julian Gardiner 1

1 Department of Education, University of Oxford, Oxford, United Kingdom, 2 Department of Psychological Sciences, Birkbeck, University of London, London, United Kingdom

In many developed countries it is now the norm for preschool children to spend time outside the home in early childhood education and care (ECEC). Research indicates that attending ECEC can promote longer-term positive life outcomes, which is more likely when the ECEC is of higher quality. In a UK study of 600 ECEC group settings for 3–4 year olds, staff qualifications were predictive of quality at private (for profit) settings. For voluntary (not for profit) settings, which were more homogenous in staff qualifications, having a staff training plan and a better staff to child ratio were found to be significant predictors of quality. However, state funded nursery classes/schools, which tend to have less favorable staff to child ratios than private and voluntary settings, also tended to have higher process quality ratings, where the presence of more highly qualified staff apparently allowed quality to be maintained with a larger number of children per staff member. A comparison of equivalent quality data from separate UK studies, conducted before and after a period of substantial policy change in relation to ECEC quality, indicated that policy change may have powerful effects in improving ECEC quality with implications for long-term child, and potentially adult, well-being.

#### *Edited by:*

Douglas F. Kauffman, Medical University of the Americas–Nevis, United States

#### *Reviewed by:*

Virginia Tompkins, The Ohio State University at Lima, United States Emily Tomayko, Oregon State University, United States Pablo Fernández-Berrocal, University of Málaga, Spain

*\*Correspondence:*

Edward Melhuish<EMAIL_ADDRESS>
#### *Specialty section:*

This article was submitted to Educational Psychology, a section of the journal Frontiers in Education

*Received:* 12 July 2018 *Accepted:* 09 April 2019 *Published:* 07 May 2019

#### *Citation:*

Melhuish E and Gardiner J (2019) Structural Factors and Policy Change as Related to the Quality of Early Childhood Education and Care for 3–4 Year Olds in the UK. Front. Educ. 4:35. doi: 10.3389/feduc.2019.00035 Keywords: early childhood education and care (ECEC), preschool, policy change, early years educational policy, early years and leadership

# INTRODUCTION

### Early Childhood Education and Care

In many developed countries it is now the norm for preschool children to spend time outside the home in early childhood education and care (ECEC). For instance, on average across OECD countries, 70% of 3-year-olds, 85% of 4-year-olds and 95% of 5-year-olds were enrolled in paid ECEC of some form (or primary education) in 2014 (OECD, 2017). In the UK much of this ECEC is in private (for profit), voluntary (not for profit) or state settings. State provision includes nursery classes attached to a school, or a nursery school catering specifically for pre-school children, with a small number of local authority nurseries, and also children's centres that combine ECEC provision with other services.

Research indicates that attending ECEC promotes school readiness and contributes to later school attainment and positive life outcomes into adolescence (Sylva et al., 2008; Melhuish et al., 2017). As well as affecting cognitive and educational outcomes, there is clear evidence that ECEC experience can have long-term consequences for socio-emotional development. In the USA, several studies evaluating programs with disadvantaged populations that use high quality ECEC as the intervention find positive effects on socio-emotional development. Barnett (2008) found that participation in the Early Head Start (EHS) program was associated with small improvements in children's behavioral problems (Sammons et al., 2002; Love et al., 2005) in terms of positive impacts on children's social-emotional development, including less aggressive behavior, more sustained attention, and higher engagement in play.

# Two Domains of ECEC Quality

Two broad dimensions of quality have been identified that facilitate children's development and learning: structural quality (Early et al., 2007)—which includes adult-child ratios, staff qualifications, group size, and the characteristics of the physical space—and process quality (Slot et al., 2015)—which includes the quality of the curriculum, pedagogical practices and the quality of children's experiences that support development. Frequently there are relationships between structural characteristics and process quality; for example the Effective Provision of Preschool Education (EPPE) study found that process quality was associated with the qualification level of ECEC staff (Sylva et al., 1999a; Melhuish et al., 2006) and the quality of early years provision has also been shown to be associated with managers' qualification level (Mathers et al., 2007; Roberts et al., 2010; Karemaker et al., 2011). In light of these findings, there has been support to improve the qualification level of ECEC staff in England, and there is evidence that it has indeed risen (Brind et al., 2014; Simon et al., 2016). This increase in qualifications may be related to improvements in quality ratings of preschool provision over time (Ofsted, 2015). Another structural characteristic that has been associated with better quality provision is higher staff to child ratios (Mathers et al., 2007; Roberts et al., 2010). For example, higher staff to child ratios have been associated with a higher quality of staff/child interactions in preschool settings (Karemaker et al., 2011). There is also evidence that type of setting may be associated with the quality of provision, for example two UK studies (Sylva et al., 1999b; Roberts et al., 2010) found higher quality in statemaintained settings.

It is likely that the relationships observed between aspects of structural quality and process quality are, at least to some extent, causal. We would therefore anticipate that changes to factors such as staff qualifications and training or staff to child ratios will ultimately affect the process quality of settings.

# Research Aims

The aim of this research is to investigate the associations between structural and process quality measures at settings in the SEED study, and to explore the hypothesis that these relationships may vary according to the type of ECEC settings considered (e.g., private provision, state-funded provision). This research is of importance because, as discussed in the following section, the process quality of the ECEC which children attend may affect their educational and later life outcomes. An understanding of how structural quality influences process quality is therefore of use in developing more effective ECEC provision.

# The Benefits of High Quality ECEC

It has been shown that positive benefits of ECEC are more likely if the ECEC experiences are of high quality. In studies of the general population, there is also evidence that the quality of the ECEC is important for children's future educational, cognitive and behavioral outcomes (Melhuish, 2004; Sylva et al., 2004, 2010; Barnes and Melhusih, 2016) all of which are relevant for subsequent mental health. Similarly, the high quality ECEC intervention in the Abecedarian Project (Muennig et al., 2011) was linked with long-term beneficial effects. Manning et al. (2010) in a meta-analysis showed reduced deviance, increased social participation, and reduced crime once children had reached adolescence. McLaughlin et al. (2007) showed that the early intervention reduced later depression, and Barnett (2008) reported that the Abecedarian program group reported fewer depressive symptoms and less substance abuse at age 21. Other evidence comes from less intensive intervention. A study conducted in Chicago (Reynolds et al., 2011) gives evidence of long term benefits for less advantaged children who started preschool aged three or four compared to those who started later. These benefits were larger for boys than for girls and were also larger among children whose parents did not complete high-school. When followed up at age 28, children who had started preschool aged three to four had higher levels of educational achievement, higher incomes and were less likely to have drug problems or involvement with crime, as compared to children who had started preschool at a later age. For example, in the Netherlands, Broekhuizen et al. (2014) found beneficial effects on behavior problems associated with higher quality of ECEC, which were also found in the UK (Sammons et al., 2002; Sylva et al., 2004). In the USA, research has indicated that higher ECEC quality (e.g., childteacher relationships and interactions) can be prospectively related to more social competence and fewer behavioral problems (Peisner-Feinberg et al., 2001; NICHD Early Child Care Research Network, 2006; Burchinal et al., 2008; Mashburn et al., 2008), with some effects lasting into adolescence (Vandell et al., 2010). Studies have also demonstrated that children's behavioral selfregulation and emotional understanding can be improved by exposure to ECEC of high quality, specifically where there is a comprehensive socio-emotional curriculum and where staff development has focussed on the responsiveness of staff/child interactions (Bierman et al., 2014; Landry et al., 2014). This indicates that changing the quality of ECEC may enhance children's outcomes, including socio-emotional development.

The relevance of high quality ECEC is likely to differ depending on child characteristics. Studies have shown that exposure to ECEC of high quality can help to close the gap between levels of behavioral problems found in boys and girls, which generally tend to be higher in boys. Among disadvantaged children, it has been found that the level of behavioral problems in boys is reduced by exposure to high quality ECEC whilst the same exposure has much less effect on the levels of behavioral problems in girls (Votruba-Drzal et al., 2004, 2010). This may be in part because boys, who tend to have lower levels of self-regulation than girls, derive greater benefit from exposure to more structured environments (Votruba-Drzal et al., Melhuish and Gardiner Policy & UK ECEC Quality

2004, 2010). A study by Broekhuizen et al. (2015) explored whether children's emotional self-regulation and their gender were independent moderators of the effects of ECEC quality or whether the generally lower levels of emotional self-regulation among boys was a partial explanation for the moderating effect of child's gender. The study concluded that children's emotional self-regulation and their gender were independent moderating factors. This study also provides evidence for differences in the susceptibility of children to the effects of ECEC quality depending on their temperaments. Specifically, children with lower selfregulation show greater decreases in social abilities when exposed to low quality ECEC and also greater increases in social abilities when exposed to high quality ECEC, as compared to children with higher levels of self-regulation. This finding bears out earlier work by Belsky (1997). A similar pattern has been observed with regard to children's reactivity. Children with more reactive temperaments are both more likely than other children to exhibit behavioral problems when exposed to low quality ECEC and are also more likely to exhibit good levels of socio-emotional skills when exposed to high quality ECEC (Deynoot-Schaub and Riksen-Walraven, 2006; Pluess and Belsky, 2009; Almas et al., 2011; Phillips et al., 2012).

# The Relevance of the Policy Context—Illustrated by the UK

In the UK, the period since 1999 has seen substantial policy developments for ECEC provision influenced extensively by research findings (Melhuish, 2016). Based on the established benefits of ECEC, successive UK government policies provided free ECEC provision for all 3–4 year olds in 2004 and undertook to improve the quality of provision through various measures, and have also extended funding for ECEC provision by increasing the number of hours funded and decreasing the age at which eligibility starts.

In 2017 provision was extended to 570 h annually for all 3–4 year olds, with an additional 570 h annually for children whose parents are working (Melhuish et al., 2017). From 2013 2 year old children from disadvantaged families (defined by the receipt of specific benefits) also became eligible for 570 annual hours of funded ECEC, and from 2015 this provision was extended to 2 year old children from households which were moderately disadvantaged, including children from families with low incomes and children with Special Educational Needs/Disabilities (SEN/D). This state provision of funding for ECEC for 2 year old children in the UK covers approximately the 40% most disadvantaged families by income.

The Statutory Framework for the Early Years Foundation Stage, which has legal foundations in the Childcare Act 2006, sets out statutory requirements for ECEC for children aged 0–5 in the UK with the aim of ensuring that all provision is of high quality so that "no child gets left behind" (Department for Education, 2014). The extent to which early years providers satisfy these requirements has been monitored by the Office for Standards in Education (Ofsted), since Ofsted began monitoring ECEC in 2001. In addition, there have been several government-funded schemes to improve quality of ECEC provision particularly from 2004 onwards, when state-funded universal provision started for 3 and 4 year olds.

This paper considers the relationship between structural and process aspects of quality in ECEC group settings for 3- and 4-year-old children from the Study of Early Education and Development (SEED) study, a large-scale ongoing longitudinal study in England that is investigating the potential impact of ECEC on children's outcomes in school and personal well-being. Then a comparison is presented between ECEC quality data from the SEED study collected in 2014–2016 and equivalent data from the Effective Provision of Pre-school Education (EPPE) study (Sylva et al., 2004, 2010). The EPPE study was conducted in six English local authority (LA) areas, chosen to provide coverage of urban and rural areas, more and less disadvantaged areas and to ensure ethnic diversity. Within each LA, centres were selected from the six most common types of provision: playgroups, local authority/voluntary day nurseries, private day nurseries, nursery schools, nursery classes, and integrated centres combining care and education. Centres were selected randomly within each type of provision in each LA. The sample consisted of 141 centres (Sylva et al., 2004).

The existence of these two large datasets on the quality of group ECEC in England, which are approximately nationally representative at their respective time points, enables a comparison before and after a period of extensive policy change that transformed the early years sector, and which was intended to increase both the uptake and quality of ECEC for children, particularly for those 3–5 years of age. This comparison can inform on whether policy change, as seen in the UK, is associated with change in ECEC quality. If so, then there is the possibility that there may be consequences for later child outcomes, including socio-emotional development and wellbeing, for the country's population. It may also be transferable to other contexts.

# Strengths and Limitations

Strengths of this study are the large sample size, comprising 598 ECEC settings, and the wide range of settings type studied. ECEC settings are also included from all regions of England. A limitation of the study is that settings were chosen that were attended by children from the SEED study, which was highly clustered and in which disadvantaged children were overrepresented. For this reason, although the SEED sample of settings is geographically diverse and includes settings of all main types, it is not a representative sample of all ECEC settings in England.

# METHODS

#### Sampling

As part of the SEED longitudinal study, a sample of 5,642 children born in England between September 2010 and August 2012 was selected using national child benefit records (state benefits available to parents with children). Sampling on the basis of geographical location (postcode districts) eligible families with children of the relevant age were selected for interview. This procedure produced a highly clustered sample. Children were selected according to family income so that approximately one third of the sample came from the most disadvantaged 20% of families by income, one third of the sample came from the next most disadvantaged 20% of families and one third of the sample came from the least disadvantaged 60% of families. A list of ECEC settings was obtained by asking parents which settings their children were attending as part of the SEED parent interviews. The sample was stratified by provider type, with settings classified as one of private, voluntary, state nursery class, nursery school, children's centre or local authority nurseries (LAN) (Melhuish and Gardiner, 2018).

Settings were selected so that the percentage in each category was similar to the percentage in that category across all settings used by children participating in the SEED study. If a provider did not wish to participate it was, wherever possible, replaced with the same type of provider from the same geographical area. The sample of ECEC settings came from all parts of England; it comprised 598 settings attended by children aged 3 to 4 participating in the SEED study.

#### Structural Quality

A staff interview was conducted with the manager of each childcare setting. During this structured interview a questionnaire was completed by the visiting researcher recording information on the structural quality of the ECEC setting provided by the setting manager. Information gathered comprised answers on the following topics: number of places at the setting, setting on single site/multiple sites, minimum age of children, maximum age of children, staff to child ratio, whether the settings had special educational needs and disability (SEN/D) provision, mean level of staff qualification, manager's highest qualification, whether the setting had a staff training plan, whether the setting had a staff training budget, frequency of staff continuing professional development (CPD), frequency of staff supervision and rate of staff turnover.

The overall staff to child ratio was calculated as the number of staff at a setting divided by the number of children at the setting, so higher ratios mean that there are fewer children per member of staff. The qualifications of ECEC staff and managers were recorded as equivalent to the following levels: Level 1 = GCSE (General certificate of secondary education; taken at age 16) D-G (lower level of pass), Level 2 = GCSE A∗ -C (higher level pass), Level 3 = A-Level (Advanced level examinations taken at age 18), Level 4 = Certificate of Higher Education (beyond age 18; lower level), Level 5 = Diploma of Higher Education (beyond age 18, higher level), Level 6 = Batchelor's degree, Level 7 = Master's degree, Level 8 = Doctorate.

#### Process Quality

Process quality was assessed by trained researchers during a half-day observational visit to each setting included in the study. Overall process quality for the settings attended by 3– 4 year olds was assessed using the revised Early Childhood Environmental Rating Scale (ECERS-R) (Harms et al., 2005). This assessed settings quality using five domains: Space and Furnishings (e.g., room layout, accessibility of resources), Personal Care Routines (e.g., welfare requirements such as health & safety and provision for sleeping), Language and Reasoning (e.g., supporting children's communication, language and literacy development; supporting children's critical thinking), Activities (e.g., provision of an exciting and accessible learning environment, resources to support specific types of play), Interaction (e.g., supervision, support for social interactions) and Programme Structure (e.g., opportunities for children to access their own curriculum, planning schedules/routines to meet children's needs). The curriculum extension of the ECERS-R scale, ECERS-E (Sylva et al., 2011) was used to assess the specifically educational quality of the settings for 3–4 year olds; this consisted of the three domains: Literacy (e.g., opportunities for emergent writing, letters and sounds), Mathematics (e.g., number skills and reasoning) and Diversity (e.g., planning for children's individual learning needs, valuing and respecting other cultures, gender diversity). The quality of staff/child interactions was assessed using the Sustained Shared Thinking and Emotional Well-being scale (SSTEW) (Siraj et al., 2015) comprising five domains: Building Trust, Confidence and Independence (e.g., self-regulation and social development, encouraging choices and independent play, planning for small group, and individual interactions); Supporting and Extending Language and Communication (e.g., encouraging children to interact with others, staff actively listening to children, and encouraging children to listen, staff supporting children's language use); Supporting Emotional Well-being (staff supporting children's social and emotional communication); Supporting Learning and Critical Thinking (e.g., supporting curiosity and problem solving, encouraging sustained, shared thinking during story time, encouraging sustained, shared thinking in investigation, and exploration, supporting concept development and higher order thinking); and Assessing Learning and Language (e.g., using assessment to support and extend learning and critical thinking, assessing language development). For all three scales the total scale score was the mean of the subscale scores. The scores on the process quality scales had a range of 1–7. Quality was classified as Inadequate (<3), minimal (≥3 and <4), adequate (≥4 and <5), good (≥5 and <6) or excellent (≥6).

These measures were selected because they are commonly used in the UK and internationally to assess the quality of ECEC settings, they have high levels of inter-rater reliability (Clifford and Reszka, 2010; Whitebread et al., 2015), and they have been shown to capture key elements of settings quality. Quality assessed using these methods has also been shown to have some predictive value for the future outcomes of children attending ECEC settings (Sylva et al., 2010).

#### Analysis Strategy

Structural characteristics of settings and process quality were compared by provider type: private, voluntary, state nursery class/school, and children's centre (the number of local authority nurseries was small and these were omitted from comparisons). Means of the continuous measures were compared using the Wilcoxon rank sum test, while the proportions for binary characteristics—including whether or not process quality was "excellent" or "good or better"—were compared using the χ-square test for proportions.

The relationship between structural factors and process quality was examined using multiple linear regression of process quality measures as predicted by the structural characteristics of settings. Manager's highest qualification was omitted from the list of structural covariates because of potential collinearity with mean level of staff qualification. Factors associated with settings achieving "excellent"/"good or better" process quality were explored using logistic multiple regression models. Preliminary analysis showed that the relationships between structural characteristics and process quality differed by setting type; these regression models were therefore fitted separately for the different types of settings. For nursery classes/schools and children's centres the logistic regression models for "excellent" and "good or better" scores were not fitted as the sample sizes were too small to make these models reliable.

# Comparing the Quality of ECEC Before and After Significant Policy Change

In order to examine how quality of ECEC provision has changed over time in England, this section compared data collected in 2015–2016 from the SEED study with data collected in 1998– 1999 for the Effective Provision of Pre-School Education (EPPE) study (Sylva et al., 1999b). This enabled a comparison between the quality of ECEC settings before and after a period of rapid policy development in the area of ECEC, which occurred across the years 1999–2014 as described by Melhuish (2016). Both studies collected data on samples of settings that were approximately representative of ECEC group settings for 3–4 year old children in England at the relevant time. Both the EPPE (Sylva et al., 1999a) and SEED (Melhuish and Gardiner, 2018) studies included data collected by similar methodologies using the ECERS-R (Harms et al., 2005) and ECERS-E (Sylva et al., 2011) measures, as well as data on structural characteristics through similar staff interviews. In both studies the ECERS-R and ECERS-E were conducted according to the respective manuals. In both studies researchers collecting quality data were trained extensively on the observational instruments and one person acted as the "standard" in a reliability exercise. Inter-observer reliability was checked through five centres being rated by a researcher and the person acting as "standard." Comparison between observers indicated good levels of interobserver reliability (Kappa range = 0.75–0.90, median = 0.81). Further information on the methodology of the EPPE study can be found in Sylva et al. (1999b) and for the SEED study in Melhuish and Gardiner (2018).

#### RESULTS

#### Type of SEED Settings

The largest type of ECEC settings in the SEED study was private (302, 50.5%), with voluntary settings providing the second largest group (143, 23.9%), nursery classes/schools made up 20.6% (123), and the 26 children's centres represented 4.3%. The four local authority nurseries (0.7%) were too few for reliable analysis.

# Structural Characteristics of SEED Settings

Summary statistics for structural characteristics of settings are shown for continuous measures in **Table 1**, and for binary measures in **Table 2**. Structural characteristics for the different settings types were compared with those for private settings which, as the largest group, was use as the baseline. There were a number of significant differences in structural characteristics between types of settings. Voluntary settings tended to be smaller than private settings and tended to have a narrower age range for children. Nursery classes/schools tended to have more highly qualified managers and staff than private settings, but also had lower staff to child ratios (i.e., more children per member of staff).

# Quality of SEED Settings and Quality by Type

Mean ECERS-R (overall quality) for all settings was 5.28, a "good" rating, whilst ECERS-E (educational quality) and SSTEW (quality of staff/child interactions) had means of 4.18 and 4.70, respectively, (both in the "adequate" range); see **Table 3**. 62.7% of settings achieved a "good or better" standard on the ECERS-R scale, with 26.8% achieving an "excellent" rating. These figures were somewhat lower for the other scales with 44.3% registering "good or better" and 14.5% "excellent" on the SSTEW scale and 26.4% achieving "good or better" ratings on the ECERS-E scale and 5.7% achieving "excellent" scores on this scale. Children's centres and nursery classes/schools had significantly higher mean quality scores than private settings (reference group) on all three quality scales. The proportions of children's centres and nursery classes/schools with "excellent" and "good or better" scores were significantly higher than the proportion of private settings achieving scores in these ranges. Voluntary settings had significantly lower mean scores than private settings on the ECERS-E scale (educational quality) and the proportion of voluntary settings with "good or better" ECER-E scores was significantly lower than for private settings.

# Process Quality in Terms of Structural Quality in the SEED Study

The results of the regression models of process quality in terms of structural characteristics of ECEC settings are given in **Tables 4**–**7**. Statistically significant associations between structural characteristics and process quality were found for all settings types. As hypothesized, the associations between structural and process quality also differed between the different settings types.

In private settings, there were significant associations between higher quality on the ECERS-R, ECERS-E and SSTEW scales and having a larger number of places, a higher mean level of staff qualification and a minimum age of 2 for children. Having SEN/D provision was significantly associated with higher ECERS-E scores. Achieving excellence on the ECER-R scale was associated with a higher mean level of staff qualification and a lower frequency of staff CPD. Larger settings (settings with a larger number of places) were associated with higher

#### TABLE 1 | Summary of continuous structural characteristics.

Mean values for a given type of setting are shown in bold italics if they are statistically significantly different from the mean value for private settings which, as the largest group of settings, is used as the reference group. The test used is the Wilcoxon rank sum test for difference in means; the cut-off for statistical significance is p = 0.05 in a two-sided test.

TABLE 2 | Summary of binary structural characteristics.

The percentage of setting with a given structural characteristic is shown in bold italics if it statistically significantly different from the percentage for private settings which, as the largest group of settings, is used as the reference group. The test used is a chi-square test for a difference in proportions; the cut-off for statistical significance is p = 0.05 in a two-sided test.

likelihood of achieving excellent ECERS-E scores, whilst excellent SSTEW scores were associated with larger settings and a higher overall staff to child ratio. Larger settings were associated with a higher likelihood of achieving "good or better" scores on all three quality scales. A minimum age of 2 for children was associated with "good or better" scores on the ECERS-E and SSTEW scales, a higher mean level of staff qualification was associated with achieving "good or better" scores on the ECERS-R and SSTEW scales; a higher overall staff to child ratio was associated with achieving "good or better" ECERS-R scores; see **Table 4**.

In voluntary settings, having a training plan in place and a higher overall staff to child ratio were associated with higher scores on the ECERS-R scale. Having a higher staff to child ratio (i.e., fewer children per member of staff) was associated with higher scores on the ECER-E scale. Having a training plan in place was also associated with higher SSTEW scores. Not having SEN/D provision was associated with an increased probability of achieving excellent ECERS-R scores. A higher staff to child ratio and a minimum age of 0–1 for children were associated with an increased probability of achieving "good or better" ECERS-E scores. Finally, having a training plan in place was associated with an increased probability of achieving "good or better" SSTEW scores; see **Table 5**.

For nursery classes/schools a lower maximum age for children was associated with higher ECERS-R and ECERS-E scores; having a training budget in place was associated with higher ECERS-E and SSTEW scores and a lower rate of staff turnover was associated with having higher SSTEW scores; see **Table 6**.

For children's centres a higher mean level of staff qualification was associated with achieving higher scores on the ECERS-R scale; see **Table 7**.

#### Results of the Comparison Between SEED and EPPE

The ECERS-R and ECERS-E quality scores were compared for the SEED and the earlier EPPE samples overall. The mean ECERS-R score (overall quality) for settings in the EPPE study was 4.29, compared to a mean of 5.28 in SEED. For ECERS-E the mean for EPPE was 3.17 whilst the mean for SEED was 4.18. A comparison of these scores by category is given in **Figure 1** (ECERS-R) and **Figure 2** (ECERS-E). There is a greater proportion of poorer quality settings (i.e., inadequate, minimal,

#### TABLE 3 | ECEC Settings quality scores.

The mean quality score for a given type of setting is shown in bold italics if, in a linear regression of quality score in terms of setting type, it showed a significant difference from the Private settings which, as the largest group, was used as the baseline (comparison) group.

The percentage of settings of a given type with excellent quality scores is shown in bold italics if, in a logistic regression of "excellent quality score (yes/no)" in terms of setting type, it showed a significant difference from the Private settings which, as the largest group, was used as the baseline (comparison) group.

The percentage of settings of a given type with good or better quality scores is shown in bold italics if, in a logistic regression of "good or better quality score (yes/no)" in terms of setting type, it showed a significant difference from the Private settings which, as the largest group, was used as the baseline (comparison) group.

and adequate) in the EPPE study than in the SEED study for both ECERS-R and ECERS-E. This indicates that the quality of ECEC settings in England on these measures has improved between the time of EPPE and the time of SEED.

From the staff interviews carried out by both studies, comparable data on staff qualifications were derived. The qualification levels of both managers and staff at ECEC settings increased between the EPPE interviews in 1998–1999 (Taggart et al., 2000) and the SEED interviews in 2014–2016 (Melhuish and Gardiner, 2018). A comparison of manager's highest qualification between the EPPE and SEED studies is shown in **Figure 3**. The percentage of managers with a degree (Level 5 or above) rose from 43% in the EPPE study to 66% in the SEED study. Comparing the mean level of staff qualification (see **Figure 4**), we see that the most common level was 3–4 in both the SEED and EPPE studies; the second most common level for EPPE was a Level 2, whilst for SEED it was a Level 5 or above. It seems likely that the rise in ECEC quality between EPPE (1998– 1999) and SEED (2014–2016) is partly due to the improvements in the qualification levels of staff and managers over this period.

#### DISCUSSION

#### Factors Associated With Process Quality

In line with previous research (Sylva et al., 1999b; Roberts et al., 2010; Brind et al., 2014; Ofsted, 2015), quality was found to vary by type of setting, with nursery classes/schools and children's centres tending to have higher process quality than the private and voluntary settings, which made up the majority of ECEC provision.

This study also found a wide ranging pattern of associations between the structural characteristics of ECEC settings and their process quality, which are relevant to policy development. As with any observational study, it cannot be assumed that these associations are causal, nor can it be assumed that, if they are causal, the direction of causation necessarily runs from structural characteristics to process quality. In some cases it is possible that causation could go in the other direction; for example, it could be that the associations between higher levels of staff qualification and higher process quality arise because higher quality settings are more successful in recruiting more highly qualified staff. There may also be unobserved confounding factors, which influence both structural characteristics and process quality. Nevertheless, it is cautiously suggested that the best explanation for the observed associations is mainly a causal one from structural characterizes to process quality, so that over time improving factors such as staff qualification levels and staff to child ratios would tend to result in improvements in ECEC settings quality.

There were some differences depending on the way that centres were funded and managed. It appears that staff qualification level was a significant driver of quality at private ECEC settings, which is in accord with existing research (Sylva et al., 1999b; Melhuish et al., 2006; Mathers et al., 2007; Roberts et al., 2010; Karemaker et al., 2011) For voluntary (not for profit) settings, which may be more homogenous in the level of staff and manager qualifications, the presence of a training plan was a significant predictor of higher quality. In line with earlier studies, a higher staff to child ratio was found to be a significant predictor of quality (Mathers et al., 2007; Roberts et al., 2010; Karemaker et al., 2011). However, it should be noted that state

 settings.

> Modelcoefficient

Frontiers in Education | www.frontiersin.org

in question.

Model coefficients which are significantly different from zero at the p = 0.05 level are shown in bold italics.

Model coefficients which are significantly different from zero at the p = 0.05 level are shown in bold italics. Cells are left blank where a finite model coefficient could not be calculated due to insufficient variability in the data.

TABLE 6 | Results of regression models of quality in terms of structural characteristics; nursery classes/schools.

Sample size = 106.

β is the model coefficient from a multivariate linear regression of quality in terms of the covariates. For the continuous covariates this gives the change in the outcome variable corresponding to a 2 standard deviation change in the covariate. For binary covariates this gives the difference between settings with and without the characteristic in question. Model coefficients which are significantly different from zero at the p = 0.05 level are shown in bold italics.

funded and managed nursery classes/schools, which tend to have lower staff to child ratios than private and voluntary settings, also tended to have higher process quality ratings. It may be that the presence of more highly qualified staff and managers allows quality to be maintained with a larger number of children per staff member. In a number of cases a narrower range of child ages was associated with higher quality provision, although this finding was not uniform since in voluntary settings for 3–4 year olds an increased probability of achieving "good or better" curriculum quality was associated with settings having a younger starting age for children. This is an area which may merit further study. At private settings for 3–4 year olds, excellence on the ECER-R scale was associated with a lower frequency of staff professional development (CPD). It is probable that this is an instance of reverse causation. That is, those settings that are seeking to improve have increased their frequency of CPD whilst those that have already achieved high quality standards have not needed to do so.

### Changes Over Time Between the EPPE and SEED Studies

The EPPE (1998–1999) and the SEED (2014–2016) samples of ECEC settings were approximately representative of the early TABLE 7 | Results of regression models of quality in terms of structural characteristics; children's centres.

Sample size = 25.

β is the model coefficient from a multivariate linear regression of quality in terms of the covariates. For the continuous covariates this gives the change in the outcome variable corresponding to a 2 standard deviation change in the covariate. For binary covariates this gives the difference between settings with and without the characteristic in question. Model coefficients which are significantly different from zero at the p = 0.05 level are shown in bold italics.

years sector at the time of the studies. The increase in quality between the EPPE and SEED studies over more than a decade on both the overall quality (ECERS-R) and curriculum quality (ECERS-E) is striking, during which time period there was a strong policy focus on increasing both the quantity and the quality of provision. This trend of increasing quality over time is in line with observations by the national organization responsible for assuring quality in the UK (Ofsted). The proportion of ECEC providers they judged to be good or outstanding increased from 74% in 2012 to 93% in 2017 (Ofsted, 2017), although it should be borne in mind that the validated quality measures used in the EPPE and SEED studies and the criteria used by Ofsted are significantly different. An increase in the qualification level for both setting managers and staff was also observed between the EPPE project and the SEED study. This is in line with evidence from the UK Labor Force Survey, which covers a similar period of time (Simon et al., 2016) It is probable that this increase is related to the rise in quality levels.

During the period of time between the two studies, raising staff and manager qualifications and facilitating inservice professional development have featured in government policy initiatives. These factors are likely to increase quality

FIGURE 1 | Percentage breakdown of ECERS-R scores by quality level in the EPPE and SEED studies. Quality is classified as Inadequate (<3), minimal (≥3 and <4), adequate (≥4 and <5), good (≥5 and <6) or excellent (≥6). Figure © 2017 NatCen Social Research, University of Oxford and Action for Children, reproduced with permission.

according to the analyses of structural factors as predictors of quality. Hence these findings may indicate that the increasing professionalization of the ECEC workforce as well as influencing qualification guidelines in the early years statutory framework, as elements of the policy change over the period, are factors in the observed change in ECEC quality in the approximately 15 years between the EPPE and SEED studies.

The improvement in quality seen between the time of the EPPE and SEED study is linked to a reduction in the incidence of poor quality and a corresponding increase in medium and high quality. This is relevant to future research that explores the relationship between ECEC quality and child development. Previously, where studies have found effects upon child outcomes linked to quality the studies have typically included ECEC centres that vary substantially across the quality range including substantial amounts of poor quality. Much of the effect upon outcomes in such studies (Sammons et al., 2002; Burchinal et al., 2008; Mashburn et al., 2008; Broekhuizen et al., 2015) derives from the poorer outcomes associated with poor quality

FIGURE 3 | Comparison of level of manager's qualification in the EPPE and SEED studies. Figure © 2017 NatCen Social Research, University of Oxford and Action for Children, reproduced with permission.

as compared with medium to high quality. The consequence therefore of reducing the prevalence of poor quality is likely to be that quality effects upon child outcomes will be reduced and potentially be too small to be systematically of statistical significance in situations where there are a large number of covariates that are associated with powerful effects upon child outcomes. In such circumstances a study may conclude that there are no quality effects upon child outcomes, and this message may be seized upon by those (e.g., politicians) eager to reduce expenditure on maintaining good quality ECEC provision. Hence there is a socio-political danger inherent in the interpretation of a study's results without reference to the larger context of research.

Burchinal (2017) has referred to the reducing size of quality effects in more recent studies and possible reasons given are the reducing incidence of poor quality ECEC, and the inability of existing measures to adequately capture the aspects of ECEC most likely to influence child outcomes. This latter reason was the motivation behind the production of the Sustained Shared Thinking and Emotional Well-being (SSTEW) scale (Siraj et al., 2015) that focuses specifically on interactional quality likely to enhance language development and self-regulation, which have been linked with better long-term educational and socioemotional outcomes (Malecki and Elliot, 2002; Trentacosta and Shaw, 2009; Moffitt et al., 2013). Further work is needed in improving ECEC quality measures, including both interactional and pedagogical aspects of quality.

#### Educational and Policy Implications

It is increasingly clear that ECEC is a substantial contributor to the longer-term educational, social and economic success of individuals (Heckman, 2008; Melhuish et al., 2015). Hence, if a country improves ECEC quality for its children then it is not only enhancing children's lives in the "here and now" it is also advancing the long-term outcomes for children, and by doing so it is investing in the future. In the current study staff qualifications were predictive of quality at private (for profit) settings, whilst for voluntary (not for profit) settings, which were rather homogenous in staff qualifications, having a staff training plan (in-service professional development) and a better staff to child ratio were predictive of higher quality. However, state funded ECEC centres, which tend to have less favorable staff to child ratios but more highly qualified staff than private and voluntary settings, tended to have higher quality. This indicates that the presence of more highly qualified staff may be more influential on quality than the staff: child ratio, at least within the range of these variables found in the UK. Earlier findings in the UK from the EPPE study led to an increased policy emphasis on improving ECEC quality through improving staff qualifications and training (Melhuish, 2016). This change in policy seems to have borne fruit in that a comparison of equivalent quality data from separate UK studies, conducted before and after this period of policy change, found improvement both in observed quality and staff qualifications, as reported in the current study. Overall these findings indicate that countries wishing to improve the quality of their ECEC provision should actively seek to improve both staff qualifications and in-service professional development. The potential for in-service professional development to improve ECEC quality is further supported by a recent RCT study in Australia (Siraj et al., 2018) where in-service professional development had clear effects upon observed quality in ECEC as well as potential effects for child outcomes. Hence it would be

#### REFERENCES

appropriate for policy in this area to be framed to increase staff qualifications and to provide enhanced opportunities for ECEC staff to obtain in-service professional development. Additionally, staff: child ratios should be maintained at as favorable a level as is pragmatically viable.

### CONCLUSION

Structural aspects of quality such as staff qualifications and continuous professional development as well as staff:child ratios are linked to process quality in group ECEC settings. The 20 years since the start of the EPPE study have seen great changes in the level of use of ECEC in the UK and in the nature of the ECEC provided, to a large extent because of changes in government policy. The quality of ECEC provision has risen significantly over this period, as has the typical level of qualification of staff and managers. The almost universal use of ECEC, for 3–4 year olds in the UK, and other countries, makes the effectiveness of the provision crucial for children's later development. Whilst some of the factors that produce high quality ECEC are clear, well qualified staff/managers and adequate staff to child ratios being the best attested, this is an area where further research is still needed. However, the lessons from these two UK longitudinal studies provide an important indication for other countries about ways that child development may be enhanced through policy change, contributing to improvements in child well-being and later adult development.

#### ETHICS STATEMENT

Ethics approval was given by the University of Oxford ethics committee. Written informed consent was obtained from all adult study participants and from the parents/legal guardians of all non-adult participants.

# AUTHOR CONTRIBUTIONS

EM proposed the research hypotheses. JG carried out the data analysis. Both authors contributed to the writing of the paper and approved it for publication.

of behavior problems among economically disadvantaged children in middle childhood. Child Dev. 81, 1460–1474. doi: 10.1111/j.1467-8624. 2010.01485.x

Whitebread, D., Kuvalja, M., and O'Connor, A. (2015). Quality in Early Childhood Education: an International Review and Guide for Policy Makers. WISE Initiative, Cambridge: University of Cambridge. Available online at: http:// www.wise-qatar.org/sites/default/files/asset/document/wise-research-7 cambridge-11\_17.pdf

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Melhuish and Gardiner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Growth Mixture Modeling Study of Learning Trajectories in an Extended Computerized Working Memory Training Programme Developed for Young Children Diagnosed With Attention-Deficit/Hyperactivity Disorder

#### Anna Orylska1,2 \*, Julie A. Hadwin3 , Aleksandra Kroemeke2 and Edmund Sonuga-Barke4,5

1 Interdisciplinary Center for Applied Cognitive Studies, SWPS University of Social Sciences and Humanities, Warsaw, Poland, 2 Department of Psychology, SWPS University of Social Sciences and Humanities, Warsaw, Poland, 3 Centre for Innovation in Mental Health – Developmental Laboratory, Department of Psychology, University of Southampton, Southampton, United Kingdom, 4 Department Child and Adolescent Psychiatry, Institute of Psychiatry, Psychology and Neuroscience, Kings' College London, London, United Kingdom, 5 Department of Child and Adolescent Psychiatry, Aarhus University, Aarhus, Denmark

#### Edited by:

Vasiliki Totsika, University College London, United Kingdom

#### Reviewed by:

Mats Granlund, Jönköping University, Sweden Jessica A. Church-Lang, University of Texas at Austin, United States

> \*Correspondence: Anna Orylska

<EMAIL_ADDRESS>
#### Specialty section:

This article was submitted to Special Educational Needs, a section of the journal Frontiers in Education

Received: 23 July 2018 Accepted: 04 February 2019 Published: 22 February 2019

#### Citation:

Orylska A, Hadwin JA, Kroemeke A and Sonuga-Barke E (2019) A Growth Mixture Modeling Study of Learning Trajectories in an Extended Computerized Working Memory Training Programme Developed for Young Children Diagnosed With Attention-Deficit/Hyperactivity Disorder. Front. Educ. 4:12. doi: 10.3389/feduc.2019.00012 This study explored (1) whether growth mixture modeling (GMM) could identify different trajectories of learning efficiency during a working memory (WM) training programme for young children diagnosed with Attention Deficit Hyperactivity Disorder (ADHD), compared with a typically developing (TD) control group, and (2) if learning trajectories and outcomes were different for simple and complex training tasks. Children completed simple visuospatial short-term memory (VSSTM) and complex visuospatial WM (VSWM) tasks for 15 min a day, 5 days a week, and for 8 weeks. Parent-reported executive functioning, and children's WM and attention control, educational achievement, and IQ were measured prior to (T1), immediately following (T2) and 3 months after training (T3). GMM analysis showed that WM training was represented as one learning curve, and there was no difference for the trajectories of the ADHD and TD groups. The learning trajectory for the VSSTM tasks across groups was represented as one learning curve and for the VSWM tasks there were three learning curves. Learning for the VSSTM tasks and for most children in the VSWM tasks was characterized by an inverted-U shape, indicating that training was effective for up to 15 sessions, was stable and declined thereafter, highlighting an optimal training timeframe. For the VSWM tasks, the two remaining groups showed either a U-shaped or a high inverted U-shaped trajectory, with the latter group achieving the highest T1T2 change score (i.e., children showed a lower starting point and the most gain in terms of learning and post-training performance). There were no broader benefits of training at post-test or follow-up. Further research should explore who would benefit most from intensive cognitive training, as well as the potential benefits for mental health and well-being.

Keywords: WM training, learning trajectories, young children, attention-deficit/hyperactivity disorder (ADHD), growth mixture modeling (GMM)

# INTRODUCTION

Developmental research has highlighted that attentional control skills, including working memory (WM), are positively linked to several indices of adjustment in childhood and adolescence, such as emotion regulation (reviews by Hadwin et al., 2016; Moran, 2016). In addition, research has shown that performance in WM tasks in children and adolescents is positively associated with educational achievement in core subjects, including text comprehension and reading (e.g., Daneman and Carpenter, 1980; Perfetti, 1985; Baddeley, 1986) and mathematical problem solving (e.g., Bull and Scerif, 2001; Swanson and Beebe-Frankenberger, 2004). These associations are argued to reflect cognitive potential that is separate from current knowledge (e.g., Gathercole et al., 2004; Phye and Pickering, 2006; Halford et al., 2007; Cowan, 2014).

Several studies have aimed to improve attentional control via WM training in clinical populations who experience difficulty with inattention. Training typically focuses on increasing shortterm memory and WM skills using adaptive procedures that are delivered across multiple 30–40 min sessions each week (Klingberg et al., 2002, 2005). Task difficulty is typically adjusted automatically to performance across sessions to maximize learning at the boundaries of an individual's competence (Cortese et al., 2015). Several studies have found that improvements in WM associated with training is underpinned by neural change (Vinogradov et al., 2012) and that some degree of adaptability is a necessary precondition for its continuity (Poldrack and Gabrieli, 2001; Lewis et al., 2009).

Evidence further suggests that WM training represents one potential avenue for reducing symptoms of inattention and hyperactivity in Attention Deficit Hyperactivity Disorder (ADHD) (Klingberg et al., 2002, 2005). Building on evidence of brain plasticity from rehabilitation science and contemporary developmental neuroscience, WM training is premised on the notion that it impacts key brain networks implicated in ADHD (Vinogradov et al., 2012). Additionally, researchers have suggested that neuropsychological deficits mediate the pathways between originating causes and ADHD symptoms. It is argued, therefore, that improvement in neuropsychological functioning, including WM, may be a prerequisite for ADHD symptom reduction (Coghill et al., 2005).

Several recent reviews of research in WM training have questioned the viability of existing training programs to impact core symptoms and improve daily functioning in individuals diagnosed with ADHD (Chacko et al., 2014). These have highlighted the lack of scientific rigor in existing studies. In addition, they have raised concerns around goodness of fit that reflects the heterogeneous symptom profile in ADHD. For example, researchers have concluded that the results of WM training studies are inconsistent because of inadequate controls and ineffective measures to understand change in core cognitive functioning (e.g., Shipstead et al., 2010). Moreover, a systematic meta-analytic review of WM training studies with children and adult populations with ADHD used stringent criteria for inclusion to ensure that all (N = 23) studies were either randomized controlled trials or quasi-experiments (Melby-Lervåg and Hulme, 2013). The review found collective shortterm improvements in practiced skills in clinical populations diagnosed with ADHD, however, there was no clear evidence for transfer to broader cognitive functioning or generalization (i.e., a reduction of ADHD symptoms in daily life).

A further meta-analysis considered the effect of cognitive training on ADHD symptoms, comparing reported outcomes across individuals who were blind or not blind to the intervention group (Sonuga-Barke et al., 2013). The analysis showed significant positive change in parent and teacher reported ADHD symptoms, however, this effect was lost when blinded assessments were analyzed. Moreover, a further meta-analysis showed that reported improvements dropped substantially and became statistically non-significant when only blinded measures were considered (Cortese et al., 2015).

Reviews of this evidence highlight that WM training for individuals diagnosed with ADHD leads to short-term improvements in WM functions that are directly taught. Increasingly, however, the proposed effectiveness of WM training to improve inattention and the quality of daily life more broadly for children and young people diagnosed with ADHD has been met with skepticism. Given that ADHD is a neuropsychological heterogeneous disorder (Nigg et al., 2005; Willcutt et al., 2005; Noreika et al., 2013), then the promise of the effectiveness of WM training may only apply to a subgroup of individuals who show specific processing impairments (Cortese et al., 2015).

The current study extended previous work to explore the possibility that training outcomes were moderated by individual characteristics. Specifically, we used growth mixture modeling (GMM) to investigate individual differences in training efficiency by analyzing trajectories of learning performance across an extended series of training sessions (Muthén and Muthén, 1998; Muthén and Shedden, 1999). GMM is a widely applied data analysis technique used to identify unobserved heterogeneity in a population and to describe and examine longitudinal change within these sub-populations (Nylund et al., 2007; Ram and Grimm, 2009). The results will have significant implications for understanding the development of key cognitive skills known to impact emotion regulation and learning and that can be delivered in an educational context for the benefit of children and young people.

We used an intensive computer WM training protocol that was developed in-house and delivered over 40 sessions and across 8 weeks to young typically developing (TD) children and those given a diagnosis of ADHD. In addition, we measured learning post-training and 3 months later to assess its impact on taught (near transfer) and novel (far transfer) tasks. We investigated three research questions to consider (1) whether trajectories of learning in performance during WM training (i.e., an index of training efficiency) were different between individuals diagnosed with ADHD vs. TD controls; (2) whether GMM would allow us to identify sub-groups with different trajectories of training efficiency for simple and complex training tasks and (3) if different trajectories were associated with near and far transfer in non-trained outcomes. Near transfer tasks included WM tasks completed by the children. Far transfer tasks included constructs that are known to be associated with performance in WM training (i.e., educational achievement and IQ) (Holmes et al., 2009; Jaeggi et al., 2010; Dahlin, 2011; Rudebeck et al., 2012). In addition, we also explored parent reported general behavior and cognitive ability in offspring.

# MATERIALS AND METHODS

The study was registered via the ISRCTN registry as a retrospective trial (https://doi.org/10.1186/ISRCTN15153056).

# Participants

We recruited one-hundred-and-twenty-six children from preschool and primary schools located in the Warsaw area of Poland. Sixty-one TD children were free of developmental disability and any other emotional or behavioral disorders (mean age = 6.70, SD = 0.39, range = 5.90–7.50). Sixty-five children (mean age = 6.70, SD = 0.39, range = 5.90–7.50) met diagnostic criteria for ADHD (American Psychiatric Association, 2000). In addition, five children in this group had a comorbid disorder including oppositional defiant disorder (N = 2) or autism spectrum disorder (N = 3). No one in the group was using psychotropic medication (e.g., methylphenidate) for ADHD disorder. The WM training group was made up of N = 126 children (N = 65 children diagnosed with ADHD and N = 61 typically developing children; see **Figure 1**).

# WM Training Intervention

The computerized WM training was developed by the first author and was based on a published WM training program (Thorell et al., 2009). Because children younger than 7 years typically rely on visuospatial STM and WM (e.g., Hitch and Halliday, 1983; Hitch et al., 1988), training included only visuospatial training tasks. Five visuospatial short-term memory (VSSTM) tasks based on a similar theme were included. In two tasks, participants were shown the position of a target (presented for 1,000 ms) in a series of 4 × 4 matrices and they were asked to recall its location by using the mouse to tap the squares on the computer screen. With progression multiple targets appeared at the same time and were displayed for 2,000 ms. The third VSSTM task required the participant to tap the squares on the computer screen to identify the position of a previous target in a series of 2 × 5 matrices. In the last two VSSTM tasks the child was shown the position of a target in an irregular place and had to recall all positions by tapping the mouse in each position on the computer screen.

Three visuospatial WM (VSWM) tasks used moving stimuli that switched into targets to change color one by one and the target (presented for 1,000 ms) was held in a different color. When the last target was shown, the stimuli stopped moving, the child was asked to point to the targets in the order of appearance. In the first two tasks all stimuli moved in regular way around circle and then around an irregular figure. In the last task the movement was irregular.

Each task had six levels of difficulty, and each of these had three sub-levels. Task performance was recorded as the highest score obtained in each session and where each level corresponded to the number of items that the child had to remember. There were 2 items at level 1 and 7 items at level 6. Learning (training efficiency) was calculated across eight blocks each with five training sessions (the possible score range for each block was 1–18).

The training interface provided children with feedback for correct and incorrect performance (a smiley or sad face). The task was continuously adapted across training so that the difficulty level reflected current performance (i.e., three correct trials were required in order to advance to the next level, and for each incorrect trial, difficulty decreased by three sublevels). Children were asked to complete three tasks per session. The training was scheduled so that children worked on each task equally across 40 training sessions.

# Pre- and Post-training Measures of Transfer

#### Working Memory

Our primary measure of memory improvement was derived from a tablet-based test of Working Memory Capabilities (Test pamie¸ci roboczej- TPR, Educational Research Institute) (Kaczan et al., 2014), designed for children aged 6 and 7 years. It measures three aspects of STM/WM (Oberauer et al., 2003, 2008), including a counting span task, a set switching task and a task that measured memory for spatial locations.

With respect to the counting span task, children counted balls that were the same color as a target box (located at the bottom in the left corner of the screen on each trial) and ignored balls of a different color. The number of balls to be counted on each screen varied from one to five. On each trial children were asked to count the number of balls displayed and to remember and repeat them in the order of appearance. Children completed two practice trials followed by two sets of two, three, four and five screens (eight sets in total, presented in a quasi-random order, so that the difference in number of items to be remembered between adjacent screens was not bigger than two). Scores represent the proportion of correctly recalled balls.

In the set switching task, children were presented with a series of drawings of boys' or girls' emotional faces (happy or sad) presented in a square divided into four quarters (Rogers and Monsell, 1995). The task was to switch between decision making criterions that included a YES/NO response; "YES" to a BOY'S face (if the face appeared in the upper quarter of the screen and irrespective of emotion) or a HAPPY face (if the face appeared in the lower quarter and irrespective of gender), and "NO" otherwise. The task had 6 practice trials and five sets of 12 trials, where the criterion (between gender and emotion) switched after two trials and child response triggered the next trial. On each trial, accuracy was recorded and scores reflected the proportion of correct trials.

In the spatial location memory task children were asked to remember the position of 2–5 ladybirds that appeared sequentially on a matrix (6 × 6 squares) and across 12 trials (there were three trials at each level of difficulty). The test starts with a training trial with two objects (ladybirds). Performance reflects the mean accuracy of remembered sequences between

the presented elements and where higher scores (expressed as a proportion of correct responses) reflected increased accuracy.

#### Executive Attention

Attentional control was measured using a version of the Erikson Flanker task (Eriksen and Schultz, 1979). Participants were asked to identify a centrally presented target (happy face vs. unhappy) and to ignore flanker faces that were the same (congruent trials) or different (incongruent trials) to the central face. All children completed 6 practice trials (3 congruent and 3 incongruent) and 2 blocks of test trials. Each block included 32 randomly presented trials; 22 congruent (11 happy and 11 unhappy faces) and 10 incongruent trials. The main outcome measure in this task is a conflict score, calculated by subtracting the mean RT of the incongruent items from the mean RT of the congruent items on correct trials and where higher scores indicate increased distractor interference from flanker stimuli.

#### Parent-Reported Executive Abilities

We used a Polish version of the Behavior Rating Inventory of Executive Function (BRIEF) (Gioia et al., 2000a). The parent report BRIEF estimates executive function abilities in children ranging from 5 to 18-years-old. It includes two domains (i) behavioral regulation (inhibition, shifting and emotional control) and (ii) metacognition (initiation, WM, planning, organization of materials and monitoring). The questionnaire has 86 items and 72 are used to compute the final score (Gioia et al., 2000a). Answers are given on a 3-point Likert scale (0-never to 2-often, making a total possible global score from 0 to 144 (and sub-scale scores for behavioral regulation and metacognition; maximum scores of 56 and 88 respectively). Parents are asked to rate behavior in relation to the context of their child's everyday home setting (Gioia et al., 2000b). This questionnaire was adapted from an existing measure that was independently translated into Polish by two researchers (both fluent in English). The final version was based on these two translations and a back translation and was evaluated by a group of experts (child psychiatrists and psychologists specializing in ADHD). Questionnaire items were accepted as final when the original and back-translated versions were identical or very similar.

#### Educational Achievement

We used the tablet based Test of Knowledge and Competences (Test Umieje¸tno´scinaStarcieSzkolnym–TUNSS; Educational Research Institute) (Kaczan et al., 2014), designed to measure knowledge and skill in mathematics, reading and writing in 6- and 7-year-olds. (1) The numeracy skills scale includes numbers, measurement, space and shape, relationships and interconnections. (2) The writing skills scale measures visuomotor, visuospatial and audiolingual skills, penmanship and writing. (3) The reading skills scale measures audiolingual, audiovisual and reading skills. The test is completed individually and it includes an introductory phase (aimed at building a relationship with children and to familiarize them with the tablet and program) and a test phase. In the test phase the child is asked to complete a maximum of 10 tasks from each scale (reading, writing and numeracy skills) and the TUNSS score provides an estimate of a child's skill levels in each area. The score was computed as a variable theta (normal distribution, scale with M = 0 and SD = 1) for each scale (reading, writing and numeracy).

#### IQ

We used the Raven's CPM (Raven et al., 1986) to measure general intelligence. The test is designed for 4–10-year-olds and consists of 36 items in three sets that test a child's ability to complete continuing patterns, to perceive spatial relations and for abstract thinking. The items are presented in the form of incomplete patterns or matrices and children are asked to choose the missing element from a given set of possible answers. Each item was scored 1 for correct answer or 0 for incorrect answer. The maximum score is 36. **Figure 2** presents a summary of all pre- and post-tests, follow up measures and WM training tasks.

# Procedure

This study was approved by the Institutional Research Ethics of Empirical Research with the Participation of People as Persons Researched. Parents and children were invited to take part in the study via a letter and each provided informed written consent. The children were asked to practice on the training task for 15 min a day on their personal computers at home, 5 days a week for 8 weeks. There were two inclusion criteria for both groups: participants were required to have an IQ score of ≥85, and we asked for an overview of the child's educational and social functioning, as reported by their teacher.

Participants were assessed individually in the University of Social Sciences and Humanities laboratory at three different time points; prior to training (pre-test/T1), post-test-no more than 1 week after training was completed (T2) and follow up-3 months after completing training (T3). All assessments were conducted by research staff in a laboratory, where children also completed the working memory, attention control measures, educational achievement, and IQ test. Additionally, parents of children completed the BRIEF scale.

# RESULTS

#### Statistical Analysis

We contrasted the training trajectories for the ADHD and the control (typical develop children–TD) groups by using latent growth curve modeling (LGCM) for time series data from the eight training blocks (one block made up 5 training sessions) (Duncan et al., 2011). First, an intercept-only model (a model with an assumption of no change in time) and then models with linear and quadratic slopes were estimated. The fit of individual models and comparisons between models were based on the chi-square value (χ 2 ), the normed chi-square (χ 2 /df), the non-normed fit index (also known as Tucker–Lewis Index, TLI), the Comparative Fit Index (CFI), the Root Mean Square Error of Approximation (RMSEA), the Standardized Root Mean Square Residual (SRMR), and the Information Criteria Indexes: the Bayesian Information Criterion (BIC), and the Akaike Information Criterion (AIC).

A non-significant χ 2 test indicates a good model fit, although its use is not free from limitations, especially for small sample sizes. Thus, the ratio of χ 2 to degrees of freedom was also used, with values lower than 2 assumed to be satisfactory (Tabachnick and Fidell, 2007). The TLI and CFI threshold values of 0.90 indicate satisfactory fit, whereas values above 0.95 indicate good fit (Hooper et al., 2008). For a well-fitting model, the RMSEA should be close to 0: values below 0.05 indicate a good model fit and a value of 0.08 represents reasonable errors of approximation, whereas the SRMR should be below 0.10. The model with the lower information criterion values are preferable as they indicate better fit. The group comparison (ADHD vs. TD) was based on χ 2 Wald test. The maximum likelihood (ML) as estimator was applied (Muthén and Muthén, 1998). Factor loadings corresponded directly to the blocks timing.

We then used growth mixture modeling (GMM) to identify sub-groups of individuals across groups with distinct training trajectories using data from the eight training tasks. GMM was performed with Mplus Version 7.3 (Muthén and Muthén, 1998). GMM allows the existence of latent homogeneous subpopulations of individuals within heterogeneous samples to be identified (i.e., the latent classes of individuals characterized by different learning curves). The maximum likelihood with robust standard errors (MLR) as estimator was applied (Muthén and Muthén, 1998). Factor loadings corresponded directly to the blocks timing. Both linear and quadratic slopes were estimated. Analyses were performed separately for (1) all training tasks combined, and for (2) VSSTM training tasks and (3) VSWM training tasks separately, and models for 1–4 classes were computed.

The models were compared using the Bayesian Information Criterion (BIC), the Akaike Information Criterion (AIC), the Bootstrap Likelihood Ratio Test (BLRT), the Vuong-Lo-Mendell-Rubin Likelihood Ratio Test (VLMRLRT), the entropy value, the subsample size, and practical usefulness of the latent training efficiency curve classes (Jung and Wickrama, 2008; Duncan et al., 2011). The model with the lower BIC and AIC values, greater entropy value (>0.80) indicated good fit (Jung and Wickrama, 2008; Duncan et al., 2011). The BLRT and VLMRLRT tests compare two models with different parameters. Significant pvalues of tests indicated that the estimated model is preferable over a model with one fewer latent class (Nylund et al., 2007). The results were replicated to avoid local solutions (Jung and Wickrama, 2008).

Finally, we compared the subgroups identified using GMM in terms of pre-, post-intervention changes in our transfer of training measures. In order to compare training effects for sub-groups of children with different trajectories using

one-way ANOVA analyses (SPSS Version 22) with GMM sub-group membership as baseline (pretest outcome measures) and change indicators including (1) pretest—posttest (T1T2) and (2) pretest—follow-up (T1T3) as the dependent (created by standardized regression residuals) (Duncan et al., 2011). Increasingly positive scores indicated positive change, scores that tend to zero reflect no change, and negative scores indicate negative change.

# Missing Data

Missing training efficiency data varied between 2.2 and 28% per participant across training blocks. In order to maintain the full time series missing data was modeled using Full Information Maximum Likelihood (FILM) estimation with robust standard errors (Asparouhov and Muthen, 2010). This approach uses all observations in the dataset to produce the maximum likelihood estimation parameters without imputing data and is reported to be one of the best approaches currently available to handle missing time series data (Graham, 2009). Missing data in non-trained outcome measures (working memory, executive attention, educational achievement and IQ) varied from 1.1 to 28.9% across all measures. We used Expectation Maximization (EM) (Dong and Peng, 2013) to deal with missing pre- and postintervention data, that were imputed with maximum likelihood values (Acock, 2005; Schlomer et al., 2010; Dong and Peng, 2013). Additionally Missing Value Analysis (Little's MCAR test) was consistent with a "missing at random" assumption for training tasks and non-trained outcome measures (Chen and Little, 1988).

# Training Compliance

Ninety participants (71% overall), 41 TD (67%) and 49 ADHD (75%), completed 15 or more training session within 8 weeks (our minimum compliance criteria). Fifteen children did not initiate training, 21 children withdrew before finishing 15 sessions. Overall, compliance to the intervention was good, in the context of a long and intensive home-based intervention parentsupported intervention delivered via the Internet (40 sessions over 8 weeks). Across 90 participants, 94% (N = 85) finished 20 sessions (1 block), 91% (N = 82) finished 25 (2 blocks), 88% (N = 79) finished 30 (3 blocks), 72% (N = 65) finished 35 (4 blocks), and 62% (N = 56) finished 40 sessions (5 blocks).

#### Process Measures

Compliance was defined as completing ≥15 of the 40 training sessions within a 8-week period. Using this algorithm, each child was categorized as compliant or noncompliant to treatment.

# Does Training Efficiency Follow Different Trajectories Between ADHD and TD Groups?

**Table 1** shows the fit indicators for the unconditional growth models (stability, linear and quadratic change) based on all training (VSSTM and VSWM) tasks performed over 8 blocks of training. For all outcomes the quadratic slope model fitted better than the intercept-only and linear models, thus, the assumption of quadratic change over training in the whole sample is preferred, over the assumption of no change or linear change.

Means and variance of intercept and slopes in each task were significantly different from zero (see **Table 2**). When group was considered as a whole there was a significant increase and then decrease in latent tasks scores across time (i.e., the quadratic slope has a negative sign). There was also a significant slope factor variance. That means that children differed, not only in their initial latent task scores, but also in their training trajectories over time. However, the training trajectories of ADHD and TD groups did not differ significantly across all tasks (χ 2 Wald test = 0.155, df = 1, p = 0.694), for the VSSTM (χ 2 Wald test = 0.895, df = 1, p = 0.344), and the VSWM (χ 2 Wald test = 3.706, df = 1, p = 0.054) tasks.

# Are There Sub-groups of Individuals Marked by Different Trajectories of Learning/Training Efficiency?

**Table 3** presents the descriptive statistics for performance across training sessions for all tasks, and for the VSSTM and VSWM tasks separately and for the participant group as a whole. Models were generated for the total group of participants and across 8 blocks of training efficiency (see **Table 4**). With respect to all training tasks, the entropy value and the AIC value supported a two-class model, whereas the BIC value supported the one-class solution. VLMRLRT and the BLRT tests indicated that two- or more-class models were not superior to the one-class solution. Based on these indices, the one-class model was considered as the best fit model. With respect to the VSSTM, the AIC, the

TABLE 1 | Fit indicates for latent growth curve modeling (LGCM) models based on all memory training tasks (WMT) combined, and for visuospatial short-term training tasks (VSSTM) and visuospatial working memory training tasks (VSWM) over 8 learning sessions and across the whole sample (N = 90).

AIC, Akaike's Information Criterion; BIC, the Bayesian Information Criterion; RMSEA, the Root Mean Square Error of Approximation; CFI, the Comparative Fit Index; TLI, the non-normed fit index; SRMR, the Standardized Root Mean square residual.

TABLE 2 | Means and variances in latent growth curve modeling (LGCM).

WMT, all training tasks; VSSTM, visuospatial short term memory tasks; VSWM, visuospatial working memory tasks; \*p < 0.01; \*\*p < 0.001.

BIC values, as well as the VLMRLRT test, supported the oneclass model. In contrast, the p-values of the BLRT test indicated that the two-class model was superior ion. Based on these indicators, but primarily the meaningfulness of the identified curves of training efficiency and class sizes, the one-class model was chosen. With respect to the heterogeneity of the learning curve of VSWM, the AIC and the BIC values supported the fouror one-class model respectively. The remaining indices supported the three-class solution. Based on the meaningful curves of training efficiency the three-class model was considered as the best fitting model.

According to GMM analyses, the whole sample (N = 90) across all WMT tasks represented an inverted U-shaped category (intercept = 9.22, SE = 0.17, slope = 0.87, SE = 0.09, quadratic = −1.13, SE = 0.01, p < 0.001), indicating gradual increase to Block 3 (15 sessions), followed by stability and then a decrease thereafter (see **Figure 3A**). Similar results were found for VSSTM tasks performance. Here training efficiency patterns followed a curvilinear trajectory, increasing to Block 3, stability and followed by a decrease (intercept = 9.71, SE = 0.18, slope = 0.86, SE = 0.09, quadratic = −0.11, SE = 0.01, p < 0.001), see **Figure 3B**. The analysis for the VSWM produced a three-class model. In this case, more than half of the sample (N = 69; 76.7%) was characterized by an inverted U-shaped (see class 2; intercept = 8.41, SE = 0.23, slope = 0.47, SE = 0.08, quadratic = −0.06, SE = 0.01, p < 0.001), with performance, increasing to block 3, stability and subsequently decrease. The second group (class 1, N = 14; 15.5%) was characterized by a U-shaped curve, with the highest starting scores then deterioration of performance up to block 5 and then slight increase (intercept = 10.05, SE = 0.83, slope = −0.94, SE = 0.25, quadratic = 0.13, SE = 0.03, p < 0.001). The smallest group (class 3; N = 7; 7.8%) was characterized by a high inverted U-shaped curve (intercept = 7.48, SE = 0.99, slope = 2.15, SE = 0.26, quadratic = −0.24, SE = 0.03, p < 0.001), highlighting the lowest starting scores and the highest improvement in performance across training (see **Figure 3C**).

# Do GMM Training Trajectory Groups Predict Transfer to Different Measures?

**Table 5** presents the descriptive statistics for outcome measures for the three trajectory groups (from GMM in VSWM training tasks). Considering associations between learning and change between pretest-posttest (T1T2) outcome measures, the results showed a statistical trend toward significance for group for the spatial location memory task [F(2,89) = 2.60, p = 0.080, η 2 = 0.06]. Here, the class 3 achieved the highest change T1T2 score in spatial location memory task (M =0.39, SD = 0.18). Furthermore, class 1 had a trend toward a significant greater improvement in T1T2 spatial location memory task (M =0.32, SD = 0.13), compared with sub-group class 2. All other results were not significant.

TABLE 3 | Performance for working memory tasks for the typically developing and children with ADHD over 8 blocks (each with 5 sessions) for all working memory training tasks (WMT) combined, and for visuospatial short-term training tasks (VSSTM) and visuospatial working memory training tasks (VSWM) separately (N = 90).

TABLE 4 | Fit indices for GMM models based performance for WM tasks for the typically developing and clinical intervention groups over 8 blocks (with including 5 sessions) for all tasks combined (WMT) and for visuospatial short term memory (VSSTM) and visuospatial working memory tasks (VSWM) separately (N = 90).

AIC, Akaike's Information Criterion; BIC, the Bayesian Information Criterion; VLMRLRT, the Voug-Lo-Mendel-Rubin Likelihood Ratio Test; BLRT, the Bootstrap Likelihood Ratio Test. Best fitted parameters are bolded.

aModel did not converge.

# DISCUSSION

The current study extended previous research to utilize GMM to explore learning/training efficiency during an extended WM training protocol for TD children and children given a diagnosis of ADHD. We examined performance curves using GMM across 40 training sessions (represented in 8 blocks) to explore performance between groups and across the sample. In addition, we investigated whether child characteristics were important in understanding individual differences in learning. Furthermore, we considered learning for simple and complex training tasks on training efficiencyand transfer to tasks that were conceptually similar to trained tasks (near transfer), as well as those not associated with training (far transfer).

Considering group differences in training efficiency, the results showed that young children diagnosed with ADHD did not differ from the TD group in their learning trajectories, and where this result was evident for all training tasks combined and when considering training on learning simple VSSTM, as well as more complex VSWM tasks. Specifically, the results showed that across groups learning during WM training (across all WM and for simple short term memory tasks) was characterized by an inverted-U highlighting initial learning (increased performance to block 3; around 15 training sessions), stability and then gradual decrease in performance. When learning more complex tasks (VSWM), three independent learning curves were identified; most children followed the same inverted-U trajectory described above, and two smaller groups were characterized by a U-shaped curve (a high starting point, followed by initial deterioration and then increase) or a sharp inverted-U, that showed the largest increase in learning over the first three blocks. Further analysis highlighted that this latter group showed the lowest pre-training WM scores and the highest T1T2 change.

The results provide a novel set of findings associated with training WM in young children. They highlight that across training tasks learning was most evident in the first 15 training sessions, it then remained stable and showed some deterioration thereafter. The results indicate that WM training is most effective in the short term, and that young children (at least in this sample) did not benefit from an extended approach to training. These results contradict previous findings from adult samples, where a greater number of training sessions was linked to more effective outcomes (Jaeggi et al., 2008; Schmiedek et al., 2010). This difference raises the possibility that the motivation for intense and sustained intellectual effort across weeks in young children is different compared with adults. It is possible that adults have

TABLE 5 | Performance for pre-test outcome measures for three trajectory groups (from GMM in VSWM training tasks).

Pretest measures: Test of Working Memory Capabilities (WM Global score and subscale scores: Counting Span, Set Switching, Spatial Short Term Memory), Attentional Control (Flanker Conflict Score), Behavior Rating Inventory of Executive Function (BRIEF) Questionnaire (Global and Behavioral Regulation and Metacognition Subscale Scores), Test of Knowledge and Competences (Global and Maths, Writing and Reading skills subscale scores) and IQ (Raven's Colored Progressive Matrices).

increased metacognitve skills in learning and therefore may be more aware of the potential benefits of their efforts. Alternatively, children may have a limited ability to acquire new mnemonic strategies, and while existing techniques allow them to learn quickly at the beginning of training, this limitation may prevent them from developing further as the training progresses. This explanation is consistent with the development of short term memory, which highlights that it is not until middle childhood (around 7 years of age) that children will typically start to utilize mnemonic strategies to enhance performance in memory tasks (Cowan, 1997). Further studies into the potential moderators of outcome, for example, an exploration of the function of mnemonic techniques and metacognitive awareness in learning would allow for further delineation of children who might benefit most from training.

The results also highlighted training in the most complex tasks revealed a different pattern of learning efficiency. Specifically, they indicated that a small subgroup of children who performed less well at pre-test benefitted most from training and achieved the highest change score (T1T2) in complex spatial WM tasks (showing a high inverted U-shaped training curve). This finding highlights most benefit for this group of children on near transfer tasks, i.e., those that tap working memory function (vs. attentional control more broadly; see also Oberauer et al., 2008). The current study did not reveal benefits to executive functioning more broadly or to parent reports of daily behavior and educational achievement, see also Chacko et al. (2014) for similar results. In addition, one further group of children in the current study did not show any benefit of training. While the lack of transfer to distant, non-trained tasks in the current study might reflect the decline in learning over the course of the program, this result is consistent with recent reviews of WM training which suggest that WM training has most impact on tasks that are most similar to those that are trained (Shipstead et al., 2010). In the current paper, the results suggest that children who show difficulties with WM benefitted most from training. Future research should be focused more carefully on identifying further sub-groups that might benefit from WM training.

The current study revealed that most children were able to show improvements in WM tasks during training and where this increase was most effective for 15 sessions of training (with stability and decrease thereafter). In addition, it highlighted that children who experienced the lowest scores on WM tasks showed most learning and broader benefits post-training in similar nontrained WM tasks. There was no evidence of broader benefits in attention tasks or on parent-reported daily functioning and educational achievement. Future research should focus on possible enhancement of the capacity working memory through training and the possibility of far transfer (e.g., academic skills). Further studies into the potential moderators of outcome, for example, an exploration of the function of mnemonic techniques and metacognitive awareness in learning would allow for further delineation of children who might benefit from training. The results indicate that those young children who show difficulties with tasks that require attentional control in an educational context might benefit from WM training, and further research should focus on understanding whether there are longer term benefits of enhancing working memory skills in the short term. In addition, the research field would benefit from the use of standardized and reliable and valid tests to assess executive functions and evaluate the effectiveness of training tasks. This could facilitate a more accurate analysis of cognitive training effectiveness and the stability of its effects over time.

The present study had several limitations. In order to understand potential change associated with time or the active nature of this intervention, the study would have benefitted from a placebo intervention group, as well as a passive control group. In addition, WM training was conducted via the Internet and the children participated under parental supervision, consequently, we were not able to monitor treatment integrity across the intervention. Moreover, a diagnostic approach that generated a continuous measure of ADHD would have enabled a comprehensive profile of symptoms across the entire sample.

#### DATA AVAILABILITY

Please see registration: ISRCTN15153056 https://doi.org/10. 1186/ISRCTN15153056 working memory training effects on young children's functioning.

#### ETHICS STATEMENT

This study was carried out in accordance with the recommendations of Study Protocol Approved by Ethics Committee, SWPS University of Social Sciences and Humanities, Warsaw, Poland "Training, motivation and maturation influences on the development of executive functions at early childhood," [Komisja ds. Etyki Badan Empirycznych ´ z Udziałem Ludzi jako Osób Badanych, Warszawa, Polska "Wpływ treningu motywacji, dojrzewania na rozwój funkcji wykonawczych u dzieci we wczesnym wieku rozwojowym"] with

#### REFERENCES

written informed consent from all subjects. All subjects gave written informed consent in accordance with the Declaration of Helsinki. The protocol was approved by the Ethics Committee, SWPS University of Social Sciences and Humanities, Warsaw, Poland; number 2/V/11-12 [Komisja ds. Etyki Badan Empirycznych z Udziałem Ludzi jako Osób ´ Badanych, Warszawa, Polska; nr 2/V/11-12].

#### AUTHOR CONTRIBUTIONS

AO, JH, AK, and ES-B: contributed to the conception and design of the study; AO: organized the database, the project administration and was the research leader; AO and AK: performed the statistical analysis; AO, AK, and JH: wrote the first draft of the manuscript; AO, JH, AK, and ES-B: wrote sections of the manuscript. All authors contributed to manuscript revision, read and approved the submitted version.

### FUNDING

This research and preparation of the paper were supported by grants from the National Science Centre, Poland (UMO-2011/03/N/HS6/04849 and UMO-2014/12/T/HS6/00216).

**Conflict of Interest Statement:** The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Copyright © 2019 Orylska, Hadwin, Kroemeke and Sonuga-Barke. This is an openaccess article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

# A Mindfulness-Based Intervention for Students With Psychiatric Disorders in a Special Education Curriculum: A Series of n-of-1 Trials on Internalized and Externalized Symptoms

Catherine Malboeuf-Hurtubise1,2 \*, Geneviève Taylor 2,3, Linda Paquette2,4 and Eric Lacourse5

1 Department of Psychology, Bishop's University, Sherbrooke, QC, Canada, 2 Research Unit on Mindfulness (GRIPA), Montreal, QC, Canada, 3 Department of Education and Pedagogy, Université du Québec à Montréal, Montreal, QC, Canada, 4 Department of Health Sciences, Université du Québec à Chicoutimi, Saguenay, QC, Canada, 5 Department of Sociology, Université de Montréal, Montreal, QC, Canada

Background: Mindfulness-based interventions (MBIs) constitute a promising option to alleviate psychological symptoms in students with psychiatric disorders.

Edited by: Claudio Longobardi, Università degli Studi di Torino, Italy

#### Reviewed by:

Sharinaz Hassan, Curtin University, Australia J.E. Hall, University of Southampton, United Kingdom

#### \*Correspondence:

Catherine Malboeuf-Hurtubise catherine.malboeuf-hurtubise@ ubishops.ca

#### Specialty section:

This article was submitted to Educational Psychology, a section of the journal Frontiers in Education

Received: 12 April 2018 Accepted: 31 July 2018 Published: 03 September 2018

#### Citation:

Malboeuf-Hurtubise C, Taylor G, Paquette L and Lacourse E (2018) A Mindfulness-Based Intervention for Students With Psychiatric Disorders in a Special Education Curriculum: A Series of n-of-1 Trials on Internalized and Externalized Symptoms. Front. Educ. 3:66. doi: 10.3389/feduc.2018.00066 Objective: This study evaluated the impact of a MBI on psychological symptoms of elementary school students with psychiatric disorders in a special education curriculum.

Method: A series of n-of-1 trials with an experimental A-B-A design and 10 assessment time-points was used. Participants were two special education classrooms of elementary school students (Group A, n = 7; Group B, n = 6; ntotal = 13) and their teachers.

Results: Analyses showed that, at follow-up, students from Group A reported significant decreases in inattention and a trend (p = 0.051) was observed in anxiety symptoms, whereas their teacher reported only deteriorating anxiety scores. No significant results were found in students from Group B.

Conclusion: Results from this study show substantial variation between students, tested cohorts and raters of internalizing and externalizing symptoms and behaviors. Therefore, the conservative conclusion from such variation can only be that more research is needed, as no firm conclusion as to the utility of MBIs for such groups of children with special education needs can be established.

Keywords: school psychology, mindfulness-based interventions, psychiatric disorders, internalized symptoms, externalized symptoms, mental health

# INTRODUCTION

In Canada, 20% of students suffer from a mental health problem, causing significant educational and psychosocial adaptation difficulties (Canadian Teachers' Federation., 2012). These problems jeopardize academic achievement and lead to impaired functioning in school-based settings (Oberle et al., 2014). Students with psychiatric disorders show important deficits on several levels: cognitive (e.g., more or less severe and frequent loss of contact with reality, incoherent speech, academic delays), emotional (e.g., anxiety, irritability, and/or depression symptoms, aggressiveness), behavioral (e.g., bizarre or incoherent behavior, limited interests, opposition, and disorganized behavior) and social skills (e.g., withdrawal, rejection from peers). They also present significant levels of internalized and externalized symptoms (Lavoie et al., 2017). These difficulties can be linked to significant deficits in emotional regulation skills. Special education classrooms for students with severe psychiatric disorders typically regroup youth showing a vast array and heterogeneity in symptoms and diagnoses, namely because of administrative constraints and paucity of space and resources. Thus, these children are very often regrouped within the same classrooms. Faced with such an important proportion of psychological problems, it appears crucial to develop empirical and skill-based interventions adapted to the reality of these very heterogeneous classrooms. Mindfulness-based interventions (MBIs) are increasingly implemented in school-based settings to foster better emotional regulation skills and higher resilience to stress in students with special education needs such as severe learning disabilities and psychological disorder diagnoses (Malboeuf-Hurtubise et al., 2017c). This article presents a first, small-scale attempt to conduct research on a MBI with two small classes of special needs boys. Specifically, the goal of this n-of-1 trial design was to evaluate the impact of a MBI on internalized and externalized symptoms in two groups of elementary school students with psychiatric disorders in a special education curriculum.

# Mindfulness-Based Interventions in School-Based Settings

Mindfulness can be defined as ". . . the process by which we pay attention in a particular way, on purpose, in the present moment, and non-judgmentally" (Kabat-Zinn, 1994). Mindfulness research in youth is still considered to be in its early phase, although there has been a significant amount of articles published in recent years documenting the impact of MBIs for children and adolescents (Taylor and Malboeuf-Hurtubise, 2016). Evidence from a recent meta-analysis suggests that MBIs hold promise in reducing internalized symptoms (d = 0.37) such as anxiety, depression, and inattention, while promoting better emotional and stress regulation skills in youth both in regular classrooms and with an identified psychological disorder (Zenner et al., 2014; Zoogman et al., 2014). For the purposes of this article, we present results from research evaluating the impact of MBIs that were adapted, developed and inspired from the Mindfulness-Based Stress Reduction (Kabat-Zinn, 2003) and/or Mindfulness-Based Cognitive Therapy (Segal et al., 2018) interventions, in which the major component of all cited interventions constitutes the practice of mindfulness meditation itself.

#### Mindfulness-Based Interventions as Universal Prevention

In school settings, the impact of MBIs has been studied with students in regular classrooms and students in a special education curriculum. In regular classrooms, from a universal prevention perspective, mindfulness has been shown to decrease inattention symptoms and overall internalized problems in elementary school students with no known psychological difficulties, when compared to students in an active emotion awareness intervention control group (Crescentini et al., 2016). Decreases in anxiety have also been reported in elementary school children from regular classrooms, in comparison to wait-list controls (Tarrasch et al., 2017). In a recent study, Sibinga et al. (2016) have reported similar decreases in depressive symptoms, rumination and negative affect in elementary school students from regular classrooms in underprivileged neighborhoods, when compared to an active health education control group, indicating that MBIs can be useful in alleviating the negative impact of stress in these students. Finally, using a similar study design, Schonert-Reichl et al. (2015) have reported that mindfulness had a positive impact on elementary school students' social and emotional regulation skills, while decreasing depressive and aggressiveness symptoms. Overall, the available evidence tends to indicate that MBIs are useful in promoting better mental health and coping strategies in school settings for children in regular classrooms with no identified psychological diagnoses, namely by acting preventively on stress and on the emotional burden that children experience daily.

#### Mindfulness-Based Interventions as a Transdiagnostic Treatment Option and as Targeted Prevention

In recent years, researchers have started to investigate and evaluate the impact of transdiagnostic treatments to alleviate symptoms of psychological disorders in youth and adults (Boswell et al., 2014). This line of research aims to evaluate and develop unified evidence-based treatment options that can be applied across different psychological conditions and diagnoses. As such, MBIs have been identified as potentially good transdiagnostic options for treating various psychological disorders in children (Malboeuf-Hurtubise et al., 2017b). However, to date, this research remains preliminary and results are still inconclusive.

Students with identified disabilities in special education curriculum classrooms rarely present homogeneous conditions and diagnoses, and, thus, would greatly benefit from the development of such transdiagnostic MBI treatments. However, the amount of research published on the impact of MBIs for elementary school students with special education needs (e.g., students in a special education curriculum or students with an identified mental health disorder) is far less voluminous. The available evidence shows contradictory results regarding the benefits of mindfulness practice in this population. For example, a recent article presenting results from a quasi-experimental, one group pre-test post-test design for elementary school students with severe learning disabilities showed that MBIs could be useful in decreasing anxiety, depression, inattention, and aggressiveness in these students, thereby alleviating the psychological distress and overall mental health burden associated with severe learning disabilities (Malboeuf-Hurtubise et al., 2017c). On the other hand, inconclusive results were found in three elementary school students suffering from major depressive and anxiety disorders, as two participants reported improvements on anxiety and depression, while their teachers reported deteriorating scores on these variables (Malboeuf-Hurtubise et al., 2017b). These preliminary, contradictory results suggested that caution should be exercised when considering the overall applicability of MBIs as a universal treatment option for children with special education needs in elementary schools. Furthermore, the small number of current studies highlights the need to document the impact of MBIs in greater detail for students who have underlying mental health issues, to determine in which contexts and with what populations (e.g., as a transdiagnostic clinical intervention for children with a variety of mental health disorders or a clinical intervention for children with a specific mental health disorder, such as an anxiety disorder, ADHD or major depressive disorder) this practice can be of use to decrease internalized and externalized symptoms.

Elementary school students struggling with severe mental health problems are particularly vulnerable to physical and psychological stress, along with psychological distress (Mental Health Commission of Canada, 2013). Given the high cost of these psychological difficulties on a personal, social and school level, it is paramount to pay attention to these issues, in order to foster better mental health and to avoid future psychosocial adaptation problems in these students (Smetanin et al., 2015). However, there is a paucity of available MBI research on this issue. Only two studies were found pertaining to the impact of a MBI for adolescents with psychiatric disorders in a psychiatric setting. Results from these studies indicated that the intervention had an overall positive impact on mood (decreasing, namely, anxiety and depression), self-regulation, self-esteem, and social skills in adolescents with severe mood and anxiety disorders, oppositional-defiant and conduct disorders, attention deficit and hyperactivity disorders and reactive attachment disorders (Biegel et al., 2009; Van Vliet et al., 2017). To our knowledge, no published quantitative study has ever evaluated the impact of a MBI for children with similar psychiatric disorders in a schoolbased setting.

#### N-of-1 Trials

N-of-1 trials are used to evaluate within-subject differences over time. Research projects using N-of-1 trials can be comprised of one person (i.e., case study) or multiple individuals (Wood and Brown, 1994; Molenaar and Campbell, 2009). This design has been suggested to be especially relevant and informative in intervention development and research, such as MBI research for youth in school-based settings. Through the use of repeated measures, N-of-1 trials provide large amounts of data that allow for a detailed study of change in targeted variables, longitudinally (Schork, 2015). Although the main concern with n-of-1 trials is not generalizability, with large enough sample sizes, results from multiple n-of-1 trials can be aggregated to provide an estimated global effect of a given treatment option for a specific population, such as students in a special education curriculum (Schork, 2015).

N-of-1 trials are considered to be a sound, experimental design alternative to studies incorporating larger sample sizes, allowing for a more detailed measure of the amplitude of change through means of regularly scheduled repeated assessment (Vohra et al., 2016). Furthermore, n-of-1 trials are considered to be especially appropriate when small pools of participants are available—e.g., in program evaluation studies targeting a specific population such as students with special education needs—and when a control group is not available (Spector, 1981; Harris et al., 2006; Gravetter and Forzano, 2011; Vohra et al., 2016). N-of-1 trials are considered to be a rigorous approach that allows an evaluation of the impact of interventions such as MBIs, while narrowing the gap between educational or psychological research and practice (Vohra et al., 2016). As an example, n-of-1 trials can be especially useful to evaluate the impact of alternative and complementary therapies such as MBIs for youth.

# PRESENT STUDY