Juan (pseudonym) is a Caucasian male of Hispanic ethnicity who was eight years 11 months at the time of intake. Juan was adopted at age 17 months and no information is available about his development prior to that age. From adoption through the time of the evaluation he resided with his adoptive parents who were middle class and resided in a southeastern part of the U.S. His medical history was unremarkable. Juan's parents reported that they first became concerned about his behavior when he was about two years old due to high rates of noncompliant and disruptive behavior at home and in daycare. They also reported that he had been consistently disruptive in school starting from kindergarten. Juan had a history of treatment with psychoactive medications, including Focalin XR, Intuniv, Adderall, and Vyvanse, but these were discontinued prior to his enrollment in the treatment described here. He had previously received psychosocial treatment for behavioral and emotional control problems, which his parents described as moderately effective. Juan resided with his adoptive mother and adoptive father and had no siblings. Juan had recently completed third grade at his local public elementary school.
Juan was referred for treatment by his mother due to her concerns about his disruptive behavior at home and school. Primary concerns from his mother included defiance, tantrums, verbal abuse, aggression, poor impulse control, inattention, excessive silliness, and low-frustration tolerance. Juan's mother and teacher reported that he did not seem to care about his performance (e.g., school work), did not seem concerned about feelings of others, did not keep promises that he made, and lacked remorse following misbehavior. His teachers also described him as lacking warmth and kindness. At school, Juan also struggled with inattention, hyperactivity, and poor peer relationships. Given these presenting concerns, Juan's mother sought to enroll him in a summer treatment program (STP) conducted at a university based mental health clinic. Juan's parents provided written consent for participation in this treatment study and Juan provided verbal assent. All procedures followed American Psychological Association standards for ethical collection and use of patient information, including disguising relevant information to hide the identity of the participating child.
Diagnoses were determined using criteria specified in the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV). Following recommended guidelines for evidence-based assessment of disruptive behavior, diagnoses were based on both symptom and impairment criteria as evaluated using multiple sources of information. Specifically, oppositional defiant disorder (ODD), conduct disorder (CD), and attention-deficit/hyperactivity disorder (ADHD) symptoms were evaluated using parent and teacher ratings on the Disruptive Behavior Disorders Rating Scale (DBDRS) and using parent responses to the computerized version of the Diagnostic Interview Schedule for Children for DSM-IV. Impairment was evaluated using parent and teacher ratings on the Impairment Rating Scale (IRS). These data (see Table 1) were independently reviewed by two Ph.D. level clinicians who each assigned Juan diagnoses of ADHD and ODD.
The diagnostic evaluation also included an assessment of internalizing disorders. Anxiety, as measured by the Revised Children's Manifest Anxiety Scale 2nd edition, was in the normative range with a total anxiety T-score of 44. Depression, as measured by the Children's Depression Inventory 2nd edition, was also in the normative range with a total depression T-score of 44.
CU traits were evaluated using two measures: the Antisocial Process Screening Device (APSD) and the Inventory of Callous-Unemotional Traits (ICU). As shown on Table 1, Juan's scores on these measures indicated very high levels of CU traits. His scores on the APSD place him in the top 1% (mother) to 5% (teacher) compared to other boys his same age. Likewise his scores on the ICU are substantially above recently proposed cutoffs on this measure.
Treatment was implemented in an STP, which is an evidenced based treatment program for youth with disruptive behavior problems that is delivered in the context of a summer camp setting. The STP occurred five days a week (Monday through Friday) from 8:00 AM to 5:00PM for eight weeks. Juan was placed in a group with 17 other children ranging from six to twelve years of age, comprised of both boys (n=15) and girls (n=2). He attended 39 out of 39 days of the intervention (there was no treatment on the fourth of July).
The primary approach to treatment was behavior management with a focus on positive reinforcement, effective commands, and immediate consequences. Using token reinforcement and response cost methods, Juan received points for appropriate behavior and lost points for inappropriate behavior as he engaged in activities throughout the day. Of most relevance to this study is that treatment also included use of a daily report card (DRC). Three behavioral targets were included on Juan's DRC: (1) interruptions, (2) rule violations during classroom activities, and (3) teasing peers. These goals were selected based on observations of his behavior during the first week of treatment, discussions with his parents, and inspecting information (ratings and written reports) from his classroom teacher in his regular school setting. Juan's goals on his DRC did not change during treatment, but the criteria that defined success on the goals varied from week to week as described below. The DRC was always backed by a reward system that consisted of the chance to earn free time at noon and again at the end of the day, along with home-based rewards delivered by his parents, which consisted of access to screen time (TV, computer, video games). This reward was determined through discussion with Juan, his parents, his lead counselor, and the clinical supervisor. Delivery of home rewards was encouraged by informing Juan and his parents at the end of each day whether Juan earned his reward and if so reminding parents to deliver the reward to Juan. The DRC was also one part of determining whether Juan earned a weekly reward; to participate in a fun field trip on Friday, Juan was required to have achieved at least 75% of his treatment goals on the majority of days in a given week.
The majority of each day consisted of recreationally-based therapeutic group activities (i.e., basketball, soccer, baseball or kickball), an art period, lunch, and swimming. These were led by paraprofessional counselors that included one graduate student lead counselor and five undergraduate counselors. Behavioral treatment was delivered during these activities by using a token economy system and cognitive-emotional treatment was delivered between activities by using discussion, role playing, and problem solving. The remainder of each day (two hours) was spent in a classroom setting that was run by a teacher and teacher aide. Children completed individual seatwork (reading, math, writing) that was appropriate for their academic ability, followed by peer tutoring activities to foster reading skills. The behavior management used in the classroom was a response-cost system in that children began class with a set number of points and points were taken for each violation of a classroom rule, which included: be respectful of others, obey adults, work quietly, use materials and possessions appropriately, remain in your assigned seat or area, raise hand to speak or to ask for help, and stay on task. All activities were supervised by Ph.D. level clinicians.
The treatment manipulation of interest was whether Juan reacted differentially to fixed versus randomly determined target behavior criteria on his behavioral goals on his DRC. That is, Juan always had the same three goals on his DRC, but the criteria used to determine whether he was successful in achieving the goal (i.e., his target behavior criteria) were decided in different ways -- either fixed ahead of time or randomly determined after the fact. For example, reducing interruptions was always a goal on his DRC, but the number of interruptions he was allowed display and still achieve a reward for reducing interruptions goal was either fixed or random, with condition varying across weeks of treatment.
During the fixed-criteria weeks (weeks 2, 5, and 6), the target behavior criteria were determined a priori by his counselors who examined his past behavior and selected a number that represented about 20% improvement. This was done at the start the week and target behavior criteria remained constant throughout the week. Behavioral goals on his DRC, including the fixed criterion for achieving success at each goal, were reviewed with Juan at the start of each day (e.g., "you need to have four or fewer interruptions to meet your interruptions goal this morning"). If he met his criterion goal, a "yes" would be circled on his DRC. Throughout the day, Juan was prompted by staff to keep track of his behavior in relation to his criterion goals (e.g., "you have two interruptions and you can't have more than four to make recess"). Juan was assigned a counselor who reviewed his DRC target behaviors and criterion goals each morning and after each reward period.
During the random-criteria weeks (weeks 3, 4, 7, and 8) the target behavior criteria were determined randomly and post hoc. Specifically, Juan was told that the number of behaviors on his DRC that he was allowed to display and still earn his reward was going to be a mystery that would be revealed just before the reward period. Behavioral goals were reviewed with Juan at the start of and throughout each day ("you need to work on reducing your interruptions today") but no specific target behavior criteria were provided. Instead, just before the reward activity that reinforced the DRC, Juan chose a number out of a bag and this number became the target behavior criterion for earning his reward. The range of possible criterion goals (i.e., the numbers placed in the bag to draw from) were based on Juan's behavior during the baseline week (i.e., first week of treatment) and ranged from one to 30.
The effect of fixed versus random criteria on the DRC was examined using an alternating treatments design in which the two treatment conditions were presented in an alternating manner. The first week of treatment was used to establish baseline rates of behavior without a DRC in place. The DRC was introduced the second week of treatment, with order of fixed versus random criteria determined by chance. Target behavior criteria procedures were changed weekly rather than daily to maximize Juan's ability to discriminate the treatment conditions, thereby reducing carryover effects, and order of treatment was determined randomly to reduce sequential confounding.
Following recommendations, data were examined three ways. First, data were examined visually to illustrate overall trends in the different treatment conditions. Second, alternation randomization tests were computed. This was done by comparing the observed mean difference between the treatment conditions to mean differences randomly generated from the data. Specifically, a difference score was computed for all 35 possible combinations of weeks (e.g., average of weeks 2, 3 and 4 minus average of weeks 5, 6, 7, and 8; average of weeks 2, 3 and 5 minus average of weeks 4, 6, 7, and 8; etc.), including the actual randomization used in the study (i.e., the observed treatment difference). The proportion of the 34 randomly generated mean differences that exceed the one observed treatment difference are reported. Third, three within-subject standardized mean difference effect sizes were computed: (a) fixed-criteria versus baseline; (b) random-criteria versus baseline; and (c) random criteria versus fixed criteria. Effect sizes were computed by subtracting the two conditions and dividing the result by the standard deviation averaged over all conditions.