The participant was a 6-year, 3-month-old boy, with an ASD, pervasive developmental disorder, moderate-to-severe speech disorder, and language delay; his communication level equaled a 10-to 18-month age range of normative development. He was recruited at the clinic where he had received traditional speech therapy for six months without improvement. Per the university IRB-approved protocol, the boy's mother and grandmother were recruited as communication partners to deliver the AAC intervention in addition to his SLP. The baseline phase consisted of introducing the boy to the AAC system on a touchscreen laptop, which was placed on a desk in the intervention environment (home), but without further instruction. Communication partners also self-talked and used the AAC system at set intervals to encourage the boy's usage of it. Baseline lengths of three, five, or seven sessions were randomly assigned among communication partners.
During intervention, communication partners implemented strict instruction and modeling protocols to teach AAC usage to the boy. Each correct touch of an icon and attempt to speak the corresponding word was reinforced with the boy's favorite cookie, music, and verbal praise. Each intervention session was divided into 20 minute segments, starting with the grandmother as partner, followed by the mother, and then SLP. During grandmother-and mother-led sessions, the SLP guided the others' intervention when needed.
The outcome variable was a count of the number of times the participant correctly touched the AAC display and spoke to imitate the computerized speech output (Hill,). Vocabulary evaluation allowed the participant to produce meaningful utterances that were more similar to his natural speech and language development than the pre-recorded computer digitized output (Hill et al,). Sessions were video recorded; interrater reliability Pearson r = 82.
Data analyses utilized only MMTA because there were too few observations for USEM. Data were missing for one of the grandmother's sessions and three SLP sessions. Error covariance structures differed slightly among partners with autoregressive(1) fitting best for grandmother and SLP, but variance components structure for mother. Figure 1 presents observed time series data as solid lines, best fitting MMTA models as dotted lines, and the MMTA formulas to compute Y'. The Pearson correlation was r = .90 between predicted and observed outcomes. In individual and aggregate analyses, communication growth was statistically greater for AAC than Baseline (p<.05).
Random slopes, but not random intercepts, statistically improved fit to the data (p<.05), indicating that shape of trajectories differed among communication partners. Specifically, faster growth occurred with mother-led AAC intervention. Visual inspection suggested that improvement in communication during the grandmother and mother sessions begins at initiation of SLP intervention. Thereafter, curvilinear improvement in communication occurred with all three partners.
Results are consistent with both hypotheses, with two minor exceptions. First, although the AAC intervention was associated with improved communication with each partner, growth appeared to begin only after the SLP intervention was initiated. Second, the fastest growth occurred with the mother-delivered AAC intervention. Thus, AAC technology functioned according to design, family members proved able to deliver AAC treatment, and evidence supported expanding the testing of this AAC language-based system. Although not reported here, these findings have been replicated in a larger, school-based sample (Chen, Hill, Ridenour, Sun, Su, Chen,).
MMTA models replicated the observed data well and provided proof-of-concept information regarding mechanisms of intervention effects. Replicating previous comparisons between statistical techniques, MMTA proved to be viable with few study participants and observations (Ridenour et al.,). For example, these data were much too sparse for USEM or time series analysis.
Documenting treatment efficacy and effectiveness in the field of AAC is challenging. One central barrier is showing that gains in speech and language skills are due to the treatment rather than maturation. Another substantial barrier is the heterogeneity of the population needing AAC interventions, thus making accumulation of small sample or N-of-1 studies useful to advancing the evidentiary base to guide clinical practice.
Compared to traditional visual inspection methods, MMTA models provide rigorous evidence to answer a question often posed by third-party payers, "are the results due to treatment or merely the child's maturation?" Data clearly show that AAC clinical services are warranted, especially in light of the lack of clinical progress over the six months preceding the study. Clinical services to train family members add significant value to payment for AAC treatment, since time spent on training family members helped to achieve the targeted communication outcomes without requiring the SLP to conduct all the sessions needed for progress (saving costs and increasing dosage). Finally, the child had unique co-morbidities related to his speech-language disorder that demonstrated the challenge of using RCTs for investigating AAC treatment.
Using a wheelchair for extended periods can lead to pressure sores, muscle spasms, altered blood pressure and flow, joint problems, muscle contractures, and painful discomfort. The team at the Human Engineering Research Laboratories has developed a series of devices to assist people with physical disabilities (Cooper et al.,; Cooper et al.,; Ding, Cooper, Pasquina, & Fici-Pasquina,). The Power Seat (PS) was designed to relieve discomfort due to sitting in a wheelchair in one position for extended periods by allowing users to adjust wheel-chair positioning (Dicianno, Mahajan, Guirand, Cooper,; Ding et al.,; Lacoste, Weiss-Lambrou, Allard, & Dansereau,). Positions range from traditional 90 degree angles to nearly supine using adjustments to the footrest, seat bottom, and seatback. However, during a pilot study, PS users largely failed to comply with prescribed PS usage but rather relied on infrequent and small angle adjustments to their seating position; they also continued to complain of pain and discomfort (Ding, et al.,; Lacoste, et al.,).
Two contributors to poor adherence were hypothesized: (1) confusion regarding PS usage and (2) neglecting to use the PS due to forgetting, failing to self-monitor discomfort levels, or low 'buy in.' To improve adherence, an extended education/assistance program was devised to improve understanding of PS functioning (termed Instruction). A second intervention, termed Virtual Coach, consisted of computer-delivered reminders to mindfully monitor physical discomfort level at the proscribed intervals and alter PS angles for relief (Ding et al.,; Liu et al.,). Compared to Baseline, both the Instruction and Virtual Coach interventions were hypothesized to be associated with (a) greater compliance with proscribed PS usage, (b) reduced discomfort, and (c) increased frequency of PS usage and duration of large angle (>15 ) positions (the theorized mechanisms of discomfort relief).
Consistent with the IRB-approved protocol, participants were recruited from the Pittsburgh region at an assistive technology clinic and a Veterans Administration wheelchair seating clinic by their clinicians. Interested clients were introduced to a study investigator to provide additional information, answer questions, and administer informed consent. Inclusion criteria were: 18 years of age or older; use of a medically-necessary, electronic powered wheelchair; the client's sitting surface could be examined daily for redness or pressure ulcers either by the client or another individual; no active pelvic, gluteal or thigh wounds nor a pressure ulcer in these regions within the past 30 days; and no more than 5 days of hospitalization in the previous month. The resultant sample (N=16) was 43.8% female; had a mean age of 51.5 years (SD=12.4 years); was 25% African-American and 75% Caucasian; weighed a mean 196 lbs (SD=43 lbs); was a mean 5'7" tall (SD=3"); 62.5% and 56.3% used a computer or smart phone, respectively; and had been using a wheelchair for a mean 22 years (SD=15 years).
PS usage (adjustment frequency, large changes in PS angles) was recorded by the PS computer. At the end of each study day, discomfort levels were measured using the Tool for Assessing Wheelchair disComfort (TAWC) (Crane, Holm, Hobson, Cooper, Reed, & Stadelmeier,). The TAWC queries General Discomfort using 13 broad summary statements regarding how a client feels while sitting in his/her wheelchair (e.g., I feel poorly positioned, I feel uncomfortable, I feel good) on a 7-point Likert scale. It also queries Discomfort Intensity at that moment using a 10-point Likert scale for 7 specific body parts and additional body parts that could be added by the respondent. The TAWC has adequate test-retest reliability, internal consistency, and concurrent validity (Crane et al.,).
A multiple baseline design was used (14-or 18-day baseline phases). At the start of baseline phases, an introduction and demonstration of PS use was provided to participants. At onset of intervention, participants were randomized to receive either Instruction (n=10) or Instruction plus Virtual Coach (n=6). Intervention phases lasted 50 days. Autoregressive lag(1) was the best fitting error covariance structure for all participants.
Three competing models were tested (Figure 2). The Autocorrelation Only Model implied that discomfort and PS usage could shift in mean level among study phases and that no variable affected any of the others from day-to-day. The Generic Model, based on Zheng et al., implied that in addition to autocorrelation, the level of each variable was associated with changes to every other variable on the next day. The Cooper & Liu Next-day Model, based on hypothesized effects of the Instruction and Virtual Coach interventions, implied that levels of PS usage changed day-to-day in response to discomfort levels. After identifying the best fitting model, its parameter values were tested for moderation by study phase (Baseline vs. Instruction vs. Virtual Coach).
As mentioned earlier in Unified Structural Equations Modeling, these competing models had to be compared while accounting for clustering of data within study phases and individuals. The subset of data from each individual's two phases were analyzed as if they were collected from a separate sample. In other words, two USEM models were estimated per individual (one for his/her baseline data and a second for the intervention phase data) and the full sample was analyzed as if model estimates were aggregated from 32 subsamples. To compare fit among the Autocorrelation, Zheng, and Cooper & Liu competing models, corresponding parameters from the 32 subsamples of data were fixed to be equal. Then, once the best fitting of these three models was determined, moderation of that model by study phase was tested by freeing the corresponding parameters to be specific to each study phase (Baseline parameters versus Instruction parameters versus Virtual Coach parameters).
Study data consist of 1,067 observations. During Baseline, only 2.8% of observations were missing; 7.0% were missing during intervention phases. Visual inspection of compliance rates for the study duration (Figure 3) suggests equivalent rates of compliance during Baseline phases of the Instruction and Virtual Coach subgroups whereas Instruction phase compliance rates appear to be less than the Virtual Coach phase. However, the large within-person variability obscures visually pinpointing mean levels, trends, size of differences among phases, whether such differences are statistically significant, and any effect that autocorrelation has on data.
The best fitting MMTA model for predicting compliance was 24.54 + 0.001(per study day)2 + 1.65(hours of wheelchair occupancy) + 36.18 (if got Virtual Coach) - 0.77 (if got Instruction) - 0.02 (per Instruction phase day). Thus, compliance rates differed considerably between Instruction and Virtual Coach. Also tested, but failed to reach p>.05, were (a) change in compliance over time during Virtual Coach and (b) whether subgroups differed during Baseline.
The model's predicted compliance rates correlated with observed compliance rates r = 0.598 (p<.001). These results suggest that after controlling for time (e.g., due to practice) and how long an individual sat in a wheelchair per day, Virtual Coach more than doubled compliance rates (60.72% vs 24.54%) on average compared to baseline whereas compliance lessened slightly per day of Instruction.
Table 2 presents MMTA efficacy estimates. Compared to Baseline, Virtual Coach was associated with increased frequency of PS use (2.1 vs 3.3) and greater large-angle PS use. In contrast, during the Instruction only phase, PS usage was less than Baseline. In terms of efficacy, general discomfort was statistically equivalent among phases whereas discomfort intensity was statistically less during Virtual Coach - by more than an entire SD (Cohen's d = -1.10). Estimates were consistent with MMTA compliance results.
Table 3 presents fit statistics comparing the competing USEM models. Results consistently suggested the Next-day model best fits the data for both PS adjustment frequency and large-angle PS usage. Freeing parameters to differ among phases further improved fit to the data only for PS frequency.
Table 4 presents the path coefficients per study phase. Day 1 covariance between frequency of PS use with discomfort intensity was null during Baseline (.09), about the same during Instruction (.11), but greater during Virtual Coach (.51). Other associations between discomfort and PS use frequency were also moderated among study phases after controlling for the first day associations. Similar to the Virtual Coach's greater coupling between PS use frequency and discomfort intensity for Day 1, this coupling was even greater from Day 1 to Day 2 during Virtual Coach (.90) compared to the other study phases (-.60 and -.24). Also consistent with Cooper and Liu's hypotheses, the strong association between general discomfort on Day 1 with PS usage the next day (.74) was weakened and reversed during Virtual Coach (-.36), but not during Instruction (.58).
Compared to Baseline, Virtual Coach was associated 2004; Smith,). with (a) improved compliance, (b) large reduction in discomfort intensity, and (c) greater coupling of PS use with discomfort intensity. During Instruction, PS use was more correlated with general discomfort, but not changes in PS usage (PS usage was less than Baseline) or discomfort intensity.
Thus, PS appears to relieve wheelchair discomfort intensity using Virtual Coach at least in part because of improved adherence. This study did not include a Virtual Coach intervention without Instruction. Based on the poor results associated with Instruction, it is reasonable to credit Virtual Coach with the outcomes. Yet, without Instruction, Virtual Coach efficacy may not have been as large (e.g., Instruction may prevent confusion or reinforce Virtual Coach).
Many barriers preclude using RCTs to research treatment for wheelchair users. The population is small and heterogeneous. Also, transportation complications and health conditions impede their research participation. Funding for RCTs is lacking. Moreover, clinicians in rehabilitation and assistive technology are trained to care and value individual's needs and outcomes (as opposed to population averages). Compared to RCTs, ICTs are more compatible with clinical milieu, interfere less with patient "flow," and offer evidence with more direct clinical interpretation and application (Graham, Karmarkar, & Ottenbacher,).
Results from MMTA and USEM provide more sophisticated information about interactions among study variables, their sequencing, and greater rigor than traditional data analysis methods for ICTs (Gabler et al.,; Smith,). Sensor and mobile computing technologies have become more reliable, user-friendly, and widely applied for repeated measurements of real world phenomena. Coupling ICTs with sensor and mobile computing technology to collect data and MMTA and/or USEM represent a qualitative advance in researching rehabilitation and assistive technology (Furberg et al., 2017).