Discussion
RECRUITMENT
In a real-world examination of the role of supportive supervision and monitoring, we did not find significant differences in maternal and child health outcomes between the AC versus the SC conditions. At 8 clinics in deeply rural South Africa, existing CHWs received enhanced training for a month and then were randomized by clinic to receive ongoing monitoring and supervision over time or standard supervision. Statistically, there were no benefits observed.These findings contrast with earlier effectiveness data from 4 studies using a similar model of supervision and monitoring that found a range of significant benefits [In the previous effectiveness studies, the Philani program controlled of all aspects of the intervention (recruitment, training, supervision). Philani has developed implementation strategies refined over decades of experience [In this study, all CHWs had already been hired by the Eastern Cape Department of Health, and most had been in their role for many years preceding the study. Before the study started, CHWs had predominantly been based in clinics and were reassigned at some point from 2011 to 2015 [A second limitation to the implementation was the ability to hold CHWs accountable and to release CHWs not meeting expectations. Because CHWs’ employer was the Department of Health, the Philani supervisors did not have the same leverage to dismiss inactive CHWs. While Philani did deliver negative performance reviews for some CHWs, the power to hold employees accountable rested with the Dept. of Health. It is not clear that it is easy to implement such accountability systems within large Depts. of Health—even though such systems are basic to sound implementation [Surprisingly, mothers in the deeply rural areas of Zithulele had far fewer risk behaviours than was the case in studies conducted in the Philani urban settings [Many of our outcomes were based on self-reports, a potential source of bias by not having independent confirmation by clinic records or rapid diagnostic tests. Alcohol use, for example, may be potentially stigmatizing. However, we had rates of 25% in other rural areas [As outlined in our clinical trials registration [To consider some alternative methods, we ran post hoc Monte Carlo simulations to see how significance may have been determined in alternative methods. If all outcomes have uniform 0.100 correlation and there are no intervention effects, then the probability we see 11 or more New, alternative methodologies that account for the differences in outcomes (like breastfeeding and depressive-like symptoms) by quantifying uncertainty may be more appropriate for multi-outcome studies like this. Such analytic approaches are going to be needed to evaluate CHWs interventions that target multiple outcomes over time.This study was pragmatic—recognizing that it is rarely possible to recruit CHWs from scratch—and seeing whether supportive, in-the-field supervision could demonstrate benefits to maternal and child health outcomes when working with already-employed government CHWs. We were fortunate that the trial had high internal and external validity in its implementation—all clinics in the district agreed to participate, as did the CHWs at each clinic, and recruitment and retention were high among the mothers. We were not able to definitively show that providing good quality supervision had a positive impact on the anticipated maternal and child health outcomes. Nevertheless, the investment of 2 experienced supervisors and a car and driver for every 20 CHWs created a support structure that ensured that CHWs were performing regular household visits—in short, doing the work for which they were paid for. Without this support structure, it seems like CHWs in rural areas do not perform regular home visits and may, therefore, not to be worth the investment. In addition, the lack of risk in deeply rural areas was unexpected, making evaluations using multiple indicators of maternal and child outcomes difficult to assess. In the future, both analytic strategies and reconfirming levels and the types of risks experienced will be critical in order to evaluate the efficacy of CHWs.