Discussion
REGRESSION, REGRESSION
Understanding fidelity of implementation throughout program introduction is critical in assessing which components of group care may require additional training and support, allowing implementors direction for course correction. Utilizing monitoring results to provide focused mentorship can potentially improve fidelity and strengthen the internal validity of the study. In this case, we see that the trial was implemented with sufficient fidelity and study results can be attributed to the intervention itself, as opposed to implementation failure.Our study findings are aligned with the concepts posited in Carroll’s framework for implementation fidelity [Group ANC implementation fidelity in low and upper middle income countries has only been measured in four other studies, in Malawi and Tanzania, Rwanda, Nepal, and Mexico [This study found that group ANC was delivered overall with high fidelity (mean overall MFA score of 3.18 (±.52)). Regression results found a significant positive association between elapsed time since implementation and fidelity for all MFA items. These findings suggest that the “soft skills” required for group care facilitation can be learned, retained and even improved over time, at least with monitoring and support visits in place. The Nepal and Mexico studies also found the need for continuous monitoring and feedback to improve facilitation skills. The Nepal study initially observed highly didactic facilitation. Researchers noted many nurse-midwives had decades of experience providing one-on-one ANC and were not accustomed to taking into account the social context or patients’ beliefs. In addition to the initial two-day training, routine post-session debriefings including real-time feedback from nurse supervisors were required to improve facilitation and ensure success of the group ANC process.Similarly, the Mexico studies had to adapt provider training by adding additional one‐on‐one, on‐site time to focus on developing facilitative leadership skills. The vertical nature of doctor-patient relationships proved further challenging, with the study finding the item with the lowest level of fidelity to be, “Whether the facilitator introduced her/himself in a friendly non-hierarchical way and guided but did not control the conversation.” The study recommended additional training to achieve the participatory approach required for group ANC.The MFA items with the lowest average scores at initial observation, room setup, curriculum knowledge and kept time, showed the highest improvements over time in the regression analysis. As this trial was a group ANC pilot, we were not surprised by the initial low scores and subsequent room for these items. The MFA items for correct assessment, promoted discussion and group participation, had the highest average scores at initial observation and the least improvements over time in the regression analysis. High initial scores for clinical assessments were expected, due to providers’ prior clinical experience. However, high initial scores for facilitation and participation were unexpected due to the innovative nature of group ANC and departure from traditional patient provider interactions. These results seem to indicate that where providers scored lowest initially, they were able to improve their scores and master the necessary skills with practice, while they maintained the skills they performed well on at the onset. We suggest initially high-scoring items may require less frequent mentoring while initially low-scoring items may require more frequent monitoring and mentoring.Certain items displayed variations in trends over time. The trend line for the average score of keeping time of group ANC sessions showed a steady increase through the first four monitoring and mentoring visits, followed by a steady decrease. This may be due to the fact that over the course of program implementation, the actual number of group ANC visits surpassed the initially planned number of visits, resulting in several logistical challenges for health centers. These results are in line with PTBi-Rwanda’s previous study, which found that at least 25% of group care visits were not implemented with fidelity to the intended two hour session, lasting more than two hours [A unique component of the Rwanda model of group ANC was co-facilitation by CHWs and providers. The MFA and fidelity tools analyzed in the previous PTBi–Rwanda study included measures of fidelity scores as well as agreement in scores across the two different tools. This study assessed qualitative measures of the group process as well as the effect of time on model fidelity, highlighting the importance of facilitation support and skills-building throughout the implementation process. We recommend further exploration into the CHW-provider interaction as well as other factors potentially impacting facilitation, such as provider age and experience, and potential gender norms and interactions between male facilitators and female participants.Based on large improvements in scores for MFA items with low scores at initial observation, we recommend future group ANC programs include intensive monitoring and mentoring at program onset, followed by periodic monitoring through the remainder of the program. It is important to note, however, that certain components of the group process may require greater support than others, such as, preparing the room for group ANC, completing the group session within one hour, and strengthening facilitator knowledge of group ANC content. We recommend the use of both quantitative and qualitative monitoring tools to provide a complete assessment of implementation fidelity. While the MFA has several strengths, solely focusing on quantitative results can fail to capture critical elements affecting implementation fidelity.Carroll’s framework identifies participant responsiveness as a potential moderator, with a lack of participant acceptance and/or engagement potentially impacting implementation fidelity. We recommend implementation fidelity monitoring take place at every level of the program, in the context of group ANC, this includes the level of the health center, providers and women. This is particularly important in addressing more challenging components of group care, such as the impact of facilitator gender on active participation from women. We suggest program implementers consider contextually appropriate mechanisms for measuring factors influencing fidelity at the participant level, such as an exit interview or complaints and feedback hotline. In ensuring the involvement of women, these mechanisms serve as a participatory approach to monitoring implementation fidelity while informing recommendations for improving fidelity of the group ANC process.