Principal Findings
EVENT
This is one of the first GPS-based mHealth studies—if not the first GPS-based mHealth study—targeted at improving HIV care in South Africa, and we found that several key challenges impeded its implementation. This study was designed to test the feasibility and acceptability of the CareConekta app and the initial efficacy of using it as an intervention to improve engagement in care among mobile women living with HIV. Although we were able to accomplish our primary aim of assessing the feasibility and acceptability of the intervention, we were unable to assess the efficacy of the intervention because we did not receive consistent location-tracking data. It is important to note that the app missed picking up on travel that was reported at follow-up. Although this is disappointing, we feel that the lessons learned from the implementation of this ambitious mHealth study are important and will be useful to other researchers considering mHealth interventions for low-resource settings. In designing this study, we made a conscious choice to assess our app under real-world conditions. We briefly considered providing phones—particularly to avoid bias against those who did not own phones and to guarantee a consistent technical level of device—but decided that this would not allow us to interpret the real-world applicability of our results. Similar decisions were made against providing data to all participants. Thus, our results can be viewed as representing implementation in real-world conditions.We developed an initial beta version of the CareConekta app and implemented it in a proof-of-concept trial in 2017. We enrolled 11 participants at the same study site. Among the 11 participants, app installation failed for 7 (64%) individuals. Because the app team was US-based, some requirements of the app did not align with the capabilities of many of the phones in use in South Africa, and we were also unable to offer real-time technological support in the event of installation difficulty. The importance of a local app development team, with available technology support, was one of the key lessons learned from this early work. We were also committed to collaborating with a local development company that works with and knows the mHealth agenda of the South African Department of Health; we wanted to be well poised for broader implementation if our app was successful. In the proof-of-concept trial, we were able to install the app on 4 participants’ phones and deploy the app for 3 months. Of the 4 participants, 1 (25%) lost her phone after approximately 1 month, but the other 3 (75%) produced heartbeats at least weekly, often daily, during the 3-month period. This sufficiently proved the concept for us to proceed with this study.In this study, during the 15 months of intensive data monitoring, no participant had GPS heartbeats every day without interruption, which is a key indicator of study feasibility. Despite specifically designing the app such that mobility data would be stored on the app in the event of interruptions in data, the app did not function correctly and did not provide the missing data. Our attempts to troubleshoot lost GPS signals unexpectedly required substantial staff time; indeed, at least 1 staff member phoned participants every week for a year to ask about missing heartbeats. Most participants did not respond, or if they did, they requested a later callback and then still did not respond. CareConekta was designed to reverse bill, which meant that even if the mobile device had no airtime and no data, the app would still be fully functional. However, through implementation, we found that if a participant had no data at all, the app would not work. That is, for reverse billing to work properly, a small amount of data was first required. This became a major stumbling block for implementation, as the top reason cited by participants for missing GPS heartbeats was a lack of data. In addition, we received numerous reports of purchasing only WhatsApp data, a product that was unfamiliar to the researchers at the time of study design but appears to have grown in popularity during the course of our study. Although the lack of electricity was not mentioned as a reason for lost GPS heartbeats, the study period coincided with regular periods of load shedding—scheduled electricity blackouts to conserve power in South Africa—which would have impacted participants’ ability to keep their phones charged. Future mHealth studies will be wise to consider the high likelihood of the lack of data and electricity during study implementation.Even as early as installation, difficulties arose in using the app. From our preliminary research, we knew that over 90% of our participant population used Android-based phones [Overall, we found that the participant acceptability of CareConekta was high, but we view this finding with caution because it does not align with the numbers of participants who reported losing phones and uninstalling the app or the high frequency of lost GPS heartbeats. It is possible that social desirability bias to report a positive experience to the interviewer influenced responses. In the follow-up responses, all participants understood the participant-facing function of the app—the clinic finder—but seemed to forget or not understand the passive geolocation tracking, despite the great efforts made to be explicit about location tracking at the time of informed consent. Future mHealth intervention developers should note that patient-facing features may be the ones that will be most understood and remembered among participants.The proportion of phone sharing reported at follow-up (15%) is consistent with that reported at the time of enrollment (14%). This frequency of sharing is similar to another recent mHealth study in South Africa that found 11% phone sharing [Ours is not the first smartphone-based study in South Africa to experience substantial feasibility challenges. A South African study conducted from 2015 to 2017 [To our knowledge, this is the first mHealth intervention to use location-based participant tracking in a real-world setting in South Africa. Interventions such as this are becoming increasingly common in the United States and Europe. For example, in a study focused on travel health published in 2016, a total of 101 Swiss adult travel clients planning to travel to Thailand for <5 weeks were provided a smartphone equipped with an app to passively monitor their location and administer a daily questionnaire [Some of the problems experienced with app implementation may have been avoided through clearer communication between the research team and app development or technical team. Although we thought that we had very close communication, some aspects were lost in translation. It is critically important that the research team investigating any mobile app includes someone who fully understands the technical specifications and requirements and can liaise with or translate the research vision to the app development team, as well as caution the research team on potentially problematic areas of design.Our study has some clear limitations, primarily that the app did not function as designed because of a combination of user actions and app malfunction. Another limitation is that we conducted the study at a single site, thus potentially limiting the generalizability of our results. However, given that we attempted to mimic real-world conditions, we anticipate that our study population will be similar to other adults attending public health clinics in South Africa.However, we feel that many important lessons learned through this experience will be useful to other researchers and make the effort worthwhile. One strength of our study is that it was among the first to develop, implement, and clearly document experiences with a GPS-based location-tracking mHealth app in a low-resource setting, particularly in a real-world setting. Thus, our findings are meaningful, even if our intervention was not successful. In addition, we report on several technical factors traditionally unreported in the literature but critical to the feasibility of mHealth interventions.