Overall, most patients indicated that they perceived the questions in the pain complexity tool and diaries as less applicable in their situation. As several patients had co-morbidities besides CMP, it was difficult for them to know how to interpret the questions. For some questions, it was unclear for them whether the answers should be given with the perspective of having CMP, or from the perspective as a person having pain and other co-morbidities. For example, it was not always possible to indicate exactly their own pain complaints or to adjust answers to questions properly when their situation changed. Sometimes, the eCoach-Pain gave more insight into patients' complexity and impact of their own complaints, which was perceived as heavy to encounter for some patients. Patients without difficulties in daily social participation or psychosomatic problems perceived answer options as less applicable in their specific situation. However, they understood that general questions were formulated for all different kinds of CMP.

The pain complexity tool was the most important tool for GPs in the eCoach-Pain. GP1 indicated that he used it to objectify referral and to get more insight into the complexity of the pain problem. However, GP1 indicated that the digital version directed the referral more than the paper-version. The eCoach-Pain automatically calculates the score and assigns the best-fitting referral option, while, with the paper-version, this can easily be overruled when necessary, according to the opinion of the GP.

*GP1: "When using the paper-version, you have more freedom in the choice of the treatment. As you can overrule the score of the patient easier. In the eCoach-Pain, the treatment options are more limited based on the answers of the patients. Which is a strength of the eCoach-Pain."*

The two PTs used the pain complexity tools in combination with the diaries. For GP1, the graphical display of the results was especially of added value as it gave insight into the effect of the treatment. As improvement, all PHCPs indicated that the graphical displays of the diaries could be upgraded, as it was not always immediately clear for them if the patient's score was positive or negative.

#### 3.3.3. Educational Sessions

The educational sessions were perceived as interesting with clarifying quiz questions and links to YouTube videos. The sessions about 'What is pain' and 'Pain and being active' were perceived as the most useful sessions. The sessions about work and work disability were not appropriate for every patient as some were retired or did not have a job. Two patients indicated that they desired more subjects and educational sessions, for example, about general health.

*R10: "It was a revelation for me, because through the information in the educational sessions, in addition with information on the same topic given by my PHCP, I understand how my brain controls the pain".*

The usability and comprehensibility of the educational sessions were perceived as good as the language used was easy to understand. However, three patients found the language level even too easy and the repetition of subjects in the text as too much. One patient indicated that it was more useful for her when the sessions were not divided over several days, but that all sessions can be followed at once.

Overall, five of the eleven patients indicated that they did not receive new information in the educational sessions in comparison with what they already knew about pain (out of earlier treatments). Some other patients indicated that they perceived recognition and acceptance of their CMP during the sessions due to explanations about the pathophysiology of pain. One patient indicated the sessions as confronting as she/he recognized her/himself for the first time as a patient with chronic pain.

*R13: "I have read all sessions and the total overview was good for me. But at the same time it was also confronting, maybe that was good, as well."*

The educational sessions were most often used by the PTs, and sometimes by the PNMH. PT1 used it to guide the content of his treatment, and PT2 and both PNMHs used it as additional education material for the patient. They indicated that patients were satisfied with the content of the educational sessions and that it gave them more insight into their pain problem. However, they perceived the educational sessions as less applicable for patients with a lower IQ-score or restricted literacy.

#### 3.3.4. Chat Function and Communication with PHCP

Two patients used the chat function, while nine patients indicated they did not. Those two patients were positive about its usability.

Four patients indicated that they had contact with their PT about the diaries and educational sessions they performed in the eCoach-Pain and rated these of added value. For at least one patient, the physiotherapy treatment was adjusted based on the results

in the eCoach-Pain. Moreover, some patients discussed the diary questions about their psychosocial status. Furthermore, patients indicated that the pain education received by their PT fitted well with the information in the educational sessions. The eCoach-Pain resulted in a better patient-PT treatment relationship. Three patients had questions about the eCoach-Pain and needed extra support from their PHCP, for example, about the content of the eCoach-Pain, when to finish using the eCoach-Pain, or extra practical tips regarding their pain complaints. Moreover, some patients mentioned that they had discussed technical issues with their PHCP, such as logging in, bugs in the sessions, or difficulties with data exchange between the PHCP and patient.

The other seven patients mentioned no contact with their PHCP about their activities in the eCoach-Pain. Reasons for this ranged from patients' holidays and sick leave periods, technical issues which limited eCoach-Pain-use, or the fact that the patient had not filled in the pain complexity tools and diaries before the next contact with the PHCP could take place.

Patients did not bother with the fact that their PHCP was able to track their activity in the eCoach-Pain, while some of them did not know this option before the interview. Two patients mentioned that they felt no need to discuss their activity in the eCoach-Pain with their PHCP. Most patients indicated that the possibility to discuss their activity online with the eCoach-Pain was of added value, especially in the situation of COVID-19 they were in during the pilot period, as live contact with PHCPs was only limited to emergency consultations.

PHCPs indicated that they did not use the chat function of the eCoach-Pain often as they preferred other ways to communicate with the patient, such as email, chat functions of other applications, or a real-life contact. Furthermore, the fact that they had to log in again to answer these messages was another reason not to use the chat function. GPs mentioned that they did not always communicate with the patient about the results of the eCoach-Pain themselves, but, instead, they asked the PT or PNMH to respond to the patient.

#### *GP1: "Because of our work-flow, it is the easiest way that the PT communicates with the patient and has a prominent role in the follow-up."*

They checked if a patient scored a red flag, and only then did they contact the patient or the PT. PT1 discussed the results during nearly each treatment session, while PT2 and the PNMH used a less frequent basis or when the patient had questions about it.

#### 3.3.5. Technical Issues

Six patients did not report any technical issues using the eCoach-Pain. Others mentioned problems in finding how to use all functions of the eCoach-Pain, bugs in sessions, or difficulties connecting their eCoach-Pain to the PHCP's profile. Two patients perceived difficulties with logging in into the eCoach-Pain because they had to renew their password more than once or had to log in several times in a row. Two patients had help from family or friends with logging in, use of a computer, or receiving reminders. There were no problems mentioned with the instruction manual, and nobody contacted the Helpdesk of the software developer during the pilot period. Four patients registered themselves with the self-subscription option via a website without any problems. The others were registered by their PHCP; in one case, the connection between the application of the patient and the application of the PHCP failed.

Overall, PHCPs indicated that the eCoach-Pain is easy to use. However, all PHCPs reported having difficulties with the two-way factor identification for logging-in, which is obligated by the General Data Protection Regulation (GDPR). They perceived a delay in receiving the codes by email or the email is marked as spam. The fact that there is an extra step for logging-in hindered them in using the eCoach-Pain more often. They also indicated that it is difficult for them to combine the eCoach-Pain with other existing applications in daily practice, as each application has its own login system, function, and layout.

#### 3.3.6. Future Usage and Recommendations of the eCoach-Pain

Most patients were satisfied with the eCoach-Pain. Some patients indicated that the eCoach-Pain supports to increase insight in how pain impacts daily activities and participation and that it answers questions about their pain complaints. Moreover, they recommend it for the use of the chat function with their PHCP. Some patients would recommend the eCoach-Pain because they were satisfied with it themselves. Most of them would recommend it to patients with other complaints than their own, as they indicated that the content did not fit perfectly based on their own situation. They would especially recommend it to patients who are recently diagnosed, have problems in daily activities and participation, are low literate, or who want to use an eCoach-Pain frequently.

*R16: "I would recommend it to people who get acquainted with pain complaints, or who have not so much knowledge yet, for them it is useful to get to know more about pain. But for people who have complaints for years, like me, I would not recommend it."*

They would not recommend it to patients with pain complaints for years, elderly who are not familiar with eHealth, or patients who do not want to use the eCoach-Pain frequently. However, some patients who are recently diagnosed would recommend it for patients with chronic complaints.

Most PHCPs indicated that they will keep on using the eCoach-Pain in the future as they find it important to offer the patient something extra besides usual care. However, due to time constraints in daily practice, GPs hope that PNMHs can get a more prominent role in the follow-up of patients and contact other PHCPs about the results in the eCoach-Pain. In this case, the PNMH has to contact the GP when expertise or referral of the patient is needed.

*GP1: "Because of the high work-load in primary care, it would be of added value when someone as a PNMH can get a more prominent role in follow-up of patients. They will also be able to keep track of the eCoach-Pain activities. We as GPs have not enough time to do this properly."*

PTs think they will keep on using the eCoach-Pain in the same way as they did during the feasibility study. However, all PHCPs indicated that the costs of the eCoach-Pain concerning their patient volume are important indicators for future usage. During this feasibility study, these costs were covered by the project budget of NPRL.

#### **4. Discussion**

The current study provides insight into the feasibility of an eCoach-Pain for patients with CMP or a high risk of becoming chronic, and for PHCPs in interdisciplinary primary care. In general, patients and PHCPs had positive experiences using the eCoach Pain. The answers to questions/statements about learnability, usability, desirability, and adherence to the application confirm that the eCoach Pain has sufficient quality for further use. However, some further adjustments for successful implementation and use are needed.

Some patients mentioned that the content of the eCoach-Pain does not fit with their situation, such as multi-morbidities and previous experiences with treatment. An explanation why patients do not find the eCoach-Pain suitable could be that patients with CMP often experience multi-morbidities, such as depression, anxiety disorders, obesity, hypertension, and diabetes [39–42]. It has been shown that these patients with multi-morbidities need a personalized treatment [43,44]. The eCoach-Pain has not enough attention for these multi-morbidities. Some patients mentioned that the eCoach-Pain was more suitable for patients with other complaints than they had. Remarkably, the patients with severe complaints for several years mentioned that the eCoach-Pain was better suitable for patients in a subacute phase or those recently diagnosed. Patients with complaints for several years indicated that the information about CMP in the eCoach-Pain was not new and perceived the education sessions as too basic for them. They indicated that the pain education was given in earlier treatments. However, patients with subacute complaints mentioned that the eCoach-Pain is better suited to patients with a clear diagnosis or, in contrast to the comment

of patient with long-term complaints, patients with a longer duration of complaints. A possible explanation for this finding could be that subacute patients are still searching for an explanation and solution for their complaints and are, therefore, more biomedically oriented and not yet focused on a biopsychosocial treatment. [45]. As accepting of CMP is an ongoing process, it could be that the patients with subacute musculoskeletal pain do not see themselves as patients with CMP [46]. Therefore, further research is needed to discover for which patients group(s) the eCoach-Pain can be used in primary care and, accordingly, how the eCoach-Pain can be aligned for personalized treatment.

The eCoach-Pain is well integrated into the treatment of the PTs. All PHCPs perceived advantages of the use of the eCoach-Pain during physiotherapy treatment. Patients indicated that eCoach-Pain connects to the treatment of the PT. Positive thoughts about blended rehabilitation care for other diseases are also seen in several other studies [47,48]. The integration of an eCoach as blended physiotherapy care for patients with temporomandibular disorders lead to an increase in self-efficacy, support of data collection and personalization of the application in the Netherlands [47]. The review of Orlando et al. (2019) showed an overall positive impact on patient and caregivers' satisfaction and it appears to enhance communication and engagement between healthcare professionals for different kinds of telehealth in rural settings [48]. However, questions about the integration of an eCoach in the treatment, such as duration of the treatment, fit in each consultation, and the frequency of the consultations remain [49]. Tilburg et al. recommend to integrate an eCoach into the total treatment and not to implement it as a separate component to the treatment. Further research needs to design and evaluate the integration of the eCoach-Pain into the treatment to deliver blended care.

eCoaches can stimulate and influence interdisciplinary collaboration in primary care [50]. Based on the findings that PHCPs indicated suboptimal collaboration during treatment, it can be concluded that interdisciplinary collaboration between the PHCPs was a point of attention. Accordingly, it seems that the eCoach-Pain did not contribute to interdisciplinary care. GPs indicated that they preferred to refer the patient automatically to a PT as they had not enough time to contact patients and discuss the treatment plan with the PTs, as purposed in interdisciplinary care. The preference of GPs, due to their lack of time, for treatment of these patients by a PT is in line with another study with eCoaches in primary care in the Netherlands [51]. In this study, the PHCPs perceived advantages of the eCoach-Pain in referring a patient to a PT or adding an extra role of a case-manager (for instance, a PNMH or specialized nurses for mental health) in the future. Previous successful implemented eCoaches used a case-manager as first contact for patients [32,34]. In addition, the Standard of Care for Chronic Pain in the Netherlands advices the use of a case-manager for patients with CMP in primary care [14]. However, the eCoaches in these earlier studies were all implemented in secondary care. Therefore, the role for case-manager in primary care needs to be optimized before implementation. Currently, there is no regular financing of a case-manager for patients with CMP in primary care in the Netherlands yet. However, it is crucial to have a case-manager when focusing on integrated and interdisciplinary primary care to stimulate a common vision and treatment plan [19].

Some patients mentioned technical problems that limited the use of diaries or education sessions, even though they received a reminder. Although the developers could not find an explanation for this, it could have influenced the adherence rates for the diaries and education sessions. Other patients indicated that they received too many reminders for diaries. Therefore, in future use of the eCoach-Pain, attention must be given to the communication between the PHCPs and patients. The PHCPs must discuss in advance the number of diaries and reminders offered, based on the preferences of the patients. Research has shown that shared-decision making for chronic illness with treatments containing more than one session leads to treatment agreement [52]. Therefore, shared-decision making in eCoach-Pain adjustments could lead to increased treatment adherence. Connection to the electronic patient file is another technical problem mentioned. PHCPs, and especially

GPs, experienced barriers in the use of the eCoach-Pain in daily practice which was not connected with their electronic patient file. This caused double registration steps, which was a reason to restrict use of the eCoach-Pain. Therefore, it would be favored to find a possibility to integrate the eCoach-Pain in the electronic patient file (Dutch: Huisarts Informatie Systeem) to avoid extra registration steps [53].

A major strength of this study is the use of qualitative and quantitative data (mixedmethods) alongside objective data on use of the eCoach-Pain from both patients' and PHCPs' perspectives. These data gave a broad overview of the usability of the eCoach-Pain, as well as the experiences. Moreover, the content of the eCoach-Pain was developed together with the PHCPs before the start of the study with a user-centered design [28]. Higgins et al. (2018) recommend user-centered designs and implementation science methods to improve the availability of eHealth tools. [54]. However, the GPs and PNMH rarely used the eCoach-Pain despite their influence in the development process. Reasons for this are the login-facility and lack of time during and after consultations, which are also seen as barriers in the study of Daniëls et al. (2019) in primary care [51].

Some limitations of this study need to be acknowledged. First, the small sample of patients that were available for this study and the limited use of the eCoach-Pain could have introduced selection bias. It could be that patients with, for example, low literacy or co-morbidities, were not asked for participation by the PHCPs with a risk for selection bias. However, despite the small sample, patients differed in demographic characteristics, resulting in sufficient confidence to have studied a representative group of users. Second, not all patients performed all measurements, so the completeness of available data per measurement differed. Six patients did not respond on the evaluation questionnaires, and, for three patients export data of eCoach-Pain-use is missing. This could have led to information bias which could have influenced the data. Third, the sample of primary care practices and PHCPs was small, which could have influenced the results. As for primary care practice 1, all patients are subscribed by a GP or PNMH, and, for primary care practice 2, the PT also subscribed patients, besides the GP and PNMH. Most patients were recruited by primary care practice 2 (*n* = 21), and most of the patients participating in the interviews were also recruited by this practice (9 out of 11). Therefore, limited results were available about the recruitment of GPs in the interviews.

#### **5. Conclusions**

In conclusion, the eCoach-Pain seems to be promising in primary care: the patients, as well as the PHCPs, experienced advantages for treatment of patients with CMP. However, adjustments to the content have to be made for better fit with patient-specific CMP complaints. Moreover, the implementation strategy seems to be an important factor for successful use among PHCPs. This should be improved for successful use in interdisciplinary primary care settings. The involvement of a case-manager for CMP should be further explored when implementing the eCoach-Pain. Thereby, it is important to use usercentered designs and implementation science methods to evaluate adjustments resulting in a successful implementation.

**Author Contributions:** C.L. was involved in data curation, formal analysis, investigation, methodology, project administration, supervision, validation, visualization, and writing—original draft. I.H. was involved in supervision, methodology, validation, and writing—review & editing. M.d.M. was involved in formal analysis, investigation, validation, and writing—review & editing. A.K. was involved in investigation, methodology, supervision, and writing—review & editing. M.K. and J.V. were involved in supervision and writing—review & editing. All authors have read and agreed to the published version of the manuscript.

**Funding:** Health Insurance Companies CZ, VGZ, and Zilveren Kruis Achmea funded the study, but they had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Medical Ethics Committee Z, the Netherlands (METCZ20190037, latest date approved 19/05/2020). Informed consent will be obtained from study participants.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The data are not publicly available due to the small number of healthcare professionals and patients included. This could interfere with the privacy of the participants.

**Conflicts of Interest:** C.L., M.d.M. and M.K. have nothing to disclose. I.H., A.K. and J.V. report grants from Health Insurance Companies CZ, VGZ, and Zilveren Kruis Achmea, during the conduct of the study.

#### **References**

### *Article* **Patient Experience and Satisfaction with an e-Health Care Management Application for Inflammatory Bowel Diseases**

**Aria Zand 1,2,\*, Audrey Nguyen 1, Courtney Reynolds 1, Ariela Khandadash 1, Eric Esrailian <sup>1</sup> and Daniel Hommes 1,2**

**Citation:** Zand, A.; Nguyen, A.; Reynolds, C.; Khandadash, A.; Esrailian, E.; Hommes, D. Patient Experience and Satisfaction with an e-Health Care Management Application for Inflammatory Bowel Diseases. *Int. J. Environ. Res. Public Health* **2021**, *18*, 11747. https:// doi.org/10.3390/ijerph182211747

Academic Editors: Marie Carmen Valenza, Irene Torres-Sanchez and Paul B. Tchounwou

Received: 19 August 2021 Accepted: 2 November 2021 Published: 9 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

**Abstract:** Background: Rising healthcare expenditures have been partially attributed to suboptimal management of inflammatory bowel diseases (IBD). Electronic health interventions may help improve care management for IBD patients, but there is a need to better understand patient perspectives on these emerging technologies. Aims: The primary aim was to evaluate patient satisfaction and experience with the UCLA eIBD mobile application, an integrative care management platform with disease activity monitoring tools and educational modules. The secondary objective was to capture patient feedback on how to improve the mobile application. Methods: We surveyed IBD patients treated at the UCLA Center for Inflammatory Bowel Diseases. The patient experience survey assessed the patients' overall satisfaction with the application, perception of health outcomes after participation in the program, and feedback on educational modules as well as areas for application improvement. Results: 50 patients were included. The responses indicated that the patients were greatly satisfied with the ease of patient–provider communication within the application and appointment scheduling features (68%). A majority of respondents (54%) also reported that program participation resulted in improved perception of disease control and quality of life. Lastly, a majority of participants (79%) would recommend this application to others. Conclusions: Mobile tools such as UCLA eIBD have promising implications for integration into patients' daily lives. This patient satisfaction study suggests the feasibility of using this mobile application by patients and providers. We further showed that UCLA eIBD and its holistic approach led to improved patient experience and satisfaction, which can provide useful recommendations for future electronic health solutions.

**Keywords:** electronic health; mobile health; mobile applications; chronic disease management; patient experience; inflammatory bowel disease

#### **1. Introduction**

Value-based healthcare (VBHC) can be described as the systematic pursuit of the triple aim in healthcare: to improve the individual's experience, improve health outcomes, and reduce costs [1]. The concept of VBHC is particularly ready for application to long-term management of chronic illnesses since rising healthcare expenditures have been partially attributed to suboptimal management of chronic illnesses, including IBD [2]. The estimated annual disease-attributable cost of IBD is \$6.3 billion [3]. Hospitalizations represented over a third of costs, outpatient services—one third. Reducing hospitalization and readmission rates, therefore, continues to be a challenge in chronic disease management. There is clearly an opportunity to reduce costs by increasing the efficiency of outpatient care and preventing hospitalizations.

Electronic health (e-health) interventions are one solution for more effective IBD care management beyond the clinical setting, both in terms of patient outcomes and cost reduction. Smartphone applications are widely available for consumers, and the large population of smartphone users make applications useful tools to manage chronic illnesses like IBD [4]. In fact, smartphone devices with mobile applications and short message reminders have been used effectively by patients with IBD of mild or moderate severity [5].

Furthermore, mobile health technologies have been shown to improve patient outcomes and quality of life [6]. Patient satisfaction with mobile technologies has been observed for many chronic diseases, including asthma [7], HIV [8], diabetes [9], atrial fibrillation [10], and IBD [11]. IBD patients generally have positive views on mobile applications, but there are desirable improvements. A study of Con et al. [11] surveying 86 IBD patients found that 98.8% of the participants were willing to use communication technologies for IBD management, with mobile applications being one of the top two preferred forms. These previous IBD mobile technologies were often created to assess a major single aspect such as quality of life [5], education curriculum [12], or diets [13]. Additional features that patients seek in their chronic disease management applications include easy user interface [14], tracking of disease symptoms [11], and easy access to medical data and services [11].

A systematic assessment of 26 IBD mobile applications found that applications offered a variety of features including diary functionality, pain tracking, bowel movement tracking, and reminders, with application's content playing a major role in driving patient behavior change [4]. The MyIBD Coach telemedicine tool, which monitors adherence, disease activity, quality of life, and mental health among other measures through validated questionnaires, was shown to be successful, with high rates of patient satisfaction and compliance [15]. It involves collaboration among healthcare providers but is not synchronized with electronic medical records and lacks educational application features on alternative medicine, behavioral health, and physical activity.

To enhance VBHC in IBD, we developed UCLA eIBD to integrate various successful features of previous applications (e.g., appointment reminders, medication trackers) in addition to a healthcare provider portal. UCLA eIBD seeks to provide patients more agency in managing their IBD by increasing their access to healthcare professionals and providing self-help educational modules. Access to care providers through a messaging application provides patients with fast feedback on their conditions and streamlines patient care [16]. The application also contains disease activity, quality of life, and work productivity surveys that facilitate interactions between patients and providers. These tools allow healthcare providers to monitor patients' disease activity and give direct feedback. This comprehensive application therefore seeks to enhance patient outcomes by including direct connections to the healthcare team and extensive module options.

We previously conducted a pilot study of UCLA eIBD, which found significantly fewer endoscopies and decreases in healthcare utilization, long-term steroid use and IBD-related costs [17]. While it is important to evaluate the efficacy and outcomes of IBD management platforms, it is just as crucial to understand patients' satisfaction with these platforms to inform their feasibility. Gathering user feedback is necessary to develop the next generation of applications, improve product design, and reduce the gap between application developers and consumers [18–20]. This study therefore aims to provide an evaluation of perceived patient satisfaction and experience with the UCLA eIBD mobile application.

#### **2. Materials and Methods**

#### *2.1. Objectives*

The primary objective was to measure patient satisfaction and experience with the UCLA eIBD mobile application for care management. The secondary objective was to capture patient feedback on how to improve the mobile application.

#### *2.2. Design and Population*

We surveyed IBD patients treated at the UCLA Center for Inflammatory Bowel Diseases from October 2017 to October 2018. Included patients were at least 18 years old; diagnosed with Crohn's disease (CD) or ulcerative colitis (UC) either by means of endoscopy, imaging, or pathology; and had objectively logged into the application in the past year (assessed on the platform). Patients with intestinal cancer, active chemotherapy, or a known intestinal infection were excluded.

All the eligible patients who had logged into the application in the past year were emailed and asked to complete a patient experience survey. Those who did not complete the survey in response to the initial email were followed up and interviewed via phone. No sample size estimation was performed.

#### *2.3. Description of UCLA eIBD*

UCLA eIBD is a mobile application that administers a clinic-centered, care management program to its users (Figure 1). It was designed to be a comprehensive tool for patients' long-term disease management in the IBD outpatient setting. The features of this application include disease activity monitoring, messaging, educational modules, lifestyle modules, and electronic cognitive behavioral therapy (eCBT). The platform is also integrated with UCLA Health's electronic medical records, allowing patients to view their testing and laboratory results within the application.

**Figure 1.** The UCLA eIBD mobile application is an integrative care management platform for patients and providers.

For disease activity monitoring, a previously validated tool called the Mobile Health Index was integrated to assess the patients' disease activity, quality of life, and work productivity [21]. If the surveys indicated poor disease control or a significant change from prior surveys, a message was automatically generated through the application to clinic staff. The enrolled patients could also elect to take these surveys in their own time if they felt they were experiencing a sudden change in their health.

Lastly, the application provided education through several optional interactive modules designed to promote healthy lifestyle habits, including nutrition (My Menu), exercise (My Yoga, My Fitness), relaxation (My Acupressure, My Meditation), and mental health (My Coach). My Menu teaches patients about specific foods to eat and avoid and includes recipes (breakfast, snack, lunch and dinner) designed for IBD patients. My Yoga provides a 6-week program promoting relaxation and flexibility for users. My Acupressure teaches patients about different pressure points for alleviating IBD pain via instructional videos and pictures. My Meditation is a self-guided mindfulness therapy tool that aims to reduce stress-related health issues.  Data Collection and Outcomes*

Patient demographic data were acquired via chart review. The data from the patient experience survey were collected via REDCap [22]. The patient experience survey (Table 1) consisted of 24 items aimed at assessing the patients' overall satisfaction with the application and their perception of health outcomes after participation in the program. Responses were provided either via a Likert scale or open text. Questionnaire items addressing the application's features and interface requested feedback on the ease of application use, ability to communicate with staff, and informativeness of modules. Questionnaire items pertaining to the patient's outcomes asked the patients how effective they felt the application was at improving disease control, work productivity, and quality of life. Lastly, the patients could provide optional open-ended feedback via free text input on ways to improve the application.

#### **Table 1.** Patient experience survey.

**Table 1.** *Cont.*

\* Optional question.

#### **3. Results**

#### *3.1. Patient Demographics*

In total, 151 patients had been active on the mobile application in the past year, of whom 50 patients responded and completed the survey and thus were included in this study. Regarding the type of IBD, 44% were diagnosed with CD (*n* = 22), 56%—with UC (*n* = 28). Our inclusion cohort had a mean age of 43 years (SD, 14 years) and an average BMI of 25.3 (SD, 6.6). Of the patients, 44% were female, and the majority were White (42%) and non-Hispanic (90%) (Table 2). Most of the patients were non-smokers (78%), and 28% of the patients reported alcohol use. The patients stated use of the following medications: anti-TNF (34%), ASA (16%), combination therapy (32%), IMM (10%), and steroids (6%). Previous abdominal surgeries were reported in 36% of the participants.

**Table 2.** Patient demographics.

#### *3.2. Patient Satisfaction*

Fifty participants out of the 151 users responded and completed the patient experience survey to provide feedback on the mobile application (Table 1). Responses to the Likert scale questions indicated that the patients were overall satisfied with the patient–provider communication interface of the application. When asked how easy it was to communicate with the program's staff overall, 52% of the participants responded with "very easy" and 16% responded with "somewhat easy". A majority of the participants also found it easy to schedule appointments through the application, with 52% and 16% responding with "very easy" and "somewhat easy", respectively. In addition, a large majority (88%) of the participants reported that the frequency of completing laboratory tests and surveys and scheduling clinic visits was "just right" (Table 1). Regarding the ease of application use, 74% of the participants indicated the application was either "very easy" or "somewhat easy" to navigate.

Additionally, a majority of the participants reported an improved perception of disease control and QoL; 54% of the participants indicated significant or some improvement in their disease control. When asked how program participation affected QoL, 26% indicated significant improvement, 30%—some improvement. Regarding work productivity, 44% indicated significant or some improvement.

When the participants were asked whether they would recommend this application to their friends, family, or other patients on a ten-point scale, with 10 being most likely,

the median score was 8, and 79% indicated a score greater than 5. When asked about how informative the application was, 46% of the patients felt that the application was "somewhat" or "very" informative.

#### *3.3. Patient Usage of Educational Modules*

A majority of the patients completed modules as part of their participation in the program. The most used modules were My Fitness and My Menu (Table 1). Among the patients who participated in the CBT modules (12%), 28% indicated significant or some improvement in their mental health.

When asked about what they liked and disliked about the modules, the patients identified positive aspects to be the modules' informative content, ease of use, and support of overall well-being (Table 3). For example, one patient said, "They're easy and I feel great afterwards." Another patient expressed liking the modules because they "encourage me to take care of my whole self instead of the focus just being on taking my meds".

**Table 3.** The patients' optional feedback on the modules (*n* = 50). The patients provided open-ended feedback about the educational modules. Their responses were grouped into categories based on the common themes identified across the responses.

The most common reason for not liking the modules was being unsure of the purpose or need for them (8%), particularly for the modules where patients already had their own interventions in place. For example, one patient said they "didn't feel [the modules] applied to me" while another expressed that they "thought [the module] was good but [I have my] own routine for working out [with regards to My Fitness]".

#### *3.4. Patient Feedback*

In the patient experience survey, the patients could provide optional suggestions about additional topics and functionalities they would like the application to cover which were not presently included (Table 4). One participant, for instance, suggested adding a subsection about nutritional advice related to veganism within the My Menu module. Other recommendations included adding a "symptoms tracker", allowing patients to indicate what symptoms or lack thereof they were experiencing and generating in-application reminders for blood draws or laboratory orders. The other patient-recommended categories to add were the ability to chart laboratory results, side effects of their consequent medications, and gender-specific health topics (Table 4).

**Table 4. The patients' optional feedback on UCLA eIBD**. The patients provided open-text suggestions to improve the application in general. These suggestions were grouped into categories of comment types, including improvements in application content such as possible additional topics and features, as well as miscellaneous critiques.

The patients' feedback regarding general comments about the application is also shown in Table 4 (miscellaneous improvement suggestions). One patient stated, "I think this is a great idea and will be very helpful to future patients. I really like being able to communicate with the office without always having to call." Most patients who provided comments also highlighted aspects that could be improved, such as the application's interface (e.g., adding a touch ID option to log in; preventing automatic logoff from the application). Other participants reported critical feedback on the application's content. For instance, one patient stated that the application "is good for people new to IBD, but doesn't offer as much for people who have had IBD for a while and want more in depth information".

#### *3.5. Summary of Principal Findings*

The outcomes suggest that the patients strongly favored the ease of patient–provider communication, with 78% being satisfied. Beneficial outcomes were also seen in patientreported measures, with 54% reporting a perceived improvement in disease control and 56% reporting a perceived improvement in QoL, indicating that a majority of patients felt the platform positively impacted their health. Additionally, the participants rated this application with a median score out of 10 (10 being most likely) to recommend this

application to friends, family, or other patients. My Fitness and My Menu were the two most used optional wellness modules, each reaching the 34% completed status.

#### **4. Discussion**

#### *4.1. Strengths and Comparisons*

Our study collected feedback on patient experiences with the UCLA eIBD application after one year of use. Our results could provide guidance for further application development and provide critical feedback for other e-health applications like this one. In fact, mobile tools such as UCLA eIBD have been shown to have promising implications in improving healthcare delivery and integrating into patients' daily lives. Earlier comparison studies of UCLA eIBD found impacts on costs and healthcare utilization and identified its unique features, such as automated messaging to care coordinators [17,23–25]. To complement the previous outcome studies, this study aimed to understand patients' satisfaction and feedback to help elucidate gaps in the current e-health technologies and inform future designs.

For instance, GI Buddy is a mobile application developed by the Crohn's Colitis Foundation which enables patients to self-monitor their disease and receive reminders about clinical appointments; however, users cannot directly interact with their providers [26]. Similarly, while the current applications for IBD may be useful for patient monitoring and self-management, many lack professional medical involvement and adherence to clinical guidelines [4]. UCLA eIBD addressed this gap by allowing users to make appointments and message their providers via the platform, in which a majority of users found it "easy" or "very easy" to communicate with their providers. Another self-management tool, myIBD Coach, showed feasibility among patients and providers [15]. As many as 79% of UCLA eIBD users would recommend this application to others (indicated by a score of greater than 5 on the recommendation score item), compared to the 93% found in the myIBD Coach feasibility study [15].

The findings of this novel patient satisfaction study demonstrate the feasibility of UCLA eIBD as a home monitoring tool and some advantages it can provide for both patients and providers. In addition to the patient–provider communication features, the platform's educational modules are more diverse than the previous tools and provide patients with more alternatives to aid traditional medicine, such as acupuncture, cognitive behavioral therapy, and meditation. These optional modules may improve IBD patients' well-being and productivity beyond the scope of their disease. For providers, tracking the various modules that the patients use can also provide guidance for tailoring treatment and counseling to the patients' interests, including nutrition, exercise, and mental health. Lastly, the holistic nature of the application, including features about alternative medicine and assessments for work productivity via the Mobile Health Index, can more completely address the complex, multidimensional factors of chronic disease management.

While the integration of mobile health in IBD management is rapidly expanding, our study also presents novel data from the patient perspective and emphasizes a patientcentered approach towards mobile application development. For instance, the patients' suggestions to improve the application were centered on specific content interests and the need for additional educational categories (e.g., female health topics), rather than technical problems or lack of need in an application. The fact that the suggestions were less focused on design features could be explained by the overall satisfaction rate of 74% of the participants finding the application easy to navigate. The current and future applications can thus utilize these methods and/or findings to adapt their platforms to address patients' specific needs, improve satisfaction with their product, and better engage patients in their medical care beyond a doctor's office visit.

#### *4.2. Limitations*

Some study limitations should be noted. As the selected patients were individuals who use smartphones, they may be more adept at the usage of applications. Participants were also actively recruited and agreed to participate in this study; thus, a selection bias may have impacted the study results due to the participants being predisposed to wanting to improve their health via e-health solutions. We further acknowledge the sample size was small and relatively homogenous; however, we feel it was adequate for the purpose of directing the future development of this UCLA application and other healthcare applications.

Additionally, the fact that we invited participants to evaluate the application's feasibility rather than making it mandatory during application usage may explain the response rate of 33%. The response rate should further be considered in the context of challenges associated with adopting e-health technologies into the healthcare space. The obstacles to widespread long-term integration of e-health technologies (e.g., loss of interest, data entry burdens) are still being investigated [27,28]. Despite the growing population of individuals who use mobile health applications, many stop using them over time [29]. Our findings help provide insight to consumer perspectives on application usability and possible explanations to circumvent these challenges.

#### *4.3. Future Directions*

In an era where the use of mobile technology has become irreplaceable in daily life, there is undoubted benefit of incorporating e-health applications in the management of chronic conditions. Studies have shown proven effect of mobile applications, but also that patients still desire improvements to the existing solutions. We showed that UCLA eIBD and its holistic approach led to greater patient experience and satisfaction, which can provide useful recommendations for healthcare providers and application developers. However, larger and controlled studies are recommended to assess its efficacy at a larger scale and its impact on costs.

**Author Contributions:** A.Z.: study concept and design; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical analysis; study supervision; A.N.: study concept and design; acquisition of data; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; C.R.: study concept and design; drafting of the manuscript; critical revision of the manuscript for important intellectual content; A.K.: acquisition of data; administrative, technical, or material support, drafting of the manuscript; critical revision of the manuscript for important intellectual content; E.E.: revision of the manuscript for important intellectual content; D.H.: study concept and design; analysis and interpretation of data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical analysis; study supervision. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** This study was approved by the Institutional Review Board at UCLA with IRB protocol number 17-001208. All the procedures performed in the studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

**Informed Consent Statement:** Informed consent was obtained from all the individuals included in the study.

**Conflicts of Interest:** The authors declare that they have no conflict of interest.

#### **References**

## *Article* **Building a Research Roadmap for Caregiver Innovation: Findings from a Multi-Stakeholder Consultation and Evaluation**

**Kieren J. Egan 1,\*, Kathryn A. McMillan 1,2, Marilyn Lennon 1, Lisa McCann <sup>1</sup> and Roma Maguire <sup>1</sup>**

**Abstract:** Across the world, informal (unpaid) caregiving has become the predominant model for community care: in the UK alone, there are an estimated 6.5 million caregivers supporting family members and friends on a regular basis, saving health and social care services approximately £132 billion per year. Despite our collective reliance on this group (particularly during the COVID-19 pandemic), quality of life for caregivers is often poor and there is an urgent need for disruptive innovations. The aim of this study was to explore what a future roadmap for innovation could look like through a multi-stakeholder consultation and evaluation. An online survey was developed and distributed through convenience sampling, targeting both the informal caregiver and professionals/innovators interested in the caregiver demographic. Data were analysed using both quantitative (summary statistics) and qualitative (inductive thematic analysis) methods in order to develop recommendations for future multi-stakeholder collaboration and meaningful innovation. The survey collected 174 responses from 112 informal caregivers and 62 professionals/innovators. Responses across these stakeholder groups identified that there is currently a missed opportunity to harness the value of the voice of the caregiver demographic. Although time and accessibility issues are considerable barriers to engagement with this stakeholder group, respondents were clear that regular contributions, ideally no more than 20 to 30 min a month could provide a realistic route for input, particularly through online approaches supported by community-based events. In conclusion, the landscape of digital health and wellness is becoming ever more sophisticated, where both industrial and academic innovators could establish new routes to identify, reach, inform, signpost, intervene and support vital and vulnerable groups such as the caregiver demographic. Here, the findings from a consultation with caregivers and professionals interested in informal caring are presented to help design the first stages of a roadmap through identifying priorities and actions that could help accelerate future research and policy that will lead to meaningful and innovative solutions.

**Keywords:** caregivers; innovation; research; co-design; interdisciplinary; digital health; participatory design; collaboration at distance

#### **1. Introduction**

Informal (unpaid) carers, also termed "caregivers", are family members and friends who support a loved one who needs help due to illness, frailty, disability, mental health problems or addiction. In the UK, it is estimated that there are at least 6.5 million informal caregivers, a workforce substantially larger than the National Health Service (NHS) [1] and the collective saving to the health and social care services is estimated to be £132 billion per year. This situation is similar across Europe, and beyond, where 80% of all care is delivered by informal caregivers [2]. As the tide of an ageing global population continues to advance in tandem with a shrinking health and social care workforce [3], there is a public

**Citation:** Egan, K.J.; McMillan, K.A.; Lennon, M.; McCann, L.; Maguire, R. Building a Research Roadmap for Caregiver Innovation: Findings from A Multi-Stakeholder Consultation and Evaluation. *Int. J. Environ. Res. Public Health* **2021**, *18*, 12291. https://doi.org/10.3390/ ijerph182312291

Academic Editors: Irene Torres-Sanchez and Marie Carmen Valenza

Received: 28 October 2021 Accepted: 14 November 2021 Published: 23 November 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

health emergency looming, whereby the pressures on caregivers across the world are set to significantly escalate.

While the recent events during the COVID-19 pandemic have demonstrated the societal value of caring, it also highlighted that many will face short, and long-term, health and wellness consequences. Informal caregivers share many of the same challenges of a professional workforce, such as NHS employees, but lack much of the associated infrastructure. For example, there is little to no: support, training, pre-agreed workload such as hours per week or a working role definition, and caregivers often must balance caring responsibilities around work, families and their own health and wellness needs. While some caregivers do enjoy benefits from caring roles, such as a sense of fulfilment from caring [4], there is now compelling evidence that caregiving adversely impacts on health and wellness, both in the short and long term [5–7]. Crisis points, such as hospitalizations, significant worsening of mental and/or physical health and irreversible changes to caring circumstances are commonplace (even in the absence of COVID-19) and frequently cause deterioration in health for caregivers, and those being cared for [8,9].

Given the considerable number of unmet needs described above, there have been many different innovations developed for the caregiver demographic. Although successful evidence has emerged [10–12], there still remains a striking number of caregivers who remain "out of reach" [8] through a combination of factors, including (but not limited to): digital and health literacy levels, socioeconomic status, health, mobility, and level of dependency required from the person cared for. Thus, there remains a continued need to consult, innovate and evaluate for, and with, caregivers and to develop research excellence in this area. Emerging from the evidence in this field, there are several areas of particular interest, which include: methodologies to improve identification of caregivers (including in hard to reach groups such as ethnic minorities [13], the development and adaption of theoretical/conceptual frameworks, techniques for cultural adaptation [14] and addressing implementation and abandonment issues [15].

Digital health and wellness solutions are a core part of the current drive to deliver equity in health to all—including informal caregivers. In 2020, for example, the WHO (World Health Organization) produced a long-term strategy for the use and scale-up of digital health, highlighting the positive impact digital health can have on the access to, and provision of healthcare as well as the health and wellbeing of the population [16]. Such findings are paralleled in recent review work [17]. Much of this work focuses on caregiver health and wellbeing or signposting to sources of support and are not restricted to one or two types of technology (e.g., mobile app, internet-based, integrate platforms, sensors) [17]. Despite such progress, the market reality is that most startups fail and many successful research studies are challenging to implement. Reducing uncertainties and increasing the availability of market knowledge, such as bettering current understanding of ongoing caregiver needs, would be considerably advantageous.

Taken collectively, innovation for informal caregivers is sorely needed but the landscape still falls short of meeting the everyday needs. Identifying both better ways to collaborate across all stakeholders and methodologies to rapidly, but robustly, develop and test innovations could help improve the translational hit rate. Therefore, here we took a first step to improve engagement with caregivers by undertaking a multi-stakeholder consultation to design a future roadmap for innovation in caregiver research.

#### **2. Materials and Methods**

Initially, this study was due to involve face-to-face interviews and focus groups, however, due to the COVID-19 pandemic we had to be agile and to achieve our aim of engaging with as many caregivers and other stakeholders as we could, we changed methodology and conducted the multi-stakeholder consultation via survey. We developed our own survey approach as we did not find a suitable existing one. Although the same content of surveys was delivered to both caregivers and professional groups (e.g., healthcare professionals, innovators), we tailored wording on occasion to represent each stakeholder group.

The surveys were iteratively designed to collate participant information regarding: (i) demographics, (ii) previous research involvement, (iii) feelings towards research and innovation, (iv) barriers to taking part in research, (v) future participation in research and, (vi) focus/outcomes of future research. No questions were mandatory and, thus, the total number of responses for each question varied.

Ethical approval was granted from the Computer and Information Science (CIS) Department Ethics Committee at the University of Strathclyde, Glasgow. The surveys, aimed at caregivers and those with a professional role interested in caregivers, were designed to take no more than 10 min to complete, and to be used across the UK. See Appendix A for example questions.

Inclusion criteria and survey design: Participants 18 years and over were invited to take part. A broad definition of an informal (unpaid) caregiver was shared with participants, "*People that provide unpaid care by looking after an ill, older or disabled family member, friend or partner*": hereafter referred to as caregivers. A minimum number of hours per week that caregivers needed to be caring for was not specified, leaving it up to participants to self- identify with the term. For professional groups, our participant information sheet stated that we were interested to find "*Professionals (not employed by Universities) who have an interest in innovating around carer health and wellbeing (e.g., health and social care professionals or those working in the technology industry/third sector)*". Where we state professionals, hereafter, this implies professionals/innovators interested in the caregiver demographic.

Survey distribution: Consent was implied after participants read, acknowledged, and accepted the initial terms of the anonymised survey. The distribution of the survey involved sharing the online version of our questionnaire through social media channels (e.g., Twitter), alongside email distribution through networks accessible to partners, such as the Digital Health & Care Innovation Centre (DHI) and Carers UK (e.g., Healthy Ageing Innovation Group and the "Digital Health & Care" Mailing list). The survey was open from 15 June 2020 until 30 September 2020.

Data handling: The survey was constructed using Qualtrics Software. Participants were reminded that any data entered must not contain any identifiable information. An integrated mixed-methods approach was taken for the data analysis. All quantitative analyses (frequencies and summary statistics) were completed using R studio (version 1.1.456). Qualitative analyses were undertaken using a content analysis approach by two researchers (KM and KE) [18]. Free text responses to questions were collated and the first question coded by KM. The coding structure cross-checked and agreed by KE as a measure of inter-rater reliability. Once agreed, the rest of the coding was completed by KM, before being cross-checked by KE. Data were frequently referred back to throughout this process to ensure the coding framework developed was appropriate. The flexible nature of content analysis tolerates the use of a combination of deductive and inductive creation of themes and patterns from the data. Deductive analysis was used to create the key themes and inductive analysis was used to create sub-themes. More specifically, the six key themes were identified from the survey structure (deductive analysis), sub-themes were identified through researcher (KM and KE) interpretation and coding of free text responses (inductive analysis). Data gathered from informal caregivers and professionals were analysed, and are presented, separately in the results section of this paper. Major themes were identified where at least half of respondents aligned with a specific finding. Where respondents identified as both a caregiver and a professional/innovator interested in caregiving, we classed these individuals as part of the caregiver demographic.

Data analysis: This work was developed without an a priori hypothesis but instead was conducted as an observational/explorative piece used to generate a future roadmap/hypothesis. Therefore, our statistical analysis is limited to frequencies, presented as stacked bars throughout the manuscript (we interpreted priorities where >50% of the respondents suggested an item was important/very important or equivalent). Qualitative analysis included the identification and discussion of key themes, such as clear priorities outlined by participants, throughout the narrative. For the purposes of readability and figure simplicity, long statements have been abbreviated and a full list of statements presented to participants can be found in Appendix B.

#### **3. Results**

#### *3.1. Description of Our Sample*

#### 3.1.1. Caregiver Demographics

The survey received 112 responses from informal caregivers (see Table 1). Caregiver respondents were aged 25 to up to 75 to 84 years old, with the mode age within the 45 to 54 years category. Over 85% (96/112) of caregiver respondents identified as a woman/female. Caregivers were highly educated, 60.7% were educated to degree level or equivalent, and 111/112 identified as white (Table 1). Just over half (50.9%, 57/112) of caregivers had been caring for 10 or more years and 6% (7/112) had been caring for 1 year or less. Caregivers varied in their ability to undertake work/study alongside their caring role: 35% were able to continue to work/study, 34% had been forced to reduce their work/study hours and 19.6% had been forced to give up work/study. Almost all (96%) of caregiver respondents were using digital technologies, such as smartphones and laptops on a daily basis. The vast majority of caregiver responses were from Scotland (96%, 108/112), with 1 response from Northern Ireland (<1%) and 3% (3/112 responses) from England.

**Table 1.** Participant Demographics. \* n = 61 for professionals' gender question.

#### 3.1.2. Professionals Demographics

The survey received 62 responses from professionals (Table 1). Professional respondents were aged from 25 to up to 55 to 64 years, most identified as women/female (79%) and 94% of respondents were white. The professional respondents were highly educated, 74% had a degree or equivalent level of education. When asked what their professional role was, 61 of the 62 participants responded. Most respondents (36%, 22/61) worked in healthcare, followed by 25% (15/61) working in social care, 23% (14/61) working in the 3rd sector, 11% of respondents put other, 3% worked in caregiver policy development and one respondent worked in a Small to Medium Enterprise (SME). All participants reported everyday use of technology. Similar to caregivers, 98% (60 /61) of professionals were from Scotland, with 1 response from England (2%).

#### *3.2. Views of Informal Caregivers*

Three key themes were established from the structure of our questioning and eight sub-themes were identified from the thematic analysis of the professionals free-text survey data. The three pre-specified (deductive) themes were; (1) Previous research participation, (2) Future research participation and (3) Future research aspirations. Sub-themes (identified through inductive thematic analysis) are presented in Table 2.

#### 3.2.1. Previous Caregiver Research Participation

Most respondents (79%, 89/112) had not previously participated in research being conducted by a university in relation to their role as a caregiver, while 18% (20/112) of participants had previously taken part in research. For those who had previously taken part in research, the majority had completed surveys: there was no reference by any of the caregivers to taking part in other forms of research, such as focus groups or interviews. Most had taken part previously due to a professional interest in research: for example, it being part of their job, or from a personal interest whereby participating in the research enabled access to specific support or services. The main reasons for not participating in research previously was lack of awareness, being unaware of any ongoing research or not being asked to participate in research prior to the current project. The other barrier towards participating in research was the lack of free time caregivers had allowing them to commit, given the other time pressures in their life. Time was, by far (both in Likert and free text responses) the most significant barrier to taking part in research, followed by money and personal health (See Figure 1).

**Figure 1.** Caregiver responses to a range of statements according to whether they "Strongly agree", "Agree", "Neither disagree or agree", "Disagree" and "Strongly Agree". Statements focused on whether: (i) caregivers have something useful to share with researchers; (ii) researchers could make potential impact on caregiver health and wellbeing, (iii) the role of a caregiver is understood, and (iv) whether caregivers feel connected to researchers at university settings. In addition, we explored views on the barriers of (v) time, (vi) money and (vii) health to participation in research.

Closer inspection of our Likert responses around current and previous experiences of caregiver research suggests over 75% of caregivers agree/strongly agree that caregivers have something "useful to share" with researchers, yet far fewer respondents felt that researchers could make a significant impact on their health and wellness as a caregiver (Figure 1). Few caregivers responding to this work felt they were connected to researchers in universities. When asked to rate whether they felt connected to university research, 29% (24/84) of participants strongly disagreed with this statement (Figure 1). It was suggested that university researchers do not have a particularly strong grasp of the realities of a caring situation.

#### 3.2.2. Future Research Participation

Online and community-based events, along with newsletters, were the most common ways in which caregivers felt they could contribute to university-led research in the future. However, when asked what the best way would be for caregivers to work alongside academia in the future, online and in person were the most popular options, and almost equally split among participants. The split opinion seems to be between the gold standard being face-to-face to ensure a rapport is built between caregiver and researcher, among other benefits, and the practicality and perhaps reality that online would be more suitable, given the time pressures faced by caregivers. The majority (53%, 39/73) of respondents would be willing to commit 30 min or more a month to participating in research.

#### 3.2.3. Future Research Aspirations

In terms of future focus for the health and wellbeing of caregivers, from Likert scale responses, there was a need stated for a diverse range of areas including: information-based research, remote monitoring technology, communication technologies to enable connection between caregivers and policy related research to help shape national agendas and policies (Figure 2a). When asked about different ways in which informal caregivers could provide input in research, many placed high value on all the options presented to them, including developing new ideas and new approaches to working together. Other popular options included ensuring that fresh solutions are relevant for other caregivers. Reviewing innovative technology ideas in development was not a clear priority for caregivers compared to other solutions. Thematic analysis of participant quotations highlighted a strong need to improve the support provided to caregivers. This included financial, emotional, psychological and training and educational support. It was very apparent that participants do not currently feel heard or that they have received sufficient levels and/or quality of support in their role as a caregiver.

**Figure 2.** (**a**) Caregivers' responses to a range of statements according to whether they are "Very important", "Important", "A little important", "Not at all important". Statements included whether caregivers were interested in (i) finding solutions, (ii) developing new approaches to work with researchers, (iii) measuring success, (iv) developing training and support, (v) planning ahead, (vi) developing new ideas, and (vii) reviewing new technologies. (**b**) A separate question for caregivers focused on whether future work would be "Very useful", "Useful", "Somewhat useful" or "Not Useful". Statements included research around: (i) information (ii) policy, (iii) communication, (iv) remote monitoring, (v) mobility, (vi) new ways of learning, (vii) voice activated technologies, (viii) finance based and (ix) VR/AR. VR = Virtual Reality, AR= Augmented Reality.

Further feedback for future work was obtained around which specific areas of collaboration would make the biggest impact on caregiver health and wellness (Figure 2b). There was a wide array of different interests in this group, for example, information science was a particular area of interest highlighted where caregivers viewed research that shares key information around: caring policy, rights, and entitlements as a priority. While more than 50% of the participants viewed research around mobility, innovation and learning, finance and voice activated research as useful/very useful, fewer participants were interested in Virtual Reality/Augmented Reality as a research priority.

#### *3.3. Professionals*

Three key themes were established from the structure of our questioning and eight sub-themes were identified from the thematic analysis. The three pre-specific (deductive) themes were; (1) Previous research participation, (2) Future research participation and (3) Future research aspirations. The key themes and sub-themes are presented in Table 3.

**Table 3.** Key themes and sub-themes from Professionals Survey.

#### 3.3.1. Previous Research Participation

Professionals were asked if they had previously been involved with or collaborated with universities and just over 62% (38/61) of respondents said they had not. When asked to comment further, many of those who had previous experience with research had done so in a professional capacity as part of their role. A key reason for having participated in research or being interested in doing so in the future was the importance and want to share their knowledge and experience.

The majority of professionals suggested that caregivers do not feel connected to universities or researchers and, similarly, many did not think that those working in universities have a good understanding of the challenges faced by caregivers (Figure 3). However, the vast majority of respondents felt that caregiver experiences would be extremely valuable to researchers and that research could have a significant impact on the health and wellbeing of caregivers (Figure 3). When asked about barriers to caregivers participating in research, lack of time was the most significant barrier, followed by money and then poor caregiver health.

#### 3.3.2. Future Research Participation

Professionals stated that online group activities and community-based events were the most popular ways which would make it easier or more likely for people to contribute to caregiver research 82% (46/56) and 66% (37/56), respectively. However, when asked to explain further, the importance of engaging those who are harder to reach and not simply using charity organisations as a means of recruiting caregivers to participate in research came through strongly in the free text answers. Perhaps, reflective of the COVID-19 situation people were living in at the time of completing the survey, online methods followed by in person face-to-face meetings were preferred with phone-based and postal methods being less popular. Many respondents, however, highlighted the benefits of face-to-face meetings.

#### 3.3.3. Future Research Aspirations

We asked professional respondents similar questions to caregivers around research priority areas. Although there was some indication that finding available solutions and measuring success were important, training and support, planning ahead, developing new ideas and developing new approaches to work with research were all a priority that over 50% of professionals thought were "important" or "very important" (Figure 4a). In terms of specific types of technology, respondents felt research and development into all areas of technology would be useful (Figure 4b). Those respondents appeared to think that what would be particularly useful were technologies enabling remote monitoring and communication technologies to facilitate connection between caregivers and other caregivers or professionals. Interestingly, virtual reality technology was of interest, but noticeably less so than other forms of technology (Figure 4b).

**Figure 4.** (**a**) Professionals' responses to a range of statements according to whether they are "Very important", "Important", "A little important", "Not at all important". Statements included whether caregivers were interested in (i) finding solutions, (ii) developing new approaches to work with researchers, (iii) measuring success, (iv) developing training and support, (v) planning ahead, (vi) developing new ideas, and (vii) reviewing new technologies. (**b**) A separate question for professionals focused on whether future work would be "Very useful", "Useful", "Somewhat useful" or "Not Useful". Statements included research around: (i) information (ii) policy, (iii) communication, (iv) remote monitoring, (v) mobility, (vi) new ways of learning, (vii) voice activated technologies, (viii) finance based and (ix) VR/AR.

When asked to elaborate on the most important outputs they felt could come from their involvement in caregiver research, the response fits into four key themes: (1) Solutions that are relevant to caregivers and truly reflective of their needs, (2) Long-term, sustainable solutions, (3) For caregivers to feel empowered and supported in their role, (4) Possible impact on policy and practice.

#### **4. Discussion**

The pressures on the caregiver demographic continue to grow. An ageing growing global population [19], the disruption caused by COVID-19 [8] and limited support for the caregivers are a perfect storm for a population already under strain. Recent UK statistics suggest that year on year, that stress outcomes have been worsening for caregivers, underlining the need for disruptive change [20]. Here, we have explored what a future roadmap for innovation could look like through a multi-stakeholder consultation and evaluation: an area of considerable importance for public health. Through incorporating the views of 174 individuals (including 112 informal caregivers and 62 professionals), there are a wide range of unmet needs, and an appetite to work closer with researchers and universities through both in person and online approaches (see Figure 5). While our responses suggest that working together to understand core needs is plausible for a sizable group of caregivers, care must be taken not to become overzealous in our interpretations many caregivers may remain difficult to reach, including those who are delivering care on 24/7 basis. Nevertheless, this work forms some early steps to engaging the wider caregiver demographic and, taken collectively, our findings suggest a need to: (i) identify and work sustainably with caregivers, (ii) listen to, and encompass the wide range of needs and perspectives of caregivers and other stakeholder groups, (iii) improve the quality and quantity of methodologies for caregiver research and (iv) widely share research knowledge (e.g., successful and non-successful innovations for research/implementation).

**Figure 5.** Summary figure summating both informal caregiver and professionals/innovators interested in the caregiver demographic.

While we achieved our main objectives, there are several limitations. Our work represents an online survey where there are biases caused by convenience sampling. For example, representation from ethnic minorities is relatively sparse, however, this was not a specific objective we aimed to address and further work is required to understand whether our findings are paralleled in such groups. We may also be inadvertently missing out on views on those with substantial caring commitments or those who do not engage in digital technologies. Second is

that, while we have attempted to be as objective as possible in our analyses and interpretations, our questions were pre-specified and, therefore, our qualitative analysis should be interpreted as deductive in terms of thematic approach. Third is that there are many important barriers to innovations that we cannot comment on, such as government incentives, cost-benefit models and start up environments. Our responses are based almost exclusively in the Scottish setting: in future, it would be advantageous to better understand other parts of the UK, alongside more nuanced differences such as differences between urban vs. rural settings. Finally, this work took place during the COVID-19 pandemic, which may have skewed representation to more digitally advanced groups of caregivers and missed important viewpoints.

Our main findings highlight a need to identify and work sustainably with caregivers: a task not straightforward given the lack of emotional, physical, and monetary support readily available. Moving forward, there is a need to find better ways to recognize the value that caregivers bring. This may be achievable through developing a more continuous interaction with caregivers, for example, moving beyond individual projects to open ended or problem focused collaborations. While many caregivers would welcome routes to engage with universities, researchers must see that the process needs to be efficient and inclusive: ideally taking no more than 20 to 30 min a month. Relevant to such a trade-off between researchers aims and citizen engagement is the concept of living labs. Living labs are a methodology/approach that can allow citizens to participate in the design, development, and evaluation of innovative solutions to address societal problems [21]. There is considerable interest in using such approaches to explore health and wellness problems, including scope to carefully explore a range of different issues through participatory design, across the life course [22,23]. These approaches are helping to redefine research methodologies, including the integration of citizen generated and nontraditional data sources [24,25]. Although careful consideration will be required of how to best build upon emerging online methodologies (e.g., ethical use of data), our findings here indicate that a hybrid model of both online and in-person approaches could work well for the caregiver demographic. "Citizen science" approaches could facilitate multiple research stages, such as ideation, co-design of the approach, data-gathering and knowledge transfer of findings through "light touch" interactions (e.g., smartphone/computer-based) or through community "pop up" events. Critically, this would allow multiple routes to help different groups of caregivers influence research agendas within the earliest stages [26]. Such integration could help address known issues around translational failure [15] and be supported through a wide variety of tools such as prioritization methodology [27] alongside recognition in academic publications [28].

Another key finding from this work is that caregivers (who were able to respond to this work) would largely welcome being "championed" and provided with a channel to engage with universities. However, the practicalities of delivering this are yet to be realized—political and social landscapes change over time, as do priorities. While both we and many other research groups [29,30] consult caregivers about current unmet needs on a regular basis, there remains a risk that some specific groups (e.g., according to age, geographical location, ethnicity, socioeconomic status, number of hours caring per week) will be excluded from conversations through selection bias [31]. Further work is needed to contextualize how representative current research is of all caregivers and, critically, we must establish community links to help those in need who do not traditionally connect with research agendas or third sector partners. Lastly, while the concept of sustained engagement proposed here may be new to many university settings, there is a need to ensure efforts augment, opposed to replace existing efforts ongoing elsewhere (e.g., health and social care data, charity, and voluntary sector annual reports).

Given that much of the conversation around caregivers, needs and innovations falls out with traditional models of empirical research (i.e., Randomized Controlled Trials), there is a need to keep furthering knowledge of research methodologies—this could be to improve the value of non-traditional data sources and/or to increase capacity for more caregivers to become involved. A source of inspiration that highlights passive involvement from participants includes the "Dreamlab" project from University College London [32], developed so that individuals can engage with researchers through charging their smartphone

to assist with cancer research. As the concept has already reached 1 million downloads, this is a clear indication that citizens are interested to connect and make impacts to research priorities. Further, the recent contributions of the COVID ZOE smartphone app have also underlined the power of citizen science, whereby seemingly small actions of individuals on a larger scale can accelerate current knowledge to help navigate the unchartered waters of an emerging pandemic [33]. Given such examples and our participant feedback, it is feasible to see smartphones being better used to connect caregivers (and other citizens/professionals) to ongoing research opportunities and emerging agendas, and to signpost others to recruitment calls. An immediate utility of such citizen science approaches could be to connect caregivers of rarer/under-researched conditions to highlight such needs to researchers and funding councils alike.

Another key take-home point from this work is the demonstration of the real depth of knowledge that caregivers hold for the development of future innovations and co-design approaches [34]. For example, caregivers demonstrated intimate knowledge of the daily barriers and enablers to existing technologies, and from the findings presented here, many caregivers are still actively seeking solutions around information needs and remote monitoring, problems that researchers may think are already solved. In terms of contributing specifically to ongoing research, findings from our own work and others [17,29] have highlighted key research topics for caring (e.g., access to information online), and such priorities could form the foundation of conversations with caregivers in a variety of settings. An achievable and realistic first step from here could be to pick exemplar caregiver priorities, and to run pilot projects that could facilitate the development of caregiver co-designed "principles" to help scaffold the way by which researchers engage with caregivers moving forward.

#### **5. Conclusions**

To conclude, the aims of this work were to initiate the development of a roadmap for future caregiver innovation. We found that, despite a number of barriers to participation, many caregivers are willing to regularly contribute to the research agenda and have a wide range of unmet needs that could be better addressed by existing and future research outputs. Visibility is still an issue to caregivers—we were only able to hear from a small subsample of the caregiver population here and even these participants did not feel visible and heard in research agendas. As a wider community, this needs to change. Clearly, accessible routes for engagement (e.g., integrated online and face-to-face approaches) and more formal recognition of their expertise would be welcomed alongside the delivery of real-world impacts on those who care. A next logical step would be to determine which approaches (such as living lab methodologies) are capable of delivering for caregivers not just in terms of research outputs/outcomes, but in terms of transparency, engagement, and experience.

**Author Contributions:** Conceptualization, K.J.E. and K.A.M.; methodology, K.J.E. and K.A.M.; writing—original draft preparation, K.J.E., K.A.M.; writing—review and editing, K.J.E., K.A.M., L.M., R.M., L.M. and M.L.; funding acquisition, K.J.E. and K.A.M. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received funding from the University of Strathclyde "Strathwide" Early Career Research Grant awards.

**Institutional Review Board Statement:** The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the University of Strathclyde Computer and Information Science Ethics Committee.

**Informed Consent Statement:** Informed consent was obtained from all subjects involved in the study.

**Data Availability Statement:** The data presented in this study are available on request from the corresponding author. The full data are not publicly available to ensure anonymity of participants who took part in this work.

**Acknowledgments:** We would like to recognize the input of all caregivers and professionals involved in this work. We are also grateful to Patricia Clark at Carers Scotland (UK) and her colleagues to help us recruit carers to this work and to colleagues within the Digital Health & Care Innovation Centre (DHI) for assisting us to recruit professionals/innovators (including Moira MacKenzie, Kara McKenzie). We would also like to acknowledge contributions from the Centre for Lifelong Learning (CLL) at the University of Strathclyde—including Alix McDonald and Gemma Gilliland.

**Conflicts of Interest:** The authors declare no conflict of interest.

#### **Appendix A**

Q1 How old are you in years? Please select one.

Q2 How do you describe you gender? Please select one.

Q2.1 If you selected in another way, please share further information here: Q3 What is your highest level of education? Please select one.

Q4 What is your ethnicity? Please select one.

Q5 How long have you been a carer (in years)

Q6 Are you still able to work or study as you care?

#### **Appendix B**

#### **References**

## *Article* **Come for Information, Stay for Support: Harnessing the Power of Online Health Communities for Social Connectedness during the COVID-19 Pandemic**

**Brian M. Green 1,\*, Casey A. Hribar 2, Sara Hayes 1, Amrita Bhowmick 1,3 and Leslie Beth Herbert <sup>1</sup>**

**Abstract:** The COVID-19 pandemic created a globally shared stressor that saw a rise in the emphasis on mental and emotional wellbeing. However, historically, these topics were not openly discussed, leaving those struggling without professional support. One powerful tool to bridge the gap and facilitate connectedness during times of isolation is online health communities (OHCs). This study surveyed Health Union OHC members during the pandemic to determine the degree of COVID-19 concern, social isolation, and mental health distress they are facing, as well as to assess where they are receiving information about COVID-19 and what sources of support they desire. The survey was completed in six independent waves between March 2020 and April 2021, and garnered 10,177 total responses. In the United States, OHCs were utilized significantly more during peak lockdown times, and the desire for emotional and/or mental health support increased over time. Open-ended responses demonstrated a strong desire for connection and validation, which are quintessential characteristics of OHCs. Through active moderation utilizing trained moderators, OHCs can provide a powerful, intermediate and safe space where conversations about mental and emotional wellbeing can be normalized and those in need are encouraged to seek additional assistance from healthcare professionals if warranted.

**Keywords:** eHealth; chronic disease; online community; social support; COVID-19

#### **1. Introduction**

Although social isolation, loneliness, and mental health distress have always been commonplace, societal stigma often means they are not openly discussed [1]. However, the COVID-19 pandemic created a globally shared stressor that saw a rise in the emphasis on mental and emotional wellbeing. From popular icons such as Michelle Obama freely speaking about experiencing "low-grade depression" as a result of the pandemic and societal unrest in America [2], to the World Health Organization's emphasis on increased mental health infrastructure due to an impending and critical increase in demand [3], the COVID-19 pandemic has changed the way we talk about, prioritize, and consume healthcare and services that address mental health needs.

Despite still being in the midst of the pandemic, research so far points toward COVID-19 s ubiquitous negative impact on mental health [3–6]. A Kaiser Family Foundation survey from April to May of 2021 found that 30% of adults in the United States (U.S.) reported symptoms of depression or anxiety, a rise of more than 10% higher than expected based on pre-pandemic trends. In addition, nearly a quarter of those with symptoms of psychological distress who reported needing mental health services were not able to access them [7]. These COVID-19 impacts are far-reaching; however, factors such as a previous

**Citation:** Green, B.M.; Hribar, C.A.; Hayes, S.; Bhowmick, A.; Herbert, L.B. Come for Information, Stay for Support: Harnessing the Power of Online Health Communities for Social Connectedness during the COVID-19 Pandemic. *Int. J. Environ. Res. Public Health* **2021**, *18*, 12743. https://doi.org/10.3390/ ijerph182312743

Academic Editors: Irene Torres-Sanchez and Marie Carmen Valenza

Received: 29 October 2021 Accepted: 1 December 2021 Published: 3 December 2021

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

lack of social support, perceived likelihood of survival, comorbid mental or physical health conditions, and challenging home demands, such as homeschooling young children, are all contributors to an increased risk for mental health distress [4,6].

In research conducted prior to the pandemic, social connectedness and social support were associated with benefits such as decreased risk of cardiovascular disease and depression, improved immune system functioning, and reduced morbidity and mortality, among others [8–11].

A metanalysis of 40 studies confirmed that comorbidities of mental health and chronic physical conditions are a burden for people, not just in the developed world where research often focuses, but also in developing and emerging countries [12]. While psychological interventions of various types can improve a patient's quality of life [13–15], there are mixed results depending upon in-patient or out-patient settings [16].

A positive state of mind as a result of strong social support, is also linked to greater medication adherence, a critical concern when it comes to living with one or more chronic health conditions [17].

Chronic health conditions are of particular interest in examining societal responses to the COVID-19 pandemic, as physical lockdowns and fears of contagion prevented the ease of access to in-person healthcare [18]. In addition, current research still suggests that those with pre-existing chronic conditions (including those with a history of malignancy) have a greater risk of developing severe COVID-19 and accompanying significant morbidity and mortality than those without [19,20]. Further, the population of chronically ill individuals tends to be older, and inherently possesses an increased susceptibility to social isolation and deteriorating mental health [21].

Especially in the context of the COVID-19 pandemic, telemedicine provides a means for patients to seek care for a variety of concerns in a convenient, accessible, and safe manner. While telemedicine for mental health is on the rise, there are still questions regarding patient willingness to embrace these new options when discussing sensitive issues. Additional concerns include payor issues with telemedicine services and the equitable access to technology required in order to undergo sessions [22–24].

One powerful tool to bridge the gap and facilitate social support and connectedness during times of isolation is online health communities (OHCs). OHCs are online platforms in which individuals with similar health conditions or experiences can share information, support, and connections [25,26]. OHCs provide a temporally flexible space that allows people to connect across geographic locations and often with anonymity [26,27]. OHCs allow patients to play as active or passive a role as they would like, while helping with identity development, self-confidence, personal validation, social interconnectedness, navigation of complex emotions, and fostering a sense of purpose [27–30]. Although OHCs are not a platform for medical care nor advice, they can provide an environment to gauge similar experiences and gain confidence to seek professional support.

In order to facilitate safe, online spaces that provide support, social connection, and information, Health Union created an Adaptive Engagement Model for OHCs [30]. At the beginning of the pandemic, Health Union had 28 active OHCs. Each community is a separate URL that matches the name of the condition, for example, Lupus.net, and has social media pages on Facebook, Instagram, and Twitter.

These OHCs are contextual and situational, and feature three core structural elements, including social support, adaptive engagement, and active moderation. Similar to physical communities, they rely on a shared identity (connection to a specific chronic health condition), social norms, and commonly an experience of societal stigma. These features, taken together in the creation and subsequent management and moderation of the OHCs, are used to facilitate the sharing of relevant health information (or content), build relationships, and harness participant interdependence through passive and active engagement opportunities [30]. Given the Adaptive Engagement Model's high emphasis on combating social isolation and creating space for normalizing conversations about life with a chronic illness, including mental health impacts, these OHCs may serve as a

stepping stone for intermediate support as people struggle with considering and initiating professional services for mental health or emotional support.

The present study was conducted primarily to determine the information needs and supportive resources needed by people with chronic health conditions from Health Union's 28 online communities during the pandemic. After seeing how community members continued to engage and support each other throughout the pandemic and become less interested in specific content regarding COVID, the authors decided to conduct a secondary analysis of these data to further study social connectedness.

The aim of this study was to survey current Health Union OHC members to determine the degree of COVID-19 concern, social isolation, and mental health distress they are facing, as well as assessing where they are receiving information about COVID-19 and what sources of support they require. Although other research focused on the mental and emotional impacts of COVID-19 and the use of telemedicine or other technologies to access healthcare services, our efforts aim to investigate the novel space in between, wherein OHCs may play a critical role in fostering wellbeing, especially when access to traditional in-person services may be limited.

#### **2. Materials and Methods**

#### *2.1. Data Collection and Survey Design*

As this was a secondary analysis of a de-identified data set originally collected for quality improvement purposes, it was considered exempt from human subject review. This research was conducted according to the guidelines of the Declaration of Helsinki and other relevant laws in the U.S. Informed consent was obtained from all survey participants via an introductory email, allowing people to voluntarily click through to accept or decline the invitation to complete the Qualtrics survey. Survey responses were anonymous and not linked to individual identifiers. All email or IP addresses were stripped from the data set prior to data cleaning, storage, and access of the final data set for analysis by the authors.

Data were collected in an online survey format hosted through Qualtrics Survey Software (Qualtrics International Inc., Seattle, WA, USA). The survey was administered in six waves between 19 March 2020 and 19 April 2021. The first survey wave was sent to members of 10 separate OHCs hosted, managed, and moderated by Health Union. Subsequent waves were expanded to eventually reach a total of 28 separate OHCs managed and moderated by Health Union. Each platform provides a space for support, engagement, and education around the chronic health condition of focus [30]. In total, the survey was sent to community members via email across the six waves. Banners and other site features advertising the survey were also displayed on each platform during data collection periods. Data collection for each wave lasted between three and eight days, tailored with the aim of securing an adequate sample size, while being inclusive of people with many different chronic health conditions (Table 1).

**Table 1.** Survey wave sample size, number of OHCs included, dates fielded, and completed responses.

In order to participate in the survey, respondents needed to be at least 18 years old, live in the United States, have a chronic health condition from a pre-specified list (full list in Supplementary Materials), and be aware of COVID-19. Survey questions focused on life with a chronic condition during the pandemic. Initial topics of interest included current treatments used to manage chronic health conditions, where information was obtained, and

concern regarding COVID-19. In addition, questions regarding changes in personal health behaviors as a result of the pandemic, changes in established treatment plans, desired support sources, communication with healthcare providers, and financial impacts were also included.

Given the evolving and uncertain nature of the COVID-19 pandemic, it was necessary to adjust topics slightly to most accurately reflect the current status of the pandemic for that time period. For example, waves three through six included questions about telehealth at the same time that telehealth was expanded by executive order through the Centers for Medicare and Medicaid Services during this time period [31]. Waves five and six included questions surrounding pandemic burnout, quality of life, and vaccination status. Due to this required flexibility, each wave ranged from 35 to 41 questions.

Participants who provided complete responses to the survey were entered in a drawing to receive a U.S. e-gift card for each wave. The drawing for the first wave featured USD 50 gift cards, while waves two through six featured USD 25 gift cards.

#### *2.2. Measures*

#### 2.2.1. Perception of COVID-19

Concern for COVID-19 was assessed in each wave through the question, "At this time, how concerned do you feel about the novel coronavirus (COVID-19)?" Concern was measured on a 7-point Likert scale (1 = Not at all concerned to 7 = Very concerned). Wave six (post-vaccine availability) had a slightly modified version of this question by gauging agreement with the statement, "I am still very concerned about my risk of contracting COVID-19" (7-point Likert scale with 1 = Strongly disagree and 7 = Strongly agree). In a similar fashion, participants were asked about their concern with COVID-19 in the context of having a chronic health condition in each survey wave (Supplementary Materials). Significant concern was a response of 6 or 7, while a lack of concern was a response of 1 or 2. In several waves (waves two, three, and four), participants were also asked in an open-ended question to provide one word that described how they were feeling about COVID-19.

#### 2.2.2. Utilized and Desired Sources of COVID-19 Information and Support

In order to determine current COVID-19 information resource use, respondents were prompted in waves one through four with the question, "What sources are you using to learn more about the novel coronavirus (COVID-19)?" Participants were prompted to select as many as applied from a list of resources, including social networking sites, Internet search engines, government websites, online blogs or support communities, and TV news reports (Supplementary Materials). Subsequently, participants were asked, "What types of information and/or support would be most helpful to you right now?" Respondents were permitted to choose up to three desired types of information or support including information from their healthcare provider about COVID-19 in relation to their health condition, emotional and/or mental health support, financial support for medications, and home delivery options (Supplementary Materials).

#### 2.2.3. Quality of Life and Health-Related Behavioral Changes

Later waves, specifically waves five and six, incorporated questions about quality of life, mental health impacts, burnout, and perception of returning to pre-pandemic "normal" life. Changes in more tangible behaviors were asked in wave five, through a 3-option ranking of doing less, doing the same as, or doing more of a specific behavior pre-pandemic versus present. For example, participants were asked to rank their current social media use with pre-pandemic levels on this scale. Less tangible changes, such as stress, impacts on mental health, and overall quality of life were asked in wave six using a 5-point Likerttype scale, including options such as mental health being much worse during COVID-19, somewhat worse, about the same during COVID-19, somewhat better, and much better during COVID-19 (Supplementary Materials). Wave five also featured the open-ended

question, "What is the biggest struggle that you're having at this point in time, as a result of (or related to) the coronavirus (COVID-19) pandemic?"

#### 2.2.4. Demographics

Participants across all waves were asked a series of demographic questions including age, gender, annual household income, primary health insurance form, residence type (suburban, urban, rural), highest level of education attained, and employment status. Age was selected from a dropdown menu, while others allowed participants to choose from a categorical list of item responses. Participants were asked to select chronic health conditions from a list including, but not limited to, COPD, migraine, asthma, HIV, rheumatoid arthritis, hypertension, and several types of cancers (Supplementary Materials).

#### 2.2.5. Analysis

Responses were analyzed using descriptive statistics and z-tests to explore differences across waves. This analysis was conducted in order to identify differences in the need for mental and/or emotional support and changes to quality of life relative to each wave of the survey and the corresponding time period of the pandemic. Responses to open-ended questions were reviewed for common themes and impactful quotes regarding the need for social support.

#### **3. Results**

#### *3.1. Participants*

In total, there were 10,177 responses across all six waves. The number of survey participants for each wave is shown in Table 1. Demographics of participants for each wave, including, but not limited to, mean age, gender, and employment status, were collected (Table 2). The most commonly experienced chronic health condition was hypertension, followed by asthma, migraine, rheumatoid arthritis, COPD, and multiple sclerosis. Nearly seven in ten had never been diagnosed with cancer. Of those diagnosed with cancer, skin cancer (of any form) was the most common.

**Table 2.** Survey participants; select demographics per wave.

The majority of participants were female; however, this is in line with prior research among people who were seeking health information or support online [30]. Unsurprisingly,

given the chronically ill nature of participants and the older average age, nearly half were either on disability benefits or fully retired, and roughly 30% were either employed full-time, part-time, or self-employed (Table 2).

#### *3.2. COVID-19 Concern*

The percentage of respondents reporting a 6 or 7 on the scale of general COVID-19 concern stayed relatively constant throughout waves one through five, peaking in the first wave with 71% of respondents (*n* = 699) and remaining in the mid-to-high 60% range through to wave five. In wave six, the question was adjusted slightly to inquire about the concern of contracting COVID-19 (after over one year of infections and the introduction of several vaccines). As such, significant concern for personally contracting COVID-19 in wave six was only 38% (*n* = 370). The trends in patterns of concern with regard to personal health history were the opposite for those with general chronic health conditions versus those with cancer. Those with a cancer history reported the highest levels of concern for COVID-19 in relation to their personal health at the beginning of the pandemic (wave one, 86%, *n* = 105). However, this significantly decreased to the 40% range for waves two through four (*p* < 0.01). In contrast, those with non-cancerous chronic health conditions saw a steady trend in concern for COVID-19 in relation to their personal medical history, with those reporting a strong concern hovering between 67% and 71% throughout the field of study.

#### *3.3. Self-Directed Research and Desire for Additional Support*

The top sources respondents used for COVID-19-related information throughout waves one through four were TV news reports (64%, *n* = 4591), government websites such as the CDC or NIH (61%, *n* = 4393), news websites (57%, *n* = 4099), and Internet search engines such as Yahoo or Google (47%, *n* = 3353). Social networking sites such as Facebook and Twitter were used by 38% of respondents (*n* = 2721), and online blogs and support communities were used by 17% (*n* = 1196). Internet searches, websites for healthcare professionals such as academic journals, TV news reports, and social media were all utilized more heavily in the beginning of the pandemic relative to later waves (*p* < 0.01 for wave one versus waves two, three, and four, individually). Online blogs and support communities were utilized significantly more in wave two (19%, *n* = 426, *p* < 0.01) than in any other wave, coinciding with peak lockdown times in the United States (April 2020).

When asked which types of information and/or support would be most helpful to respondents, the three most commonly chosen sources were up-to-date and accurate information about COVID-19 (47%, *n* = 2921), emotional and/or mental health support (25%, *n =* 1581), and financial support for other necessities/bills (22%, *n* = 1356). Most notably, the desire for emotional and/or mental support was highest in the final wave wave four (*p* = 0.01 compared to wave two and *p* < 0.01 for wave three).

#### *3.4. Burnout, Isolation, and Mental Health Distress*

The impacts on mental and emotional health were assessed most directly in waves five and six, after roughly a year of the COVID-19 pandemic. Nearly 60% (*n* = 1143) of respondents in wave five said the pandemic had increased the level of stress and/or anxiety in their daily life, compared to 9% (*n* = 183) who did not share this perception (*p* < 0.01), and 62% (*n* = 1240) were currently worried about returning to "normal" activities (*p* < 0.01). Additionally, in wave five, when asked about family and friends, 38% (*n* = 764) reported that they are keeping in touch with loved ones less than before the pandemic, compared to only a quarter (*n* = 500) reporting an increase in connection (*p* < 0.01). Notably, about one quarter of those who sought telehealth for any reason in this group did so for a mental health counseling or therapy session (25%, *n* = 371).

In wave six, 60% (*n* = 588) said quality of life was somewhat or much worse as a result of the pandemic, while only 9% (*n* = 86) reported an improvement (*p* < 0.01). When asked directly about mental health impacts, 56% of this group felt their mental health was worse or much worse during the pandemic compared to pre-pandemic (*n* = 547, *p* < 0.01), and only 6% (*n* = 63) reported improvements in mental health.

#### *3.5. One-Word Perceptions*

Responses to the one-word, open-ended question about current feelings regarding COVID-19 were reviewed and a word cloud figure was generated (Figure 1). In the word cloud format, the most commonly cited words are represented as larger in the figure. As shown in Figure 1, earlier in the pandemic, aligned with lockdowns and CDC calls to practice social distancing, the terms "anxious", "scared", and "concerned' were the most frequently cited. In wave six, more than a year after the pandemic began and with vaccines becoming available, words like "frustrated", and "tired" are still prominent; however, new terms like "hopeful", "optimistic", and "cautious" are entering into the vocabulary again, occupying more prominent positions in the shared consciousness.

**Figure 1.** One-word perceptions of the pandemic in word-cloud format: (**a**) Wave four (July 2020); (**b**) Wave five (April 2021).

#### *3.6. First-Hand Accounts of Isolation and Longing for Connection*

Wave five included an additional open-ended response where respondents were asked about their biggest current struggle in relation to the COVID-19 pandemic. Upon the first review of responses, isolation, stress, mental health concerns, and longing for social connectedness were common themes. Several notable responses included:

*"The biggest struggle for me is the isolation. The last time I was out to eat with friends or shopping in a store was the end of February 2020. It's a more mental/emotional struggle most days."*

*"Boredom due to seclusion. Normally I busy myself with helping others but being secluded in my room with little to do has begun to wear down my normally positive attitude."*

*"I don't have any help at home, and it's hard for me to manage. I feel incredibly isolatedwhich causes increased depression. I cannot even participate in communal worship because of immunosuppressant medications that increase my risk for COVID. I would like to work, as much as I can, but as a piano teacher, it is not possible, and that is more isolating and makes one feel more 'useless.' Being ill, unemployed, and having no one to have physical contact or interactions with is not normal. It is not conducive to mental health, [which is] hard enough for those with chronic pain and health conditions."*

Alongside survey responses, Health Union's OHCs were continuously monitored for trends in member engagement. Notably, concurrent with the start of the survey fielding in late March 2020 (and the initial increasing mandates to stay at home due to the COVID-19 pandemic), the following unprompted response was shared Health Union's OHC dedicated to asthma:

*"Warm greetings to everyone in the asthma world! As I sit here on day 22 of isolation during the COVID-19 pandemic, I am reminded of my reasons for wanting to share my experiences as an asthma patient and a lung cancer survivor, to name a couple of my health issues. So I am in isolation and I can't help but think about how grateful I am to have this forum to turn to. Not just in today's current environment but always. It is so helpful to hear from so many others who are in the same boat. Asthma often makes me feel isolated and alone. In reality, I am alone, but WE are together in our little corners of the planet doing our best to stay safe and healthy and live our lives. The current climate of the world has intensified that for us."*

#### **4. Discussion**

The overall objective of this study was to determine OHC members' changing perceptions of the COVID-19 virus and its impact on mental wellbeing and need for emotional or social support, as well as to determine where people are seeking their information and support needs from. Although government websites, TV news reports, and Internet searches dominated as information sources early in the pandemic, online blogs and support communities (including OHCs) reached their peak reported usage during the time of mass lockdowns and uncertainty in the U.S. (April 2020). This difference in the time of the pandemic suggests that people living with chronic health conditions sought social connectedness, validation, and peer-to-peer information in a time when they were experiencing more distress. This notion was further strengthened by the overall increasing desire for emotional and/or mental support as the pandemic progressed from the time of waves two through four.

While TV news sites, general Internet searches, and need for financial support were always of high priority to respondents no matter the time frame, the steadily increasing interest in OHCs speaks to the desire for human connection and social support from others during an unpredictable time. By providing a safe and always available online space for these connections, OHCs can provide trusted information, validate concerns and emotions, and provide social support in order to enhance wellbeing.

General concern for COVID-19, specifically concern about the risk of being infected, decreased by wave six when a multitude of infections in the U.S. had already occurred (including potentially among respondents or their family members, thereby potentially decreasing their fear of the unknown). "COVID fatigue" was at an all-time high according to public opinion polls and news reports [32], and several vaccines were newly granted emergency use authorization by the FDA and were becoming widely available. The concerns associated with the earlier phases of the pandemic, such as fear of the unknown and concern about becoming infected, were replaced by new concerns. These included burnout, negative quality of life impacts, reduced connection to loved ones, and mental health deterioration. A large majority reported that the pandemic increased the stress and anxiety they felt in their daily lives, and that they were still apprehensive about returning to a pre-pandemic "normal".

During this same time period, although daily COVID-19 concerns may have been subdued, these were replaced with mental and emotional exhaustion. Despite these high levels of distress, survey participants indicated that they did not engage with formal mental health services. In spite of the high overall use of telehealth services in wave five (73%), only a quarter of these visits were for mental health concerns. This suggests that although mental health issues and the need for support rose throughout the pandemic, there are still barriers (whether they are social, financial, practical, or physical) to being able to access mental health services. From a socialization standpoint, OHCs may be able to facilitate

trust among users of those communities and help normalize discussions around how to seek mental or emotional support, as well as provide encouragement and validation to others who may be uncertain about using telehealth for mental health services.

Based on responses to the one-word perceptions of COVID-19 and the first-hand accounts of isolation shared in response to open-ended questions, people with chronic health conditions are willing and ready to discuss these sensitive topics in OHCs. By continuing to utilize the Adaptive Engagement Model, conversations around emotional and mental health can be validated, and encouraged within a safe and supportive online environment.

OHCs are not without their own inherent risks, however, as shown in recent news accounts of the failure of leading social media companies to appropriately respond to negative impacts [33]. Personal attacks and factually incorrect medical information may arise, and depend on individual members to think critically about information presented and separate personal affiliation from safety [25,34,35]. This can be especially dangerous when it comes to sensitive discussions around emotional and mental health topics, and mentions of potential self-harm. In order to combat this, some OHCs, such as those operated by Health Union, utilized trained moderators, wrote community rules that were shared and enforced through moderation practices, and modeled appropriate responses to users of the OHC. Moderators may have a background in health or social services, and include experienced patient advocates and trained employees who monitor for safe discourse and provide conversation encouragement, resources, and validation where appropriate.

As mentioned above, there is also the risk for mentions of self-harm when creating open dialogue around sensitive and emotional issues. One aspect of OHCs is the unpredictability of people sharing comments or concerns that may not be germane to shared content or topics, and this is often the case when people mention mental health or emotional health concerns. As the COVID-19 pandemic shows in stark relief, existing OHCs, organized around health topics, may quickly become the source that people turn to when seeking support from others. Being prepared for such conversations, with both a strategy as well as an experienced and trained group of community moderators, is critical to quickly adapt to the demands of the pandemic and similar public health crises.

#### *Limitations*

There are several limitations to this study. First, in order to adapt to the changing nature of the COVID-19 pandemic while keeping surveys manageable for chronically ill respondents, several questions from the first survey were removed or tailored across subsequent waves. This made it impossible to compare responses to all variables across all six waves; however, the sample sizes for each survey wave were large enough to gain an understanding of data points of interest from smaller wave groups. Additionally, participants were recruited from people who visited one of the 28 OHCs, clicked on a survey advertisement, and proceeded to the survey consent page, or who had previously opted-in to receive email communications. This recruitment method may have led to sampling bias, specifically for those who had already participated in an OHC environment. Further, although the survey was designed to be as user-friendly as possible, it was not completed by all participants which may indicate that those who completed the survey are more comfortable with the online survey technology, and thus, OHCs.

Future research would be best served by investigating need for mental and/or emotional health support at this current juncture in the pandemic, and by utilizing similar or identical questions throughout all iterations that further explore why people choose to come to (or avoid) OHCs, what information they may be looking for, how they find support or give support to others within these spaces, and if they feel empowered to seek further help from a healthcare professional when needed.

#### **5. Conclusions**

Along with all of society, individuals with chronic illnesses fully experience the impacts of the COVID-19 pandemic, and are likely and are likely to seek out information early and avail themselves of risk reduction strategies. Changes in desired resources over time show an increasing interest from basic information about the virus, transmission risk, and ways to minimize risk, to seeking emotional support. The interest and willingness to talk about mental health impacts and the need for support speaks to the desire for human connectedness in times of societal upheaval and the resultant severe isolation.

The literature reviewed provides a backdrop for the current study and illustrates issues relevant to social connectedness for people with chronic health conditions. This includes the potential negative impact of social media on mental health, which has salience for people with chronic health conditions, particularly during the pandemic. While the increase in the availability of telehealth during the pandemic has the potential to increase the use of mental health services, it is less clear that people were able to avail themselves of these opportunities.

The first-hand accounts of people living with chronic health conditions struggling to find support and social contact are illustrated in the responses to open-ended questions and show a range of emotional impacts and coping strategies. Although mental and emotional distress are common, not all who need professional support feel comfortable or know how to access those services.

OHCs can provide an intermediate and safe space where conversations about mental and emotional wellbeing can be normalized and those in need are encouraged to seek additional assistance from healthcare professionals if warranted. However, OHCs cannot do this through passive engagement only. Active moderation of OHCs using trained and experienced moderators can provide a safe space with planned, real-time strategies to address crisis situations, including future pandemics or public health emergencies, as they arise. This research supports the thesis that OHCs, when managed and moderated appropriately, have the power to normalize mental health discussions, thus providing a unique value to those who experience mental health concerns.

**Supplementary Materials:** The following are available online at https://www.mdpi.com/article/10 .3390/ijerph182312743/s1, File S1: Wave 5 Survey Questions.

**Author Contributions:** Conceptualization, B.M.G. and S.H.; survey questions, S.H. and L.B.H.; methodology, L.B.H.; formal analysis, L.B.H.; resources, A.B.; data curation, L.B.H.; writing—original draft preparation, C.A.H.; writing—review and editing, B.M.G., C.A.H., S.H., A.B. and L.B.H.; project administration, L.B.H. and A.B. All authors have read and agreed to the published version of the manuscript.

**Funding:** This research received no external funding.

**Institutional Review Board Statement:** This study was determined as Exempt under 45 CFR 46.101(b)(#2). The study was conducted according to the guidelines of the Declaration of Helsinki and other relevant laws in the U.S.

**Informed Consent Statement:** Informed consent was obtained from all survey participants involved in the study.

**Data Availability Statement:** The data presented in this study are available by request from the corresponding authors. Additional summary reports using Data from the surveys described in this study are available here: https://health-union.com/blog/covid-19-resources-pharma-marketing (accessed on 30 November 2021).

**Conflicts of Interest:** The authors declare no conflict of interest with the reported research results.

#### **References**

### *Article* **The Interactions of Media Use, Obesity, and Suboptimal Health Status: A Nationwide Time-Trend Study in China**

**Qinliang Liu and Xiaojing Li \***

School of Media & Communication, Shanghai Jiao Tong University, Shanghai 200240, China<EMAIL_ADDRESS>
**\*** Correspondence<EMAIL_ADDRESS>
**Abstract:** Obesity and suboptimal health status (SHS) have been global public health concerns in recent decades. A growing number of works have explored the relationships between media use and obesity, as well as SHS. This study aimed to examine the time trend of the associations between media use (including traditional media and new media) and obesity, as well as SHS. The data were derived from three national random samples of the Chinese General Social Survey (CGSS), which was separately conducted in 2013, 2015, and 2017. In total, 34,468 respondents were included in this study, consisting of 16,624 males and 17,844 females, and the average age was 49.95 years old (*SD* = 16.72). It found that broadcast use and television use were positively associated with obesity and showed an increasing trend over time. Cellphone use emerged as a risk factor for obesity in 2017 and showed an increasing trend. By contrast, newspaper use, television use, and internet use were negatively associated with SHS, and television use showed a decreasing trend in the association with SHS, while internet and newspaper use showed an increasing trend. In conclusion, media use was positively associated with obesity while negatively associated with SHS. It showed a decreasing trend in the associations between traditional media use and obesity, while revealing an increasing trend in the associations between new media use and obesity, as well as SHS. The practical implications of the findings are discussed.