A Comparison of Psychometric Properties Between Internet and Paper Versions of Two Depression Instruments (BDI-II and MADRS-S) Administered to Clinic Patients Digital Education in Health Professions: The Need for Overarching Evidence Synthesis Internet Versus Mailed Questionnaires: A Randomized Comparison The Effectiveness of Technology-Based Strategies to Promote Engagement With Digital Interventions: A Systematic Review Protocol Reliability of short form-36 in an Internet- and a pen-and-paper version Faked depression: comparing malingering via the internet, pen-and-paper, and telephone administration modes. Development of a digital Childhood Health Assessment Questionnaire for systematic monitoring of disease activity in daily practice. Comparing Administration of Questionnaires via the Internet to Pen-and-Paper in Patients with Heart Failure: Randomized Controlled Trial Application of Information Technology: McMaster PLUS: A Cluster Randomized Clinical Trial of an Intervention to Accelerate Clinical Use of Evidence-based Information from Digital Libraries
Background Self-report measures can guide clinical decisions and are useful when evaluating treatment outcomes . However , many clinicians do not use self-report measures systematic ally in their clinical practice . Internet-based question naires could facilitate administration , but the psychometric properties of the online version of an instrument should be explored before implementation . The recommendation from the International Test Commission is to test the psychometric properties of each question naire separately . Objective Our objective was to compare the psychometric properties of paper- and -pencil versions and Internet versions of two question naires measuring depressive symptoms . Methods The 87 participating patients were recruited from primary care and psychiatric care within the public health care system in Sweden . Participants completed the  Beck Depression Inventory ( BDI-II )  and the  Montgomery-Åsberg Depression Rating Scale — Self-rated ( MADRS-S )  , both on paper and on the Internet . The order was r and omized to control for order effects . Symptom severity in the sample ranged from mild to severe depressive symptoms . Results Psychometric properties of the two administration formats were mostly equivalent . The internal consistency was similar for the Internet and paper versions , and significant correlations were found between the formats for both MADRS-S ( r = .84 ) and the BDI-II ( r = .89 ) . Differences between paper and Internet total scores were not statistically significant for either question naire nor for the MADRS-S question dealing with suicidality ( item 9 ) when analyzed separately . The score on the BDI-II question about suicidality ( item 9 ) was significantly lower when administered via the Internet compared with the paper score , but the difference was small ( effect size , Cohen ’s [ d ] = 0.14 ) . There were significant main effects for order of administration on both question naires and significant interaction effects between format and order . This should not , however , pose a problem in clinical use as long as the administration format is not changed when repeated measurements are made . Conclusions The MADRS-S can be transferred to online use without affecting the psychometric properties in a clinical ly meaningful way . The full BDI-II also seems to retain its properties when transferred ; however , the item measuring suicidality in the Internet version needs further investigation since it was associated with a lower score in this study . The use of online question naires offers clinicians a more practical way of measuring depressive symptoms and has the potential to save re sources Synthesizing evidence from r and omized controlled trials of  digital health education  poses some challenges . These include a lack of clear categorization of digital health education in the literature ; constantly evolving concepts , pedagogies , or theories ; and a multitude of methods , features , technologies , or delivery setting s. The Digital Health Education Collaboration was established to evaluate the evidence on digital education in health professions ; inform policymakers , educators , and students ; and ultimately , change the way in which these professionals learn and are taught . The aim of this paper is to present the overarching methodology that we use to synthesize evidence across our digital health education review s and to discuss challenges related to the process . For our research , we followed Cochrane recommendations for the conduct of systematic review s ; all review s are reported according to the PRISMA ( Preferred Reporting Items for Systematic Review s and Meta-Analyses ) guidance . This included assembling experts in various digital health education fields ; identifying gaps in the evidence base ; formulating focused research questions , aims , and outcome measures ; choosing appropriate search terms and data bases ; defining inclusion and exclusion criteria ; running the search es jointly with librarians and information specialists ; managing abstract s ; retrieving full-text versions of papers ; extracting and storing large data sets , critically appraising the quality of studies ; analyzing data ; discussing findings ; drawing meaningful conclusions ; and drafting research papers . The approach used for synthesizing evidence from digital health education trials is commonly regarded as the most rigorous benchmark for conducting systematic review s. Although we acknowledge the presence of certain biases ingrained in the process , we have clearly highlighted and minimized those biases by strictly adhering to scientific rigor , method ological integrity , and st and ard operating procedures . This paper will be a valuable asset for research ers and method ologists undertaking systematic review s in digital health education Background The use of Internet-based question naires for collection of data to evaluate patient education and other interventions has increased in recent years . Many self-report instruments have been vali date d using paper- and -pencil versions , but we can not assume that the psychometric properties of an Internet-based version will be identical . Objectives To look at similarities and differences between the Internet versions and the paper- and -pencil versions of 16 existing self-report instruments useful in evaluation of patient interventions . Methods Participants were recruited via the Internet and volunteered to participate ( N=397 ) , after which they were r and omly assigned to fill out question naires online or via mailed paper- and -pencil versions . The self-report instruments measured were overall health , health distress , practice mental stress management , Health Assessment Question naire ( HAQ ) disability , illness intrusiveness , activity limitations , visual numeric for pain , visual numeric for shortness of breath , visual numeric for fatigue , self-efficacy for managing disease , aerobic exercise , stretching and strengthening exercise , visits to MD , hospitalizations , hospital days , and emergency room visits . Means , ranges , and confidence intervals are given for each instrument within each type of question naire . The results from the two question naires were compared using both parametric and non-parametric tests . Reliability tests were given for multi-item instruments . A separate sample ( N=30 ) filled out identical question naires over the Internet within a few days and correlations were used to assess test-retest reliability . Results Out of 16 instruments , none showed significant differences when the appropriate tests were used . Construct reliability was similar within each type of question naire , and Internet test-retest reliability was high . Internet question naires required less follow-up to achieve a slightly ( non-significant ) higher completion rate compared to mailed question naires . Conclusions Among a convenience sample recruited via the Internet , results from those r and omly assigned to Internet participation were at least as good as , if not better than , among those assigned mailed question naires , with less recruitment effort required . The instruments administered via the Internet appear to be reliable , and to be answered similarly to the way they are answered when they are administered via traditional mailed paper question naires Background Digital interventions provide effective and potentially cost-effective models for improving health outcomes as they deliver health information and services that are widely disseminated , confidential , and can be tailored to needs of the individual user . Digital interventions have been used successfully for health promotion , mental health , and for enabling self-management of long-term conditions . However , their effectiveness is limited by low usage rates , with non-engagement a major challenge . Hence , it is crucial to find effective strategies to increase user engagement with digital interventions . Objective This systematic review will aim to evaluate the effectiveness of technology-based strategies  to promote engagement with digital interventions . Methods We will follow Cochrane Collaboration guidelines on systematic review methodology . The search strategy will be executed across seven e- data bases ( including MEDLINE , EMBASE , PsycINFO , CINAHL ) using the concepts “ digital intervention ” and “ engagement ” , limited by study type ( r and omized controlled trial ) . Grey literature and reference lists of included studies will be search ed . Titles and abstract s will be independently screened by 2 authors . Then the full text of potentially eligible papers will be obtained and double screened . Data from eligible papers will be extracted by 1 author and checked for accuracy by another author . Bias will be assessed using the Cochrane bias assessment tool . Narrative synthesis will report on all included studies , and where appropriate , data will be pooled using meta- analysis . All findings will be reported according to the Preferred Reporting Items for Systematic Review s and Meta-Analyses guidelines . Sources of heterogeneity will be further investigated if required . Results Our research is in progress . The final draft of the systematic review is being written and will be su bmi tted before the end of 2015 . Conclusions The review findings will inform research ers and digital intervention providers about optimal use of technology-based strategies to promote engagement with digital interventions . Trial Registration PROSPERO International Prospect i ve Register of Systematic Review s : CRD42014010164 ; http://www.crd.york.ac.uk/ PROSPERO /display_record.asp?ID = CRD42014010164#.VTZmmiFViko ( Archived by WebCite at http://www.webcitation.org/6XxQC8fT8 ) Use of Internet versions of question naires may have several advantages in clinical and epidemiological research , but we know little about if Internet versions differ with respect to validity and reliability . We aim ed to compare Internet- and pen- and -paper versions of short form-36 ( SF-36 ) with respect to test – retest reliability and internal consistency . Women referred to mammography ( n = 782 ) were r and omised to receive either a paper version with a prepaid return envelope or a guideline on how to fill in the Internet version . A subgroup was asked to answer the question naire once again in the alternative version . Test – retest reliability was assessed by the intra-class correlation coefficient . Internal consistency was calculated as Cronbach 's alpha . The between-version test – retest reliability for the eight subscales were between 0.63 and 0.92 . Cronbach 's alpha for the two versions were all between 0.75 and 0.93 with minor differences between the Internet- and the pen- and -paper version . We found little or no evidence of a difference in test – retest reliability and internal consistency when we compared an Internet- and a pen- and -paper version of SF-36 BACKGROUND The Internet and telephone are valuable re sources for mental health practitioners when access to face-to-face assessment or therapy is difficult . However , despite the high stakes ( such as access to medication or financial compensation ) often involved in psychological testing , little is known about how test administration mode influences test scores when patients are malingering . The aim of this research was to examine for the first time the equivalence of a malingered measure of depression when administered over the Internet , over the telephone , and with pen and paper . SUBJECTS AND METHODS A mixed experimental design was used with participants ( n=91 ) r and omly allocated to Internet , telephone , or pen- and -paper groups .  Participants completed the Edinburgh Depression Scale , first under st and ard instructions and then under instructions to fake as if experiencing severe depression . A manipulation check was included to ensure participants had understood and followed instructions to fake . RESULTS As predicted , participants were able to significantly increase their depression scores when asked to fake depression . Participants reported significantly lower faked depression scores over the telephone than in pen- and -paper format . Faked depression scores in the online group did not differ significantly from those in the telephone and pen- and -paper groups . However , participants in all groups met criteria indicative of a provisional diagnosis of depression . CONCLUSIONS Overall , the results indicate that there may be no meaningful difference between malingering across administration modes . If an individual is malingering depression , administration mode has minimal influence . These findings provide preliminary support for the use of online/telephone technologies in the assessment of depression OBJECTIVE To develop a reliable and user-friendly digital Childhood HAQ ( CHAQ ) to facilitate systematic monitoring of disease activity at the outpatient clinic in juvenile idiopathic arthritis ( JIA ) patients . METHODS The digital CHAQ was tested with patients who visited the outpatient paediatric rheumatology clinic of the Erasmus MC Sophia Children 's Hospital . These patients completed in a r and omized order the paper form and digital CHAQ while being observed . Validity was tested by comparing outcomes with the paper form CHAQ .  User-friendliness was evaluated through a short question naire . RESULTS A digital CHAQ was developed and revised several times according to our observations . Outcome is automatically calculated and can be printed . Fifty-one patients completed both the digital and paper form CHAQ . Correlation coefficient between both outcomes of the CHAQ Disability Index was 0.974 . No statistically significantly differences in median outcome were found in visual analogue scale ( VAS ) pain ( 25.6 vs 25.9 mm ) and VAS well-being ( 20.1 vs 19.5 mm ) . Although the mean time ( 5.06 min ) to complete the digital CHAQ was significantly longer than the mean time ( 3.75 min ) to complete the paper form , the majority of patients ( 75 % ) preferred the digital version . User-friendliness received maximum positive score . CONCLUSION We developed a reliable and user-friendly digital CHAQ , which can be easily and systematic ally completed during routine clinic visits . Such digitalization of question naires can be applied in any field to make systematic monitoring of disease activity in daily practice possible Background The use of the Internet to administer question naires has many potential advantages over the use of pen- and -paper administration . Yet it is important to vali date Internet administration , as most question naires were initially developed and vali date d for pen- and -paper delivery . While some have been vali date d for use over the Internet , these question naires have predominately been used amongst the healthy general population . To date , information is lacking on the validity of question naires administered over the Internet in patients with chronic diseases such as heart failure . Objectives To determine the validity of three heart failure question naires administered over the Internet compared to pen- and -paper administration in patients with heart failure . Methods We conducted a prospect i ve r and omized study using test-retest design comparing administration via the Internet to pen- and -paper administration for three heart failure question naires provided to patients recruited from a heart failure clinic in Toronto , Ontario , Canada : the Kansas City Cardiomyopathy Question naire ( KCCQ )  , the  Minnesota Living with Heart Failure Question naire ( MLHFQ ) , and the Self-Care Heart Failure Index ( SCHFI ) . Results Of the 58 subjects enrolled , 34 completed all three question naires . The mean difference and confidence intervals for the summary scores of the KCCQ , MLHFQ , and SCHFI were 1.2 ( CI -1.5 to 4.0 , scale from 0 to 100 ) , 4.0 ( CI -1.98 to 10.04 , scale from 0 to 105 ) , and 10.1 ( CI 1.18 to 19.07 , scale from 66.7 to 300 ) , respectively . Conclusions Internet administration of the KCCQ appears to be equivalent to pen- and -paper administration . For the MLHFQ and SCHFI , we were unable to demonstrate equivalence . Further research is necessary to determine if the administration methods are equivalent for these instruments BACKGROUND Physicians have difficulty keeping up with new evidence from medical research . METHODS We developed the McMaster Premium LiteratUre Service ( PLUS ) , an internet-based addition to an existing digital library , which delivered quality - and relevance -rated medical literature to physicians , matched to their clinical disciplines . We evaluated PLUS in a cluster-r and omized trial of 203 participating physicians in Northern Ontario , comparing a Full-Serve version ( that included alerts to new articles and a cumulative data base of alerts ) with a Self-Serve version ( that included a passive guide to evidence -based literature ) . Utilization of the service was the primary trial end-point . RESULTS Mean logins to the library rose by 0.77 logins/month/user ( 95 % CI 0.43 , 1.11 ) in the Full-Serve group compared with the Self-Serve group . The proportion of Full-Serve participants who utilized the service during each month of the study period showed a sustained increase during the intervention period , with a relative increase of 57 % ( 95 % CI 12 , 123 ) compared with the Self-Serve group . There were no differences in these proportions during the baseline period , and following the crossover of the Self-Serve group to Full-Serve , the Self-Serve group 's usage became indistinguishable from that of the Full-Serve group ( relative difference 4.4 ( 95 % CI -23.7 , 43.0 ) . Also during the intervention and crossover periods , measures of self-reported usefulness did not show a difference between the 2 groups . CONCLUSION A quality - and relevance -rated online literature service increased the utilization of evidence -based information from a digital library by practicing physicians
When digital versions of self-report symptom scales are compared to pen and paper versions , most scales show high interformat reliability .
This supports the reliability of results obtained in psychotherapy research on the Internet and the comparability of the results to traditional psychotherapy research .
BACKGROUND Research on Internet-based interventions typically use digital versions of pen and paper self-report symptom scales .
However , adaptation into the digital format could affect the psychometric properties of established self-report scales .
Several studies have investigated differences between digital and pen and paper versions of instruments , but no systematic review of the results has yet been done .
OBJECTIVE This review aims to assess the interformat reliability of  self-report symptom  scales used in digital or online psychotherapy research .