Real-time, evidence-based medicine instruction: a randomized controlled trial in a neonatal intensive care unit. Failure of Internet-based audit and feedback to improve quality of care delivered by primary care residents. The impact of an individual tutorial session on MEDLINE use among obstetrics and gynaecology residents in an academic training programme: a randomized trial. What factors are associated with the integration of evidence retrieval technology into routine general practice settings? Online access to MEDLINE in clinical settings: impact of user fees. Resident utilization of information technology. Educational workshop improved information-seeking skills, knowledge, attitudes and the search outcome of hospital clinicians: a randomised controlled trial. McMaster PLUS: a cluster randomized clinical trial of an intervention to accelerate clinical use of evidence-based information from digital libraries. Effect of a triage-based E-mail system on clinic resource use and patient and physician satisfaction in primary care: a randomized controlled trial.
The study assesses potential for improving residents' evidence-based medicine searching skills in MEDLINE through real-time librarian instruction.
                Ten residents on a rotation in a neonatal intensive care unit participated.
                Residents were randomized into an instruction and a non-instruction group. Residents generated questions from rounds and searched MEDLINE for answers. Data were collected through observation, search strategy analysis, and surveys. Librarians observed searches and collected data on questions, searching skills, search problems, and the test group's instruction topics. Participants performed standardized searches before, after, and six-months after intervention and were scored using a search strategy analysis tool (1 representing highest score and 5 representing lowest score). Residents completed pre- and post-intervention surveys to measure opinions about MEDLINE and search satisfaction.
                Post-intervention, the test group formulated better questions, used limits more effectively, and reported greater confidence in using MEDLINE. The control group expressed less satisfaction with retrieval and demonstrated more errors when limiting. The test and control groups had the following average search scores respectively: 3.0 and 3.5 (pre-intervention), 3.3 and 3.4 (post-intervention), and 2.0 and 3.8 (six-month post-intervention).
                Data suggest that measurable learning outcomes were achieved. Residents receiving instruction improved and retained searching skills six-months after intervention. To determine the effectiveness of Internet-based audit and feedback to physicians to improve care for diabetes and hypertension.
                Time-series analysis of an intervention.
                The study setting was Harvard Vanguard Medical Associates, a 14-site multispecialty group in greater Boston. The study period was July 1997-June 1999.
                were 12 primary care internal medicine residents who provided care to adult patients with diabetes (n = 76 pre-intervention and n = 88 post-intervention), hypertension (n = 329 pre-intervention and n = 338 post-intervention), or both (n = 62 pre-intervention and n = 71 post-intervention). We determined the proportion of each resident's patients whose care fulfilled national guidelines for quality (i.e. diabetes patients had hemoglobin testing in the previous 6 months or hypertension patients received a beta-blocker or diuretic in the same time period). After meeting individually with each resident to obtain informed consent and to encourage participation, we sent each resident information for accessing his or her practice profile on a secure website. The main outcome measures were (i) the proportion of resident physicians who accessed their profiles and (ii) change following the intervention in the proportion of patients whose care followed national guidelines.
                Over a 1-year period, only four of the 12 residents accessed their websites. One of the residents visited her site three times, while the other three residents visited their sites once each. In interrupted time-series analyses, the intervention had no discernible effect on adherence to practice guidelines for diabetes or hypertension.
                The lack of participation in this Internet-based intervention may have important implications for the development of future programs that require physicians to interact with technology to improve quality of care. Over the past decade, on-line databases have become increasingly popular among health care professionals. As a group, these 'end-users' report utilizing databases to keep abreast of medical progress, to conduct research and to address specific patient care issues. Throughout the literature, medical professionals ('content experts') have proved to be less effective searchers than librarians ('search experts'). The potential implications of this discrepancy are worrysome. For any given clinical scenario, for example, published reports may reach contradictory conclusions. A poorly skilled searcher may not retrieve enough articles to appreciate this fact. Optimizing searching skills is therefore a worthwhile goal. As a first step, many medical schools introduce students to on-line databases, most notably MEDLINE. Residency is an ideal time to continue this training. A recognized obstacle to provide residents with formal MEDLINE instruction is time constraint. We therefore conducted this study to ascertain the impact an individual 1-hour tutorial session would have on MEDLINE utilization among obstetrics and gynecology residents training at an academic medical centre. Outcome measures included MEDLINE search frequency, duration, recall, precision and searcher satisfaction. Search recall measures the searcher's ability to retrieve articles deemed relevant to the question at hand. Search precision gauges the searchers' ability to eliminate irrelevant articles. Although the sessions were well received, we were unable to demonstrate an improvement in the outcome measures analysed. Further research is therefore indicated so that cost-effective educational strategies can be recommended for wide-scale use. Information retrieval systems have the potential to improve patient care but little is known about the variables which influence clinicians' uptake and use of systems in routine work.
                To determine which factors influenced use of an online evidence retrieval system.
                Computer logs and pre- and post-system survey analysis of a 4-week clinical trial of the Quick Clinical online evidence system involving 227 general practitioners across Australia.
                Online evidence use was not linked to general practice training or clinical experience but female clinicians conducted more searches than their male counterparts (mean use=14.38 searches, S.D.=11.68 versus mean use=8.50 searches, S.D.=9.99; t=2.67, d.f.=157, P=0.008). Practice characteristics such as hours worked, type and geographic location of clinic were not associated with search activity. Information seeking was also not related to participants' perceived information needs, computer skills, training nor Internet connection speed. Clinicians who reported direct improvements in patient care as a result of system use had significantly higher rates of system use than other users (mean use=12.55 searches, S.D.=13.18 versus mean use=8.15 searches, S.D.=9.18; t=2.322, d.f.=154 P=0.022). Comparison of participants' views pre- and post- the trial, showed that post-trial clinicians expressed more positive views about searching for information during a consultation (chi(2)=27.40, d.f.=4, P< or =0.001) and a significantly greater number reported seeking information between consultations as a result of having access to an online evidence system in their consulting rooms (chi(2)=9.818, d.f.=2, P=0.010).
                Clinicians' use of an online evidence system was directly related to their reported experiences of improvements in patient care. Post-trial clinicians positively changed their views about having time to search for information and pursued more questions during clinic hours. The effect of introducing user fees on the frequency and quality of MEDLINE searching with GRATEFUL MED by physicians in clinical settings was tested. After training and free use (prior study), consenting participants were randomly allocated to pay searching costs (pay group) or continue without fees (no pay group). Fifty-nine physicians participated. Among the prior study's frequent searchers, the pay group searched at less than one third of the rate of those assigned to no pay. For less frequent searchers in the prior study, only 48% of those assigned to pay did any searches, compared with 85% for the no pay group (P = 0.006), and for those who did search, their frequency was almost half. However, there was no significant difference in the quality of searches; both groups demonstrated about equivalent recall (P = 0.77), but significantly lower precision (P = 0.03) than for the librarian's independent searches. Similarly, there was no difference in the proportion of searches affecting clinical decisions for the two groups. Thus, imposing user charges for online searching in clinical settings after a period of free use adversely affects searching quantity, but not quality. MEDLINE providers should consider whether user fees will undermine its benefits. To determine if a simple educational intervention can increase resident physician literature search activity.
                Randomized controlled trial.
                University hospital-based internal medicine training program.
                Forty-eight medical residents rotating on the general internal medicine service.
                One-hour didactic session, the use of well-built clinical question cards, and practical sessions in clinical question building.
                Objective data from the library information system that included the number of log-ons to medline, searching volume, abstracts viewed, full-text articles viewed, and time spent searching. Median search activity as measured per person per week (control vs intervention): number of log-ons to medline (2.1 vs 4.4, P <.001); total number of search sets (24.0 vs 74.2, P <.001); abstracts viewed (5.8 vs 17.7, P=.001); articles viewed (1.0 vs 2.6, P=.005); and hours spent searching (0.8 vs 2.4, P <.001).
                A simple educational intervention can markedly increase resident searching activity. A double-blind randomised controlled trial was conducted on a group of Hong Kong hospital clinicians. The objective was to test if a three-hour educational workshop (with supervised hands-on practice) is more effective (than no training) to improve clinical question formulation, information-seeking skills, knowledge, attitudes, and search outcomes. The design was a post-test-only control group; recruitment by stratified randomization (by profession), blocked at 800. End-user training was more effective than no training in improving clinical question formulation, in raising awareness, knowledge, confidence and use of databases, but had made no impact on preference for secondary databases. It changed the attitude of clinicians to become more positive towards the use of electronic information services (EIS). Participants had higher search performance and outcomes (satisfaction with information obtained (NNT = 3), EIS satisfaction (NNT = 3) and success in problem solving (NNT = 4)). The workshop improved knowledge and skills in evidence-based searching, but this effect gradually eroded with time. Search logs confirmed that follow-up is required if effects are to be sustained. Longer effects on search behaviours appear to be positive. A randomised controlled trial is valuable in identifying cause-and-effect relations and to quantify the magnitude of the effects for management decision-making. Physicians have difficulty keeping up with new evidence from medical research.
                We developed the McMaster Premium LiteratUre Service (PLUS), an internet-based addition to an existing digital library, which delivered quality- and relevance-rated medical literature to physicians, matched to their clinical disciplines. We evaluated PLUS in a cluster-randomized trial of 203 participating physicians in Northern Ontario, comparing a Full-Serve version (that included alerts to new articles and a cumulative database of alerts) with a Self-Serve version (that included a passive guide to evidence-based literature). Utilization of the service was the primary trial end-point.
                Mean logins to the library rose by 0.77 logins/month/user (95% CI 0.43, 1.11) in the Full-Serve group compared with the Self-Serve group. The proportion of Full-Serve participants who utilized the service during each month of the study period showed a sustained increase during the intervention period, with a relative increase of 57% (95% CI 12, 123) compared with the Self-Serve group. There were no differences in these proportions during the baseline period, and following the crossover of the Self-Serve group to Full-Serve, the Self-Serve group's usage became indistinguishable from that of the Full-Serve group (relative difference 4.4 (95% CI -23.7, 43.0). Also during the intervention and crossover periods, measures of self-reported usefulness did not show a difference between the 2 groups.
                A quality- and relevance-rated online literature service increased the utilization of evidence-based information from a digital library by practicing physicians. E-mail communication between patients and their providers has diffused slowly in clinical practice. To address concerns about the use of this technology, we performed a randomized controlled trial of a triage-based e-mail system in primary care. DESIGN AND PATIENTS/PARTICIPANTS: Physicians in 2 university-affiliated primary care centers were randomized to a triage-based e-mail system promoted to their patients. E-mails from patients of intervention physicians were routed to a central account and parsed to the appropriate staff for response. Control group physicians and their patients did not have access to the system. We collected information on patient e-mail use, phone calls, and visit distribution by physician over the 10 months and performed physician and patient surveys to examine attitudes about communication.
                E-mail volume was greater for intervention versus control physicians (46 weekly e-mails per 100 scheduled visits vs 9 in the control group at the study midpoint; P <.01) but there were no between-group differences in phone volume (67 weekly phone calls per 100 scheduled visits vs 55 in the control group; P =.45) or rates of patient no-shows (5% in both groups; P =.77). Intervention physicians reported more favorable attitudes toward electronic communication than did control physicians but there were no differences in attitudes toward patient or staff communication in general. There were few between-group differences in patient attitudes toward electronic communication or communication in general.
                E-mail generated through a triage-based system did not appear to substitute for phone communication or to reduce visit no-shows in a primary care setting. Physicians' attitudes toward electronic communication were improved, but physicians' and patients' attitudes toward general communication did not change. Growth of e-mail communication in primary care settings may not improve the efficiency of clinical care.
There is very limited evidence on effective interventions promoting the adoption of ICTs by healthcare professionals. Small effects have been reported for interventions targeting the use of electronic databases and digital libraries. The effectiveness of interventions to promote ICT adoption in healthcare settings remains uncertain, and more well designed trials are needed.