diagnosis is one of the  principal tasks of primary care , yet incident - reporting systems typically receive very low rates of diagnostic errors ( 1 ) . more targeted techniques , such as record screening or interviewing gps about their most memorable errors have been more successful at identifying diagnostic error but have been retrospective in their approach ( 2,3 ) .
they can not recreate the diagnostic process that resulted in an error and can not reliably determine patient features at presentation , information elicited by the clinician ( due to incomplete records and selective memory ) , other diagnoses considered , how clinical information was interpreted and what inferences were made at the time of the diagnosis ( 4 ) .
a us study of closed malpractice claims in the ambulatory setting estimated that cognitive factors ( e.g. judgment errors , vigilance and memory lapses , lack of knowledge ) were implicated in virtually all diagnostic errors , either alone ( in 55% of errors ) or in association with patient  and / or system - related factors ( 5 ) .
a recent study of difficult diagnoses in general practice , funded by the uk department of health , identified information gathering as the most important determinant of diagnostic accuracy ( 6 ) . in agreement with other studies , it found no effect of experience ( years in practice ) on diagnostic accuracy or the gathering of critical information , but did find that there was a reduction in the overall amount of information elicited with increasing experience ( 7,8 ) . computerized diagnostic decision support systems ( cddss ) may be able to play an important role in assisting information gathering , yet previously , cddss required clinicians to either enter the clinical features observed or tick them off on a list in order to receive diagnostic suggestions .
when ( i.e. type of diagnostic problem , timing during the consultation ) and  what type of support would be useful .
suggesting differential diagnoses at the start of the consultation ( on the basis of the patient 's age / sex / risk factors and presenting complaint ) is a fairly easy type of support to provide from a technical point of view , would not require clinicians to change their current practice nor the way that they record information , and could represent an easy means of implementing decision support .
suggesting , would be to draw attention to important diagnoses that might otherwise be ignored .
there is encouraging evidence that asking clinicians to engage in the systematic generation and testing of hypotheses can increase diagnostic accuracy in challenging problems without degrading performance in simple problems ( 9 ) .
a second type of support can be triggered at the end of the consultation , only when serious diagnoses have not been excluded .
alerting has the advantage of being selective , and therefore more likely to be attended to by the clinician .
the disadvantage is that it would be provided after the clinician has gone through the problem and made a diagnosis ( i.e. after the problem has been mentally represented in a certain way ) and this is known to be very resistant to restructuring and change ( 10 ) .
alerting is more resource - intensive in its development , as it will require gps to change the way that they record information collected during the consultation .
complete and accurate recording of information would be needed for the support to be triggered effectively and be useful to the clinician . in this paper
, we aimed to conduct a review of existing systematic reviews to assess the current consensus on how cddss can meet the requirements of supporting the cognitive task of diagnosis , and the currently perceived barriers that prevent the integration of cddss with electronic health record ( ehr ) systems .
since several systematic reviews of the effect of electronic decision support systems have been conducted , and the purpose of this study was not to perform another review , but to synthesize the requirements for cddss , we chose to conduct a  meta - review of existing systematic reviews and meta - reviews .
an electronic search for journal articles was conducted in june 2012 , spanning medline , embase , psycinfo and web of knowledge .
two researchers ( mn and bd ) then independently reviewed the selected full text articles , to agree a final list .
two researchers ( bd and ok ) then analysed the full text of shortlisted articles .
common reasons for exclusion included non - systematic reviews of computerized dss effectiveness and reviews of cddss effectiveness in other areas of medical decision making ( e.g. , prescription ) .
the resulting shortlist contained 32 articles ( 31 identified by electronic search ) which were read for relevance .
the repeat search in december 2014 identified a further 13 potentially eligible papers , of which one was included ( figure 1 ) .
we , therefore , were unable to comment exclusively from either a diagnostic or a primary care perspective , but have where possible identified in table 1 the number of cddss studies as opposed to other types of decision support ( order entry , guidelines , etc . ) .
our results suggest that there are four significant challenges to be met if a fully integrated cddss is to be realized .
first , a more standardized computable approach to knowledge representation is needed , and second , one that can be readily updated as new knowledge is gained .
third , a deep integration with the ehr is needed in order to trigger at appropriate points in cognitive workflow , this latter being the final challenge .
the current barriers are a failure to use dynamic vocabulary tools , which are able to capture and code relevant diagnostic findings , integrate these with the ehr and coupling these with an individualized diagnosis made by the physician . 

ahmadian et al . have examined the role of standardized terminologies in dss ( 11 ) .
they found that systems that used standard terminologies ( such as icd , snomed - ct ) were more likely to be integrated with an ehr system than stand - alone systems .
the terminology used needs to be sufficiently fine - grained and unambiguous as to adequately represent the decision rule and allow reasoning with the clinical data .
this is potentially much more complex than for clinical workflow applications . in particular , the uncertainty and challenges posed by the representation and management of a variety of complex diagnostic problems sit at the intersection of dss and computational intelligence provided by agent - based technology ( 12 ) .
a demonstration of that potential has been successfully shown in the eu fp6 ist healthagents project ( ist200427214 ) ( 13 ) .
this focused on the development of a distributed , agent - based dss , which implemented a series of automated classifiers based on pattern recognition methodologies for the diagnosis and prognosis of brain tumours . 

the implication of maintaining ontology for computerized diagnostic dss is that work will need to be done to add new problems .
( 15 ) , both make the point that not only are cddss complex interventions where the net effect is the product not just of the technology but of its uptake and use , but also that the effectiveness of the cddss will vary according to the condition studied . 

semantic interoperability with the her  standards and granularity issues . to achieve the full potential of cddss ,
the reason for this is that double entry of data is a complete barrier to the uptake of cddss .
ahmadian et al . also found that cddss that used standards for the representation of semantic concepts were more likely to be integrated ( 11 ) . 

. carried out a regression analysis of predictors of effectiveness in cddss studies , where significant factors included automatic triggering of reminders at the point of care , integration with the ehr , a direct recommendation for care and requisite reason to bypass ( 17 ) .
this finding was replicated by garg et al . , who also found that suggesting was more effective than critiquing ( 16 ) . 

common reasons for exclusion included non - systematic reviews of computerized dss effectiveness and reviews of cddss effectiveness in other areas of medical decision making ( e.g. , prescription ) .
the resulting shortlist contained 32 articles ( 31 identified by electronic search ) which were read for relevance .
the repeat search in december 2014 identified a further 13 potentially eligible papers , of which one was included ( figure 1 ) .
we , therefore , were unable to comment exclusively from either a diagnostic or a primary care perspective , but have where possible identified in table 1 the number of cddss studies as opposed to other types of decision support ( order entry , guidelines , etc . ) .
our results suggest that there are four significant challenges to be met if a fully integrated cddss is to be realized .
first , a more standardized computable approach to knowledge representation is needed , and second , one that can be readily updated as new knowledge is gained .
third , a deep integration with the ehr is needed in order to trigger at appropriate points in cognitive workflow , this latter being the final challenge .
the current barriers are a failure to use dynamic vocabulary tools , which are able to capture and code relevant diagnostic findings , integrate these with the ehr and coupling these with an individualized diagnosis made by the physician . 

ahmadian et al . have examined the role of standardized terminologies in dss ( 11 ) .
they found that systems that used standard terminologies ( such as icd , snomed - ct ) were more likely to be integrated with an ehr system than stand - alone systems .
the terminology used needs to be sufficiently fine - grained and unambiguous as to adequately represent the decision rule and allow reasoning with the clinical data .
this is potentially much more complex than for clinical workflow applications . in particular , the uncertainty and challenges posed by the representation and management of a variety of complex diagnostic problems sit at the intersection of dss and computational intelligence provided by agent - based technology ( 12 ) .
a demonstration of that potential has been successfully shown in the eu fp6 ist healthagents project ( ist200427214 ) ( 13 ) .
this focused on the development of a distributed , agent - based dss , which implemented a series of automated classifiers based on pattern recognition methodologies for the diagnosis and prognosis of brain tumours . 

the implication of maintaining ontology for computerized diagnostic dss is that work will need to be done to add new problems .
( 15 ) , both make the point that not only are cddss complex interventions where the net effect is the product not just of the technology but of its uptake and use , but also that the effectiveness of the cddss will vary according to the condition studied . 

semantic interoperability with the her  standards and granularity issues . to achieve the full potential of cddss ,
the reason for this is that double entry of data is a complete barrier to the uptake of cddss .
ahmadian et al . also found that cddss that used standards for the representation of semantic concepts were more likely to be integrated ( 11 ) . 

. carried out a regression analysis of predictors of effectiveness in cddss studies , where significant factors included automatic triggering of reminders at the point of care , integration with the ehr , a direct recommendation for care and requisite reason to bypass ( 17 ) .
this finding was replicated by garg et al . , who also found that suggesting was more effective than critiquing ( 16 ) . 

the literature on computerized cddss for medical diagnosis is sparse compared with the evidence - base that supports guideline reminders , prompts , prescribing support and other aspects of decision support .
our literature review was a meta - review of existing systematic reviews , the focus of which was to elicit requirements for the future development of cddss , and is not reported as a new systematic review .
however , we were able to identify four key requirements for the development of future cddss .
these findings must be tempered by the fact that very few of the decision support systems in the reviews were diagnostic in nature , and , given the nature of the reviews , it is impossible to separate out solely primary care cddss .
we did not sift out the different original papers from the source systematic reviews , but they were a very heterogeneous group , consisting of a variety of clinical problems and different specialist settings .
risk of bias assessment is difficult in a meta - review , and must rely on appropriate methods in the source reviews .
unfortunately , the extreme heterogeneity of source studies precluded narrative synthesis in these systematic reviews , and we are unable to make further comment on the potential for bias .
there is an urgent requirement to study primary care cddss in carefully designed randomized studies with diagnostic accuracy and patient outcomes .
we take the view that for computerized diagnostic decision support systems to be effective they need to be automatic ; linked seamlessly to the ehr and to its knowledge engine , and to give prompts and reminders at  timely points in the decision - making process . the system needs to offer an advantage to clinicians over and above the ability to generate safety - critical diagnostic alerts . with increasing litigation against general practitioners , and more than half of all claims being related to diagnostic error
, there would be considerable enthusiasm for a system that was able to utilize the dynamic vocabulary tools to quickly capture and code relevant diagnostic findings , and couple these with an individualized diagnosis based on the best - available evidence ( 19 ) .
this information would be inserted into the ehr far faster and more accurately than via most ehr interfaces .
in addition , the more richly coded information would be ideal material for enrichment of clinical prediction rules via knowledge mining techniques . a primary care focussed dss
would then enable either suggesting or alerting functions , or both to be built into the ehr system .
a recent study has shown that , with simulated cases , suggesting can improve diagnostic accuracy over unaided decision making , whilst alerting had no benefit .
there are significant opportunities offered by technical and informatics advances in the past few years that may finally allow for the development of effective computerized diagnostic decision support .
these include the development of the semantic web and standards for representing knowledge in a computable way ( 21 ) .
the pervasive availability of information in a machine - processable format , and the ability to link to devices wirelessly , means that we should no longer consider the ehr as a static , desk - bound system but rather as a knowledge - manager ( 22 ) .
this is important as it will eventually lead to a shift in professional and market place expectations of the role of ehr systems ( 23 ) .
 abstract 
 background : computerized diagnostic decision support systems ( cddss ) have the potential to support the cognitive task of diagnosis , which is one of the areas where general practitioners have greatest difficulty and which accounts for a significant proportion of adverse events recorded in the primary care setting . 
 objective : to determine the extent to which cddss may meet the requirements of supporting the cognitive task of diagnosis , and the currently perceived barriers that prevent the integration of cddss with electronic health record ( ehr ) systems 
 . 
 methods : we conducted a meta - review of existing systematic reviews published in english , searching medline , embase , psycinfo and web of knowledge for articles on the features and effectiveness of cddss for medical diagnosis published since 2004 . 
 eligibility criteria included systematic reviews where individual clinicians were primary end users . 
 outcomes we were interested in were the effectiveness and identification of specific features of cddss on diagnostic performance . 
 
 results : we identified 1970 studies and excluded 1938 because they did not fit our inclusion criteria . a total of 45 articles were identified and 12 were found suitable for meta - review . 
 extraction of high - level requirements identified that a more standardized computable approach is needed to knowledge representation , one that can be readily updated as new knowledge is gained . 
 in addition , a deep integration with the ehr is needed in order to trigger at appropriate points in cognitive workflow . 
 
 conclusion : developing a cddss that is able to utilize dynamic vocabulary tools to quickly capture and code relevant diagnostic findings , and coupling these with individualized diagnostic suggestions based on the best - available evidence has the potential to improve diagnostic accuracy , but requires evaluation . 
INTRODUCTION
METHODS
RESULTS
Sample characteristics
DISCUSSION