hypertension is an important worldwide public health problem and is a major risk factor for cardiovascular disease , the leading cause of death and disability worldwide .
testing for hypertension is the most commonly performed screening test in medical practice and is important not least because individuals with a normal blood pressure ( bp ) at the age of 50 still run a 90% life - time risk of developing hypertension .
furthermore , trial evidence indicates that even small differences in systolic bp of 24  mm  hg are clinically important , thus accurate measurement is vital .
errors in measurement occur through three main sources : device errors , user errors and patient errors .
device errors are potentially the most fundamental of these three sources , and , in particular , accurate measurement is underpinned by the requirement for an instrument that measures pressure accurately in patients across the range of pressures . for many years
, the standard instrument for bp measurement was a mercury sphygmomanometer , but in recent years this has been superseded in many clinical settings by automated electronic devices . the need to validate
these protocols stipulate the minimum testing population with an acceptable range of mean and s.d . of errors or an s.d . below a threshold , which is dependent on the level of mean difference , but do not specify the proportion of individuals within those populations that should have accurate readings . however , it is possible for a bp monitor to meet the above validation criteria , but still record bp measurements in error by > 5  mm  hg in half of the individuals assessed .
it has , therefore , been argued that validation standards should be based , in part , on the percentage of individual readings within an accurate range . to understand the potential effect of such a change in policy , we aimed to systematically examine the proportion of accurate readings attained by automatic digital bp devices in published clinical validation studies .
we included all published studies of automatic digital bp devices that ( 1 ) validated studies of bp devices using a recognized protocol in a clinic setting or in the community ; ( 2 ) used a sphygmomanometer with upper arm bp measurement ; ( 3 ) included adult patients ( 18 years of age ) with no significant severe intercurrent health problems and ( 4 ) included data on the mean and s.d . of the bp difference between calibrated standard sphygmomanometer ( mercury or random - zero sphygmomanometers ) and ( semi-)automatic ( oscillometric or aneroid ) sphygmomanometer .
reasons for study exclusion were devices used in special clinical situations ( for example pregnancy , at altitude , during exercise , in trauma patients , neonates , children and serious diseases ) , measurements not from upper arm bp monitoring devices ( for example wrist and abpm ) , comparison with other sphygmomanometers ( for example direct central pressure and ambulatory intra - arterial pressure ) and published papers reporting no mean difference or s.d .. we searched ovid versions of medline and embase , and cochrane library through wiley interscience from january 1989 to june 2008 .
medical subject headings and synonym terms used were sphygmomanometer , bp , hypertension , calibration and monitor device .
two authors reviewed the titles and abstracts of all identified articles and selected full - text papers .
we extracted data on the percentage of bp measurements within 5  mm  hg , mean and s.d .
we also obtained information on detail of process , including study design , sample size , device characteristics and the relevant guidelines used to assess the device accuracy .
one author ( yw ) extracted data and this was rechecked by a second author ( ch ) independently , disagreements were resolved by discussion with a third author ( rp ) .
the reviewers were not masked to any aspect of the studies ( for example journal type , author names or institution ) .
copenhagen : the nordic cochrane centre , the cochrane collaboration , 2008 for the statistical analysis and forest plots ( sorted by mean effect and device type for systolic bp and diastolic bp ) .
of differences between measured and observed bp ( the observer control measurement used in the protocol ) , and proportion of measurements within 5  mm  hg of observed bp and ordered the rows of the figures by the percentage of measurements with errors < 5  mm  hg . when this information was not available , we replaced it by a bias - corrected modelled value , defined as 0.27 + 0.76  p , where p is the percentage of errors < 5  mm  hg assuming a normal distribution , and the bias correction coefficients ( 0.27 and 0.76 ) were derived by linear regression on the trials that report all outcomes .
we analysed the effect of year of protocol by separating studies into four groups ( before 1995 , 19962000 , 20012005 and after 2006 ) and used analysis of variance and the  to estimate the amount of variation explained by year of protocol these calculations were performed using stata ( intercooled stata 10 for windows ) .
we also calculated the additional number of measurements required assuming a normal distribution for a selection of devices to achieve 95% of readings within 5  mm  hg based on the assumption of a mean difference equal to zero and a dispersion of 2.5  mm  hg ( similar to the top - rated device for each protocol ) .
we included all published studies of automatic digital bp devices that ( 1 ) validated studies of bp devices using a recognized protocol in a clinic setting or in the community ; ( 2 ) used a sphygmomanometer with upper arm bp measurement ; ( 3 ) included adult patients ( 18 years of age ) with no significant severe intercurrent health problems and ( 4 ) included data on the mean and s.d . of the bp difference between calibrated standard sphygmomanometer ( mercury or random - zero sphygmomanometers ) and ( semi-)automatic ( oscillometric or aneroid ) sphygmomanometer .
reasons for study exclusion were devices used in special clinical situations ( for example pregnancy , at altitude , during exercise , in trauma patients , neonates , children and serious diseases ) , measurements not from upper arm bp monitoring devices ( for example wrist and abpm ) , comparison with other sphygmomanometers ( for example direct central pressure and ambulatory intra - arterial pressure ) and published papers reporting no mean difference or s.d .. we searched ovid versions of medline and embase , and cochrane library through wiley interscience from january 1989 to june 2008 .
medical subject headings and synonym terms used were sphygmomanometer , bp , hypertension , calibration and monitor device .
two authors reviewed the titles and abstracts of all identified articles and selected full - text papers .
we extracted data on the percentage of bp measurements within 5  mm  hg , mean and s.d .
we also obtained information on detail of process , including study design , sample size , device characteristics and the relevant guidelines used to assess the device accuracy .
one author ( yw ) extracted data and this was rechecked by a second author ( ch ) independently , disagreements were resolved by discussion with a third author ( rp ) .
the reviewers were not masked to any aspect of the studies ( for example journal type , author names or institution ) .
copenhagen : the nordic cochrane centre , the cochrane collaboration , 2008 for the statistical analysis and forest plots ( sorted by mean effect and device type for systolic bp and diastolic bp ) .
we summarized data as mean and s.d . of differences between measured and observed bp (
the observer control measurement used in the protocol ) , and proportion of measurements within 5  mm  hg of observed bp and ordered the rows of the figures by the percentage of measurements with errors < 5  mm  hg .
when this information was not available , we replaced it by a bias - corrected modelled value , defined as 0.27 + 0.76  p , where p is the percentage of errors < 5  mm  hg assuming a normal distribution , and the bias correction coefficients ( 0.27 and 0.76 ) were derived by linear regression on the trials that report all outcomes .
we analysed the effect of year of protocol by separating studies into four groups ( before 1995 , 19962000 , 20012005 and after 2006 ) and used analysis of variance and the  to estimate the amount of variation explained by year of protocol these calculations were performed using stata ( intercooled stata 10 for windows ) .
we also calculated the additional number of measurements required assuming a normal distribution for a selection of devices to achieve 95% of readings within 5  mm  hg based on the assumption of a mean difference equal to zero and a dispersion of 2.5  mm  hg ( similar to the top - rated device for each protocol ) .
from 5912 potentially relevant records , 79 articles ( 10  783 participants ) reporting 113 studies ( including subgroups for different devices and protocols ) met the eligibility criteria ( figure 1 ) .
of these , 96 studies were recommended devices by the various protocols , 13 were not recommended and 14 devices were validated in the community ( including six not recommended by the protocol ) .
studies were from 22 different countries : us ( 23 ) ; uk ( 20 ) ; france ( 15 ) ; italy ( 10 ) ; ireland and poland ( 6 each ) ; japan ( 5 ) ; canada ( 4 ) ; russia , germany and turkey ( 3 each ) ; denmark , greece , spain and switzerland ( 2 each ) ; and argentina , brazil , china , israel , netherlands , sweden and tanzania ( 1 each ) .
studies used five different protocols for validating devices : 35 used the esh international protocol ( esh - ip ) , 9 the bhs protocol , 19 ansi / aami protocol and 22 used both the bhs and ansi / aami protocols .
the results from these are presented in figures 2 , 3 , 4 , respectively . a further 16 studies used different protocols : 3 the american heart association bp measurement guidelines , 2 the health insurance portability and accountability act protocol and 11 studies did not state which protocol they used .
all of these 16 were excluded from the present analysis , and on re - analysis , four aneroid devices were removed from the analysis . for each study , the citation , device tested and mean and s.d .
of differences are shown , together with the reported proportion of measurements within 5  mm  hg of the observed value .
some studies and/or devices appear more than once in which results were reported using different sampling schemes and/or protocols or where more than one group validated a device .
95% ci : 6795% ) , 37/41 ( 90 , 8199% ) and 34/35 ( 97 , 9299.5% ) of devices passed the relevant protocol ( bhs , aami and esh - ip , respectively ) .
no particular pattern was seen either in terms of the date of assessment or of manufacturer in terms of likelihood of a machine in terms of accuracy ( data not shown ) .
devices passing the bhs protocol had between 60 and 86% of measured values within 5  mm  hg of the observed value ( figure 2 ) .
devices failing the bhs protocol ( marked with a  ) had between 35 and 59% of test device values within 5  mm  hg . devices passing the aami protocol had between 47 and 94% of measured values within 5  mm  hg of the correct value ( figure 3 ) and for the four devices failing 3658% within 5  mm  hg .
for the esh - ip protocol devices passing the protocol , it had 5489% within 5  mm  hg , and for the one device reported to fail the international protocol , 52% were within 5  mm  hg ( figure 4 ) .
for example , in the reinders ' study , when automated measurements using the bhs protocol were taken simultaneously with the reference standard , 96% were within 5  mm  hg of the reference value ; when taken sequentially , 80% were within 5  mm  hg . in devices tested in community
/ clinic - based studies ( studies performed from a population - based perspective ) , between 35 and 46% of measured values were within 5  mm  hg of the observed value .
for the omron hem-705cp , 86% of measurements were within 5  mm  hg of the observed value under the bhs protocol compared with 46% within 5  mm  hg in the community - based study .
for the ivac model 4200 , these figures were 59% under the bhs protocol and 39% in the community - based study .
when we studied the performance of the monitors for diastolic bp , results were broadly similar .
figure 5 shows the mean and single s.d . of errors in bp monitors in repeated studies of the same monitor using either the same or different protocols .
the results for the same device could vary substantially when a different protocol was used .
for example , the omron hem-907 device could vary by 18% of readings within 5 
mm  hg depending on which protocol was used : 80% ( using the aami protocol ) compared with 62% ( esh - ip protocol ) .
the same device could also vary by 22% of readings within 5  mm  hg when the same protocol was used : 62% ( el assaad 2002 ) compared with 84% ( de greef 2007 ) .
however , the year of protocol was significantly correlated with the proportion of measurements ( r=0.257 , p<0.01 ) . overall ,
19% of the variation could be explained by changes over time ( =2581.11/13600.77=0.19 ; f=6.513 , p<0.001 ) .
subgroup analysis by protocol showed the esh - ip to be the contributor to this correlation ( the majority of its studies were later with 64% after 2005 ) , whereas the bsh and aami ( earlier studies ) were not . even if a given device could be improved to have a mean zero , it would still be necessary to reduce the variation .
for example , for a device with 74% of bp measurements < 5  mm  hg [ the welch allen vital signs ( s.d . : 5 ) ] , a further six bp measurements would be required to reduce the variation , so that 95% of readings fall within 5  mm  hg . to obtain the same precision for the pharma smart ps-2000 ( s.d . : 7 ) , a device with 66% < 5  mm  hg , a further 11 measurements are required ( based on the assumption that in each individual the bp differences between the tested device and the reference method are random ) . for the esh - ip protocol when using a device with 81% < 5 
mm  hg [ artasan cs410 ( s.d . : 5.9 ) ] , a further eight bp measurements are required to reduce variation , so that 95% of readings are within 5  mm  hg ; whereas for the omron m6 comfort ( s.d . : 7.4 ) with 68% < 5  mm  hg , a further 12 measurements are required .
the mean difference between devices that passed the protocols was relatively stable : bhs range : 2.41.5  mm  hg , aami : 2.82.7  mm  hg and esh - ip : 3.22.0  mm  hg .
devices that passed international validation protocols for bp accuracy vary greatly in quality . taking 
percentage of readings within 5  mm  hg of the observed value ' as a measure of quality , we found wide variations in quality among devices that passed protocol testing , whether the protocol was bhs , aami or esh - ip , with slight improvements in performance using the esh - ip protocol .
this is a reflection in part of the numbers of individuals required to fulfil the protocols , which affects their ability to distinguish between machines of differing accuracies : the bhs protocol requires testing on 85 patients , and , therefore , has the power to distinguish differences between devices of > 15% , whereas the international protocol requires only 33 patients ( less than the other protocols ) , and can only detect differences between devices of > 24% .
moreover , devices passing a protocol are not necessarily more accurate than devices that failed , and devices that pass may perform substantially worse when tested in a community setting .
in addition , a device that passes validation does not mean that it provides accurate readings in all patients ; inaccuracy is more likely in older and diabetic patients . in general
, we found devices performing badly do so because of wide variation between differences ( large s.d . ) rather than systematic over- or underestimation of bp ( figures 2 , 3 , 4 ) .
this is reflected in the relatively stable mean difference , which leads to a large number of additional measurements for devices relative to only a small drop off in precision ( that is lower proportion of measurements within 5  mm  hg ) .
this effect is negated by multiple readings over multiple clinic visits . yet , as a bp approaches the threshold for diagnosis , this inaccuracy can substantially under- or overestimate the burden of hypertension .
in addition , community clinic - based validations were fewer and the results much worse .
this is particularly of concern , as this setting is where devices are most often used for the assessment and ongoing management of hypertension .
however , it is worth noting that the change from mercury to electronic bp measurement in practice has led to no consistent change in mean bp after their introduction , but there was a large and significant fall in terminal digit preference suggesting improved precision of recording with electronic devices .
this study had a number of limitations : the reproducibility of protocol testing is currently unknown .
we found no studies , which repeated earlier validations . some studies may have been missed .
although the search of 5912 potentially relevant records was comprehensive , with analysis of 3356 potentially relevant records , studies in which machines failed the validation are less likely to be published , particularly if the manufacturer is involved in the funding .
however , missing such studies would have the effect of worsening the results and hence would not alter the conclusions .
all validation studies found used the standard cuff size . a significant proportion of adults will need to use the larger cuff and yet there is no evidence as to whether this will be accurate .
the analysis was limited by the lack of reporting of proportional data in some cases .
development of an international consensus on minimal standards of reporting , including representations of proportional data , would go some way to allowing comparison of devices under different validation protocols .
our estimates of the proportion within 5  mm  hg of the observed value may not be accurate , because they assume a study - specific normal distribution that leads to some degree of inaccuracy , but were sufficient to inform the ordering of the figures .
for example , the aami protocol allows a choice of data collection methods ( simultaneous vs sequential ) and a choice of data - analysis methods , either of which may affect the final results
. problems with validation protocol methods not being detailed enough has earlier been cited as a problem for interpretation .
second , not all protocols directly address the margin of error between measurement and observed value , and those that do allow margins of error as wide as 10 or 15  mm  hg , result in measurements that are up to half the time > 5  mm  hg from the observed value in some monitors that pass protocols .
third , we found few examples of the protocols being tested in community settings ( that is , real clinics rather than artificial settings ) , and the results from these studies were generally poor .
this is of concern because it may suggest that most protocol studies overestimate device quality , although we found too few community studies to draw firm conclusions . finally , the british and american protocols are difficult or costly to obtain , which could discourage further evaluation of bp monitors .
protocols that include measurements for a certain proportion of readings to be within a certain value and have unambiguous procedures could encourage the manufacture of higher quality bp monitors .
monitors need to be re - evaluated in community settings (  real clinics ' ) to verify their accuracy in practice .
the advent of a central , open access repository of evaluation studies would be beneficial to clinics and healthcare providers when choosing digital monitors .
devices may pass even rigorous protocols with as few as 60% of readings within 5  mm  hg of the observed value .
the limited evidence available suggests that performance in clinical practice will be even worse than under protocol testing .
bp standards are essential to provide clinicians and patients with accurate information on which to base diagnostic and treatment decisions .
 the aim of this study is to systematically examine the proportion of accurate readings attained by automatic digital blood pressure ( bp ) devices in published validation studies . 
 we included studies of automatic digital bp devices using recognized protocols . 
 we summarized the data as mean and s.d . 
 of differences between measured and observed bp , and proportion of measurements within 5  mm  hg . 
 we included 79 articles ( 10  783 participants ) reporting 113 studies from 22 different countries . 
 overall , 25/31 ( 81% ) , 37/41 ( 90% ) and 34/35 ( 97% ) devices passed the relevant protocols [ bhs , aami and esh international protocol ( esh - ip ) , respectively ] . for devices that passed the bhs protocol , the proportion of measured values within 5  mm  hg of the observed value ranged from 60 to 86% ( aami protocol 4794% and esh - ip 5489% ) . 
 the results for the same device varied significantly when a different protocol was used ( omron hem-907 80% of readings were within 5  mm  hg using the aami protocol compared with 62% with the esh - ip ) . even devices with a mean difference of 
 zero show high variation : a device with 74% of bp measurements within 5  mm  hg would require six further bp measurements to reduce variation to 95% of readings within 5  mm  hg . current protocols for validating bp monitors give no guarantee of accuracy in clinical practice . 
 devices may pass even the most rigorous protocol with as few as 60% of readings within 5  mm  hg of the observed value . 
 multiple readings are essential to provide clinicians and patients with accurate information on which to base diagnostic and treatment decisions . 
Introduction
Methods
Eligibility and search strategy
Data abstraction
Data analysis
Results
Discussion