screening with mammography has been shown to reduce mortality from breast cancer [ 1 , 2 ] . however , the sensitivity to non - palpable cancer of screening mammography in radiographically dense - breasted women is as low as 3048% .
extremely dense - breasted women have an 18-fold increase in interval cancer found between annual mammograms , compared with fatty - breasted women . magnetic resonance imaging ( mri )
has been demonstrated to be , and recommended as , an efficacious adjunct to mammography for very high - risk , dense - breasted women [ 5 , 6 ] .
three limitations to mri screening for breast cancer are cost , intravenous injection of gadolinium - containing contrast medium , and lower specificity of mri compared with mammography   with increased false positive callbacks and biopsies . for radiographically dense - breasted women , whole - breast ultrasound as an adjunct to screening mammography has shown promise .
berg et al . increased cancer discovery 42% by adding handheld whole - breast ultrasound performed by radiologists .
kelly et al . used an automated whole - breast ultrasound ( awbu ) device capturing a cin loop of 2d breast images .
this blinded study of mostly dense - breasted women showed a 100% increase in cancer detection , and a 200% increase in discovery of invasive cancers 1  cm or less , compared with mammograms alone .
these cin loops were recorded and are available for reader trials similar to those performed for comparison of screening mammography with and without computer - aided detection ( cad ) . for awbu to be a useful adjunct to screening mammography for dense - breasted women , interpretation of examinations must be shown as beneficial , when performed by community radiologists .
this paper evaluates the performance of such radiologists in detection before and after awbu is added to a test set of screening mammograms of radiographically dense - breasted women .
mammograms standard cranio - caudal ( cc ) and medio - lateral oblique ( mlo ) views of each breast were available for all cases . if implants were present , displacement views were included . original analog films ( 66 cases ) or prints of digital films ( 36 cases )
all cases used in the study provided informed consent , and the protocol was approved by the institutional review boards at each hospital , or the western institutional review board .
awbus automated whole - breast ultrasound ( awbu ) is a computer - based system for performing , recording , and reading whole - breast ultrasound examinations similar in appearance to 2d freehand imaging ( sonocine , reno , nv ) .
the transducer is attached to a computer - guided mechanical arm that acquires images in cc rows overlapping 7 to 10  mm insuring complete coverage of both breasts .
the awbu software creates a cin loop for review of approximately 3,000 images , simulating real - time imaging .
the windows-based reading station uses a high - definition 1,600    1,200 monitor and special software to increase cancers conspicuity .
twelve board - certified breast radiologists who use breast ultrasound in their practices were recruited as readers for the trial .
one had reviewed limited awbus 8  years earlier during the developmental phase of the technology .
each reader had a 4-h tutorial with one author ( kk ) explaining the awbu reading station operation .
the readers reviewed and discussed approximately 12 awbus with known cancers , not part of the test set .
they were not in the test set because either palpable findings were present or there were no concurrent mammograms .
nothing concerning the study was discussed , other than the use of the data form ( appendix a ) and the number of cases to be reviewed .
a set of 51 malignant cases ( 3 cases with bilateral cancers ) , including invasive and in situ cancer were collected for the trial ( table  1 ) . screening mammography and awbu
the mammograms were heterogeneously dense or extremely dense breast tissue ( birads 3 or 4 ) on the original reports .
twelve cancers were included that were not prospectively reported on either imaging technique , but are visible in retrospect .
four of these became palpable within 1  year , three in more than 1  year ; five were discovered in a subsequent screening round , three by awbu only , and two by both awbu and mammography . 

table  1pathological diagnosis of 51 positive cases ( 54 cancers)1  cm>1 to 2  cm>2  cmtotaldcis2046idc1719541ilc3216mixed idc and ilc0101total22221054dcis ductal carcinoma in situ , idc invasive ductal carcinoma , ilc invasive lobular carcinoma pathological diagnosis of 51 positive cases ( 54 cancers ) dcis ductal carcinoma in situ , idc invasive ductal carcinoma , ilc invasive lobular carcinoma fifty - one normal cases performed from 2003 to 2008 were matched with each of the positive cases for the following factors : 
 facilitydigital or analog mammogramultrasound machine modelamerican breast cup size ( a  dd)acr birads breast densityimplant ( saline or silicone ) and location ( pre- or retropectoral)breast cancer historyagethe normal case matching factors 1 to 7 closest to the age of the positive case was matched as the normal partner case .
the mean difference in age between the positive case and its matched normal was 31  days .
digital or analog mammogram ultrasound machine model american breast cup size ( a  dd ) acr birads breast density implant ( saline or silicone ) and location ( pre- or retropectoral ) breast cancer history testing occurred on a subsequent date at each reader s own site with only the reader and a research assistant ( monitor ) present .
she had no knowledge of the test set makeup , had no mammography or ultrasound training , reviewed the test data forms in real - time for completeness , and transferred the data to the study database . at each test site
102 mammograms were placed on a film alternator in random order , generated once , and used for all readers . excluding breaks , the test subject s time for review was recorded .
the upper half of a data form ( appendix a ) was completed for each case , checked by the monitor , and entered into the database .
four questions were asked : 
 would you request further evaluation based on this mammogram , or recommend routine screening?where is / are the most suspicious ( up to 3 ) lesions , identifying their location by breast and clock face position?what would be your prediction of the final acr birads after any needed diagnostic workup was completed?what is the reader s confidence level that the woman has or does not have cancer ( dmist likelihood scale ) ?
where is / are the most suspicious ( up to 3 ) lesions , identifying their location by breast and clock face position ?
what would be your prediction of the final acr birads after any needed diagnostic workup was completed ?
what is the reader s confidence level that the woman has or does not have cancer ( dmist likelihood scale ) ? the american college of radiology breast imaging reporting and data system ( birads ) is a seven - point scale ( 0 = incomplete , needs additional assessment ; 1 = normal ; 2 = benign ; 3 = probably benign ; 4a = possible malignancy ; 4b = probable malignancy , or 5 = highly suggestive of malignancy ) designed to categorize the results of mammography and other imaging studies [ 3 , 11 ] .
similar to the dmist , readers were asked to predict a birads score before any diagnostic workup .
the dmist likelihood rating is a seven - point scale to express the confidence of the diagnosis , and ranges from definitely not cancer to definitely cancer [ 3 , 11 , 12 ] .
a correct location response was recorded for an hour position marked within the half of the breast centered at the middle of the cancer . a true positive ( tp )
was recorded for mammography for any malignant case if  callback was marked for mammography and any correct tumor location was identified .
a tp was recorded for mammography plus awbu if  callback was marked on either or both halves of the form in the malignant cases , with at least one correct location identified .
thus , a correctly identified tp found with mammography would remain tp even were it not identified again on awbu .
awbu findings could change the outcome to tp if a cancer was correctly identified with awbu , but missed with mammography .
we evaluated readings on a per - case ( i.e. , per - patient ) basis rather than a per - score basis because screening serves as a  go no - go 
gatekeeper for subsequent workup . a true negative ( tn ) was recorded for mammography for any normal case if  callback was not marked for mammography .
a tn was recorded for mammography plus awbu for any normal case if  callback was not marked on the second half of the form .
this allowed the reader to reverse a callback for an asymmetric density seen mammographically but cleared by the awbu as no suspicion .
to validate tn cases , all cases were followed for at least 1  year or more . a false positive ( fp )
was recorded for mammography in two situations : 
 callback was marked for mammography in a normal case.callback was marked for mammography in a cancer case , but none of the marked locations corresponded to the cancer .
callback was marked for mammography in a cancer case , but none of the marked locations corresponded to the cancer .
an fp was recorded for mammography plus awbu in the same two situations as above when callback was marked for awbu . a false negative ( fn )
was recorded for mammography when callback was not marked in a cancer case in the mammography portion of the form .
similarly , an fn was recorded for mammography plus awbu when callback was not marked in a cancer case in either portion of the form .
the 102 abwus were reviewed by readers on a review station brought by the research assistant acting as a monitor .
they worked approximately 8  h daily for 3  days , with breaks at the readers choosing .
the readers were given the corresponding mammograms with each awbu and completed the second half of the data sheet with the knowledge from the mammogram - only evaluation available .
the same questions were answered for awbu and the reading time of each awbu recorded .
analyses were conducted in a multi - reader multi - case ( mrmc ) framework where each reader screened all cases and each case contained both screening techniques .
the mrmc design efficiently reduces the number of readers and cases needed to detect improvements across techniques .
analyses appropriate for an mrmc design were chosen both to correctly model correlations between readings on the same case across readers and to estimate correctly standard errors .
unless specified otherwise , analyses were conducted in sas software version 9.2 ( sas institute inc .
we present f statistics , shown as f(numerator degrees of freedom , denominator degrees of freedom ) , and p values for comparisons between mammography plus awbu and mammography alone .
cases identified for further imaging were assessed by four binary measures : sensitivity = number of tp / number of cancer cases ; specificity = number of tn / number of non - cancer cases ; positive predictive value ( ppv ) = number of cancer cases/(number of tp + fp cases ) ; and negative predictive value ( npv ) = number of non - cancer cases/(number of fn + tn ) .
random - effect logistic regression models were used to test whether each binary measure differed significantly between mammography plus awbu versus mammography alone . to account for the mrmc framework
accuracy was assessed through birads ratings and dmist likelihood scores , comparing two commonly used indicators of accuracy between mammography plus awbu versus mammography alone : areas under the curve ( auc ) and figures of merit ( fom ) .
the fom incorporates information from each reader on the region of suspected malignancy , as well as their confidence level in the finding , incorporated in an auc . because it includes both confidence level and location accuracy , the fom is more powerful than auc in detecting differences between techniques .
we include both analyses , as described below : areas under the curve ( auc ) were estimated in dbm mrmc 2.1   ( available from http://perception.radiology.uiowa.edu ) using the trapezoidal / wilcoxon method .
we also present reader - averaged receiver operating characteristic ( roc ) curves ; average values were calculated from separate roc analyses conducted on each reader in the proc logistic procedure .
figures of merit ( fom ) were estimated by using jackknife alternative free - response receiver operating characteristic methodology as implemented in jafroc version 1.0   ( available from http://www.devchakraborty.com ) .
the fom is defined as the probability that a cancer on an abnormal image is scored higher than a falsely marked location on a normal image and is analogous to the roc curve ; a higher fom indicates improvement in reader performance . confidence in identification of cases for further imaging we used linear regression , comparing birads ratings and dmist likelihood scores across the two screening techniques among tp cases ; mean ratings and scores are estimated by the regression for each screening technique .
to account for the mrmc framework , we included random effects similar to the dbm model ; the model included a fixed effect for technique , classified as mammography plus awbu or mammography alone , and random effects for readers and cases .
mammograms standard cranio - caudal ( cc ) and medio - lateral oblique ( mlo ) views of each breast were available for all cases . if implants were present , displacement views were included . original analog films ( 66 cases ) or prints of digital films ( 36 cases )
all cases used in the study provided informed consent , and the protocol was approved by the institutional review boards at each hospital , or the western institutional review board .
awbus automated whole - breast ultrasound ( awbu ) is a computer - based system for performing , recording , and reading whole - breast ultrasound examinations similar in appearance to 2d freehand imaging ( sonocine , reno , nv ) .
the transducer is attached to a computer - guided mechanical arm that acquires images in cc rows overlapping 7 to 10  mm insuring complete coverage of both breasts .
the awbu software creates a cin loop for review of approximately 3,000 images , simulating real - time imaging .
the windows-based reading station uses a high - definition 1,600    1,200 monitor and special software to increase cancers conspicuity .
twelve board - certified breast radiologists who use breast ultrasound in their practices were recruited as readers for the trial .
one had reviewed limited awbus 8  years earlier during the developmental phase of the technology .
each reader had a 4-h tutorial with one author ( kk ) explaining the awbu reading station operation .
the readers reviewed and discussed approximately 12 awbus with known cancers , not part of the test set .
they were not in the test set because either palpable findings were present or there were no concurrent mammograms .
nothing concerning the study was discussed , other than the use of the data form ( appendix a ) and the number of cases to be reviewed .
a set of 51 malignant cases ( 3 cases with bilateral cancers ) , including invasive and in situ cancer were collected for the trial ( table  1 ) . screening mammography and awbu
the mammograms were heterogeneously dense or extremely dense breast tissue ( birads 3 or 4 ) on the original reports .
twelve cancers were included that were not prospectively reported on either imaging technique , but are visible in retrospect .
four of these became palpable within 1  year , three in more than 1  year ; five were discovered in a subsequent screening round , three by awbu only , and two by both awbu and mammography . 

table  1pathological diagnosis of 51 positive cases ( 54 cancers)1  cm>1 to 2  cm>2  cmtotaldcis2046idc1719541ilc3216mixed idc and ilc0101total22221054dcis ductal carcinoma in situ , idc invasive ductal carcinoma , ilc invasive lobular carcinoma pathological diagnosis of 51 positive cases ( 54 cancers ) dcis ductal carcinoma in situ , idc invasive ductal carcinoma , ilc invasive lobular carcinoma fifty - one normal cases performed from 2003 to 2008 were matched with each of the positive cases for the following factors : 
 facilitydigital or analog mammogramultrasound machine modelamerican breast cup size ( a  dd)acr birads breast densityimplant ( saline or silicone ) and location ( pre- or retropectoral)breast cancer historyagethe normal case matching factors 1 to 7 closest to the age of the positive case was matched as the normal partner case .
the mean difference in age between the positive case and its matched normal was 31  days .
digital or analog mammogram ultrasound machine model american breast cup size ( a  dd ) acr birads breast density implant ( saline or silicone ) and location ( pre- or retropectoral ) breast cancer history testing occurred on a subsequent date at each reader s own site with only the reader and a research assistant ( monitor ) present .
she had no knowledge of the test set makeup , had no mammography or ultrasound training , reviewed the test data forms in real - time for completeness , and transferred the data to the study database . at each test site
102 mammograms were placed on a film alternator in random order , generated once , and used for all readers . excluding breaks , the test subject s time for review was recorded .
the upper half of a data form ( appendix a ) was completed for each case , checked by the monitor , and entered into the database .
four questions were asked : 
 would you request further evaluation based on this mammogram , or recommend routine screening?where is / are the most suspicious ( up to 3 ) lesions , identifying their location by breast and clock face position?what would be your prediction of the final acr birads after any needed diagnostic workup was completed?what is the reader s confidence level that the woman has or does not have cancer ( dmist likelihood scale ) ?
where is / are the most suspicious ( up to 3 ) lesions , identifying their location by breast and clock face position ? what would be your prediction of the final acr birads after any needed diagnostic workup was completed ? what is the reader s confidence level that the woman has or does not have cancer ( dmist likelihood scale ) ?
the american college of radiology breast imaging reporting and data system ( birads ) is a seven - point scale ( 0 = incomplete , needs additional assessment ; 1 = normal ; 2 = benign ; 3 = probably benign ; 4a = possible malignancy ; 4b = probable malignancy , or 5 = highly suggestive of malignancy ) designed to categorize the results of mammography and other imaging studies [ 3 , 11 ] .
similar to the dmist , readers were asked to predict a birads score before any diagnostic workup .
the dmist likelihood rating is a seven - point scale to express the confidence of the diagnosis , and ranges from definitely not cancer to definitely cancer [ 3 , 11 , 12 ] .
a correct location response was recorded for an hour position marked within the half of the breast centered at the middle of the cancer . a true positive ( tp )
was recorded for mammography for any malignant case if  callback was marked for mammography and any correct tumor location was identified .
a tp was recorded for mammography plus awbu if  callback was marked on either or both halves of the form in the malignant cases , with at least one correct location identified .
thus , a correctly identified tp found with mammography would remain tp even were it not identified again on awbu .
awbu findings could change the outcome to tp if a cancer was correctly identified with awbu , but missed with mammography .
we evaluated readings on a per - case ( i.e. , per - patient ) basis rather than a per - score basis because screening serves as a  go no - go 
gatekeeper for subsequent workup . a true negative ( tn ) was recorded for mammography for any normal case if  callback was not marked for mammography .
a tn was recorded for mammography plus awbu for any normal case if  callback was not marked on the second half of the form .
this allowed the reader to reverse a callback for an asymmetric density seen mammographically but cleared by the awbu as no suspicion .
to validate tn cases , all cases were followed for at least 1  year or more . a false positive ( fp )
was recorded for mammography in two situations : 
 callback was marked for mammography in a normal case.callback was marked for mammography in a cancer case , but none of the marked locations corresponded to the cancer .
callback was marked for mammography in a cancer case , but none of the marked locations corresponded to the cancer .
an fp was recorded for mammography plus awbu in the same two situations as above when callback was marked for awbu . a false negative ( fn )
was recorded for mammography when callback was not marked in a cancer case in the mammography portion of the form .
similarly , an fn was recorded for mammography plus awbu when callback was not marked in a cancer case in either portion of the form .
the 102 abwus were reviewed by readers on a review station brought by the research assistant acting as a monitor .
they worked approximately 8  h daily for 3  days , with breaks at the readers choosing .
the readers were given the corresponding mammograms with each awbu and completed the second half of the data sheet with the knowledge from the mammogram - only evaluation available .
the same questions were answered for awbu and the reading time of each awbu recorded .
analyses were conducted in a multi - reader multi - case ( mrmc ) framework where each reader screened all cases and each case contained both screening techniques .
the mrmc design efficiently reduces the number of readers and cases needed to detect improvements across techniques .
analyses appropriate for an mrmc design were chosen both to correctly model correlations between readings on the same case across readers and to estimate correctly standard errors .
unless specified otherwise , analyses were conducted in sas software version 9.2 ( sas institute inc . ,
we present f statistics , shown as f(numerator degrees of freedom , denominator degrees of freedom ) , and p values for comparisons between mammography plus awbu and mammography alone .
cases identified for further imaging were assessed by four binary measures : sensitivity = number of tp / number of cancer cases ; specificity = number of tn / number of non - cancer cases ; positive predictive value ( ppv ) = number of cancer cases/(number of tp + fp cases ) ; and negative predictive value ( npv ) = number of non - cancer cases/(number of fn + tn ) .
random - effect logistic regression models were used to test whether each binary measure differed significantly between mammography plus awbu versus mammography alone . to account for the mrmc framework
accuracy was assessed through birads ratings and dmist likelihood scores , comparing two commonly used indicators of accuracy between mammography plus awbu versus mammography alone : areas under the curve ( auc ) and figures of merit ( fom ) .
the fom incorporates information from each reader on the region of suspected malignancy , as well as their confidence level in the finding , incorporated in an auc . because it includes both confidence level and location accuracy , the fom is more powerful than auc in detecting differences between techniques .
we include both analyses , as described below : areas under the curve ( auc ) were estimated in dbm mrmc 2.1   ( available from http://perception.radiology.uiowa.edu ) using the trapezoidal / wilcoxon method .
we also present reader - averaged receiver operating characteristic ( roc ) curves ; average values were calculated from separate roc analyses conducted on each reader in the proc logistic procedure .
figures of merit ( fom ) were estimated by using jackknife alternative free - response receiver operating characteristic methodology as implemented in jafroc version 1.0   ( available from http://www.devchakraborty.com ) .
the fom is defined as the probability that a cancer on an abnormal image is scored higher than a falsely marked location on a normal image and is analogous to the roc curve ; a higher fom indicates improvement in reader performance . confidence in identification of cases for further imaging we used linear regression , comparing birads ratings and dmist likelihood scores across the two screening techniques among tp cases ; mean ratings and scores are estimated by the regression for each screening technique . to account for the mrmc framework
, we included random effects similar to the dbm model ; the model included a fixed effect for technique , classified as mammography plus awbu or mammography alone , and random effects for readers and cases .
sample subjects averaged 59.4  years of age ( sd = 10.2 ; range = 4183 ) .
the 51 cancer patients and 51 normal subjects were well - matched with an insignificant mean difference of 31.0  days in age between abnormal and normal cases ( t test  =  1.47 , df  =  50 , p  =  0.15 ) .
identification of cases for further imaging table  2 details individual performance in the identification of cancer cases for further imaging .
mean sensitivity increased from 50% to 81% , an improvement of 63% in the number of cancer cases identified ( 25.4 vs. 41.4 , f(1 , 1,161 )  =  165.95 , p  <  0.001 ) .
specificity ( 6058% ; 30.7 vs. 29.1 , f(1 , 1,161 )  =  1.11 , p  =  0.29 ) , ppv ( mean  =  4767% ; f(1 , 1,297 )  =  0.02 , p  =  0.89 ) , and npv ( mean  =  6575% ; f(1 , 933 )  =  0.61 ,
p  =  0.44 ) did not change significantly with the addition of awbu . 

table  2reader performance categorized by imaging technique ( n  =  102 , 51 positive cases)reader # true positivestrue negativesfalse positivesfalse negativesmm+amm+amm+amm+a12845322127301562284525213330166325443020323115742643202843231385264332302821168632432037431478726412527332418108234035311720271191640432521262211102739343627151412112637354124101714122237373420172314mean # of cases25.441.430.729.329.021.816.99.6% of 51 cases49.8%81.2%60.2%57.5%56.9%42.7%33.1%18.8%mean # of added cases16.01.47.27.3mean % of 51 cases added31.4%2.7%14.1%14.3%% improvement compared with m alone63%4%25%43%m mammography , m+a mammography plus automated whole - breast ultrasound ( awbu)reader # presented by best to worst performance based on sensitivity on m+aindividual success varied from 11 to 24 more cancer cases detected by awbu . as a percentage of the cancers detected with mammography the range in improvement was 42150% .
not only did all readers find more cancers individually , but all found 1629% more cancers than the best mammography reader did with mammography alone .
for the best performing mammography reader the cancer detections added by awbu was predictably lower , as more cancers had already been identified with mammography .
for the poorest performer on mammography , the addition of awbu resulted in a 150% improvement , bringing his overall cancer detection rate near the average for the group.table  3 shows the average reader performance by tumor size for the 45 image sets of patients with invasive cancer .
this is due largely to the relatively poor performance at detecting these cancers with mammography , where only 26% of cases were correctly identified . by adding awbu ,
table  3reader performance with 45 invasive cases1  cm>1 to 2  cm>2  cmtotal#%#%#%#% # of cancers1710022100610045100mean cancers by mammography4.42613.5613.05020.946mean additional cancers by awbu6.7396.6302.03315.334mean total cases detected11.16520.1915.08336.280% improvement compared to mammography alone151%49%67%73%for cases with more than one invasive tumor , the larger of the two was used . for interval cancers after imaging , size is the greatest diameter of the tumor seen retrospectively on the awbu or mammogram , otherwise the diameter is that reported by pathological diagnosis reader performance categorized by imaging technique ( n  =  102 , 51 positive cases ) m mammography , m+a mammography plus automated whole - breast ultrasound ( awbu ) reader # presented by best to worst performance based on sensitivity on m+a reader performance with 45 invasive cases for cases with more than one invasive tumor , the larger of the two was used . for interval cancers after imaging , size is the greatest diameter of the tumor seen retrospectively on the awbu or mammogram , otherwise the diameter is that reported by pathological diagnosis accuracy the roc area was greater for mammography plus awbu for both birads ( 0.808 versus 0.701 ; f(1 , 123 ) = 14.79 , p  <  0.001 ) and likelihood scores ( 0.810 versus 0.703 ; f(1 , 85 ) = 17.88 , p  <  0.001 ) as estimated by multi - reader multi - case analyses .
this is highlighted in fig .  1 by roc curves that are generated by averaging the results of separate roc analyses for each reader
. the birads and likelihood auc curves for mammography and mammography plus awbu in both cases almost superimpose when confidence in malignancy by mammography is high , but when confidence in malignancy by mammography is low , as in the lower portions of the graphs , the curves in both cases diverge significantly . in both cases the mammography plus awbu approaches the y - axis indicating better cancer recognition . 

1receiver operating characteristic curves averaged across 12 readers for mammography alone ( circles and dashed line ) and mammography plus awbu ( triangles and solid line)figure  2 shows the areas under the roc curves for each reader and for the average of all readers as estimated by multi - reader multi - case analyses .
these individual line graphs mirror the improvement in reader performance shown in table  2 . 

2changes in areas under the receiver operating characteristic curve(s ) for each reader ( hollow circles ) and averaged across 12 readers ( solid circles)similar to roc areas , the figures of merit ( fom ) were higher for mammography plus awbu across all readers , compared with mammography alone using both the birads scores ( 0.786 versus 0.613 ; f(1 , 270 )  =  34.1 , p  <  0.001 ) and dmist likelihood scores ( 0.791 versus 0.614 ; f(1 , 238 ) = 37.9 , p 
receiver operating characteristic curves averaged across 12 readers for mammography alone ( circles and dashed line ) and mammography plus awbu ( triangles and solid line ) changes in areas under the receiver operating characteristic curve(s ) for each reader ( hollow circles ) and averaged across 12 readers ( solid circles ) confidence in identification of cases for further imaging readers reviewing cancer cases were more confident in correctly identifying cases for further imaging , i.e. , tp reading , using mammography plus awbu compared with mammography alone . on average , both birads scores ( mean = 4.8 versus 4.2 , f(1 , 740 ) = 81.91 , p  <  0.001 ) and dmist likelihood scores ( mean = 4.8 versus 4.1 , f(1 , 740 ) = 82.21 , p  < 
interpretation times average reading time per study for the 102 awbus was 7  min 58  s ( 7:58 ) varying from 5:54 to 12:51 .
the difference in review time was unrelated to the number of cancers identified by each reader ( correlation = 0.02 , p  =  0.96 ) .
this is shown by a 63% increase in callbacks of cancer cases with only a 4% decrease in correct identification of the true negative cases .
the confidence of the diagnoses of the 102 cases with predictive birads and dmist likelihood scales was confirmed by using auc and fom methodology . with a short training period experienced radiologists using 2d awbu significantly improve their ability to diagnose cancer in dense - breasted women .
the slower transducer speed enforced by the awbu decreases inter - image distance , allowing the reader more time to identify small masses . at a review speed of 10  images per second
the observer has 0.5  s to identify a 5-mm mass . a high - resolution computer screen , along with a post - processing technique to expand the grayscale at the black end of the spectrum , results in visually sharper margins and more contrast of masses against the background tissue .
this automated process for breast ultrasound eliminates operator variability , provides greater consistency , and ensures reproducibility of quality images .
study radiologists increased discovery of t1a and t1b invasive cancers 150% over mammography alone ( table  3 ) . the average review time per
awbu study was about 11  min shorter than the 19  min for radiologists in the acrin 6666 trial of handheld screening ultrasound .
as half of our test set subjects had cancers , it would be expected that the average review time we observed for awbu would be significantly longer than in a typical screening population with mostly normal studies .
although the test set was confidential , the readers probably quickly realized that it was enriched .
a false increase in tps would occur if all the correctly identified cancers were not subsequently confirmed with biopsy .
also , analysis was performed on a case basis in the three patients in whom cancers were present in both breasts ; it was assumed if one of the cancers was identified , the cancer in the other breast would be found by the subsequent workup .
any of the following factors could have decreased the readers accuracy with awbu ( decreased true positives and negatives , and increased false positives and negatives ) compared with a normal screening situation . 

some readers do not perform screening ultrasound.limited experience with awbu  this was the first exposure to awbu for 11 of the readers.unfamiliarity with some ultrasound formats  images from many different manufacturers were used .
limited experience with awbu  this was the first exposure to awbu for 11 of the readers .
unfamiliarity with some ultrasound formats  images from many different manufacturers were used . in spite of these hindrances
our observations clearly show that radiologists improve detection of cancers , especially small invasive ones , by adding awbu to mammography findings .
radiologists will significantly improve their cancer detection rates in dense - breasted women by adding awbu to mammography .
this procedure has the potential for both standardizing the performance of whole - breast ultrasound and shortening the time required for radiologists .
 objectiveradiologist reader performance for breast cancer detection using mammography plus automated whole - breast ultrasound ( awbu ) was compared with mammography alone.methodsscreenings for non - palpable breast malignancies in women with radiographically dense breasts with contemporaneous mammograms and awbu were reviewed by 12 radiologists blinded to the diagnoses ; half the studies were abnormal . 
 readers first reviewed the 102 mammograms . 
 the american college of radiology ( acr ) breast imaging reporting and data system ( birads ) and digital mammographic imaging screening trial ( dmist ) likelihood ratings were recorded with location information for identified abnormalities . 
 readers then reviewed the mammograms and awbu with knowledge of previous mammogram - only evaluation . 
 we compared reader performance across screening techniques using absolute callback , areas under the curve ( auc ) , and figure of merit ( fom).resultstrue positivity of cancer detection increased 63% , with only a 4% decrease in true negativity . 
 reader - averaged auc was higher for mammography plus awbu compared with mammography alone by birads ( 0.808 versus 0.701 ) and likelihood scores ( 0.810 versus 0.703 ) . 
 similarly , fom was higher for mammography plus awbu compared with mammography alone by birads ( 0.786 versus 0.613 ) and likelihood scores ( 0.791 versus 0.614).conclusionadding awbu to mammography improved callback rates , accuracy of breast cancer detection , and confidence in callbacks for dense - breasted women . 
Introduction
Materials and methods
Imaging studies
Readers
Procedure
Statistical analysis
Results
Discussion
Conclusion
Appendix A