atrial fibrillation is the most common heart arrhythmia affecting around 2.2 million people in the usa .
depending on the ablation device used , it is carried out using either electroanatomic mapping systems , an x - ray guided approach , or a combination of both .
x - ray guidance is , for example , required if a cryoballoon is used for ablation as current cryoballoons can not be directly located by electroanatomic mapping systems . in this paper , we focus on x - ray guided approaches .
one weakness of x - ray imaging is poor soft - tissue contrast . as a consequence , the left atrium ( la )
however , to reduce the risk of contrast - induced nephropathy , physicians try to keep the use of ca to a minimum often highlighting only a part of the left atrium . to provide orientation to the physician when
no ca is present , a model of the patient 's la , for example , generated by a preoperative ct or mri scan of the patient can be overlaid .
the use of such an overlay was found to reduce procedure time and fluoroscopy time significantly . since a preprocedural 3d scan
is often acquired to obtain prior knowledge about a patient 's anatomy of the left atrium and to rule out unusual pulmonary vein configurations , it is readily available for augmented fluoroscopy applications . in general , however , the coordinate systems of the preprocedurally acquired 3d heart model and the patient during the intervention differ , and a registration step is usually needed . today , this registration is usually carried out manually .
if the 3d heart model was acquired using ct , several authors proposed to use further landmarks in addition to the la for registration having obtained associated 3d models by segmentation .
examples are the carina , the coronary sinus ( cs ) [ 4 , 5 ] , the spine , or a combination of spine and heart anatomy . for 3d volumes acquired by mri , for example , mri angiographies ( mras ) , however
, a segmentation of the cs or the carina can be very challenging or has not yet been adopted for routine clinical procedures .
fortunately , registration based on contrast injection has been shown to be a fast and accurate alternative . a good moment to obtain contrast - enhanced x - ray images is after the transseptal puncture , in particular if physicians use contrast injections to verify puncture success .
example images of such injections are shown in figure 1 to illustrate the images available .
since manual registration either needs the attention of the treating physician or requires a trained assistant , more automatic registration methods are desirable . 

there is a significant body of research on registration of 3d objects to 2d fluoroscopic images , for example , for bones [ 1012 ] , implants , joints , or vessels .
, a registration of the la is more complicated for at least two reasons : first , for implants and bones , all parts of the object are visible . during a contrast injection , however , only a part of the left atrium may appear under x - ray .
second , depending on the amount of ca injected , the overall la visibility may be poor .
for example , in our case ep physicians use contrast at the beginning of the procedure to verify the success of the transseptal puncture .
it may also be used later to enhance the anatomy , for example , to make sure that a catheter , for example , a circumferential mapping catheter or a cryoballoon catheter , was placed correctly .
this is different to vessel angiography . in this case , higher amounts of contrast are injected to derive diagnostic information , for example , about a stenosis . for ablation procedures in the la , the ca density and
so the visibility of the la under x - ray may be poorer as the small amount of contrast is injected into the high blood volume of the la . as a consequence , further effort is needed to develop robust registration methods that can also be applied if ca is used sparingly
. currently , very few publications deal with registration based on contrast agent : in a first approach towards automatic la registration , thivierge - gaulin et al .   tried to find a 3d pose of a model such that its projected shadow matches the contrasted area in a selected image , enhanced by digital subtraction angiography ( dsa ) , best .
however , if only a small amount of contrast agent is injected into a somewhat large chamber such as the left atrium , this approach will not lead to a distinct optimum , because the anatomy may not be fully opacified . based on ct images , a second approach by zhao et al .   relied on digitally rendered radiographies ( drrs ) of the segmented left atrium .
this approach uses a weighting scheme that puts the focus on the roof of the la .
how well this method performs for injections into other areas of the la is still unclear .
besides contrast - based registration , 3d data and 2d data can also be aligned using devices .
a feature - based method by sra et al .   uses a segmentation of the coronary sinus ( cs ) in a ct volume to register a 3d la model to a single 2d fluoroscopic image . a further approach by brost et al .  
uses a segmentation of the cs in an mri volume and a 3d reconstruction of the cs catheter from two fluoroscopic images .
unfortunately , how well the cs can be extracted from a 3d mri data set depends very much on the mri scan protocol . furthermore , due to the strong motion of the cs , it is difficult to relate the position of the cs catheter to the position of the la , especially for patients having no sinus rhythm but heart arrhythmia [ 18 , 19 ] . 

( 2 ) contributions and outline of this work . at the beginning of section 2 ,
we discuss the use of the normalized cross - correlation for registration of a 3d object to 2d x - ray projections based on extracted contrast agent .
they require a segmented left atrium as input and can , in contrast to , accommodate both ct and mri data .
the first method is described in section 2.2 . here , we take explicitly apparent edges extracted from a 3d model and compare them to la edges present in the fluoroscopic images as proposed by [ 10 , 11 ] for automatic registration of bones and by   for manual registration of the la , respectively . although our method is conceptually similar to a drr generation followed by gradient correlation , this calculation can be carried out on a gpu much more quickly than a drr .
the second contribution described in section 2.3 is the introduction of a novel similarity measure for biplane x - ray that is tailored to cases in which only parts of an object are visible .
based on a 3d model of the la , our second method estimates the contrast agent distribution inside the 3d object from a simultaneously acquired pair of fluoroscopic images taken under two different view angles .
then we evaluate how consistent the contrast agent distribution estimate ( cade ) is with the acquired fluoroscopic images .
as the cade depends on the transformation used for registration , the transformation leading to the most plausible cade is used as final position estimate .
we propose to treat the registration results for each frame of the sequence no longer as independent . based on our previous publication ,
we use a markov chain approach to exploit the temporal dependency between successive frames instead . in section 3
, we evaluate a similarity measure based on the projected shadow which is close to the approach by thivierge - gaulin et al .   and the two new methods as well as combinations of them .
these sequences are acquired simultaneously from two different angles using a biplane system . for each plane , the projection matrix that describes the x - ray camera setup
we denote the associated projection operator by pa and pb for the a - plane and the b - plane of the system , respectively .
we also assume that a 3d model of the patient 's la is available , either as a triangle mesh or as a binary volume , as they can be converted into each other .
the contrasted area is found based on a difference image ( dsa ) idsa = iu  ic , i   involving a frame ic that contains contrast agent and an uncontrasted frame iu . to distinguish between contrasted and uncontrasted frames , either manual annotation or a learning based method
can be used , for example , the method described in . depending on the chosen contrasted frame ic
, idsa may contain artifacts , for example , due to motion of the diaphragm or from catheters , if they are at different positions in iu and ic .
such motion artifacts depend , unlike the information about contrast agent , to a large degree on the choice of iu .
for example , if the catheters in iu are at the same position as in ic , their intensities cancel out .
otherwise , idsa has high positive values at the position of the catheter in ic and high negative values at the position of the catheter in iu . to keep motion artifacts to a minimum
, we propose a best reference selection , which chooses an appropriate reference frame iu^ that matches the chosen contrasted frame ic as much as possible . out of all uncontrasted frames , that frame iu^ is selected which minimizes the l1-norm of the resulting dsa image:(1)iu^=arg  miniux=0n  y=0miux , yicx , y.by following ( 1 )
, we get frames for which the catheters and the diaphragm cancel out as much as possible
. see figure 3(b ) for an example . in idsa , only pixels with positive values contain contrast agent . to extract them
afterwards , we compute a filtered image if by applying a median filter with a large kernel size . smaller structures , for example , caused by motion artifacts that remained despite the optimized choice of iu , do not pass this filter , and the noise in the contrasted area is reduced as well . finally , a binary image ithr of the filtered image if is computed using a threshold at f + f , where f and f  denote the mean and standard deviation of if , respectively .
thus , a contrasted pixel p   is indicated by ithr(p ) = 1 .
a previous approach   tried to find a transformation t of the 3d model such that the projected shadows st , st of the model into the a - plane and the b - plane of a biplane c - arm system fit best to the contrasted region . using the normalized cross - correlation ( ncc ) , denoted as n , of two images i1 , i2 with corresponding mean values 1 , 2 and standard deviations 1 , 2(2)ni1,i2=x=0n  y=0mi1x , y1i2x , y212,the similarity of the projected shadow and idsa can be measured . instead of idsa , one can also use the binary version of it , that is , ithr .
then a registration transformation can be estimated by maximizing either one of the two functions(3)shaddsaidsaa , idsab , t=nidsaa , stanidsab , stb,shadthrithra , ithrb , t=nithra , stanithrb , stb .
unfortunately , a registration approach only based on contrasted area has multiple solutions if the amount of ca is so little that it does not completely fill the la ; see figure 2 .
this results in perceivable edges of the contrasted area which can then be used as registration features .
edge - based registration can be carried out using only the silhouette boundary of the projected object   ( see figure 2 ) or all apparent edges [ 10 , 11 ] ( see figure 3(f ) ) .
we decided to consider all apparent edges to improve robustness against injections of small amounts of ca , as the silhouette - based approach requires that the la is contrasted in its entirety . for a partially contrasted left atrium , internal contours may , however , also appear in the fluoroscopic images .
this was already found to be beneficial for manual la registration . instead of considering edges implicitly by comparing the dsa image to a drr using gradient correlation , we computed them explicitly . to extract edges in the fluoroscopic images , we used the filtered image if . after applying a median filter
they , however , correspond rarely to anatomical structures and would trigger a response , if an edge filter was applied . to obtain an edge response only at the boundaries of the contrasted area , the image needs to be homogenized before applying an edge filter .
using a simple threshold method would result in a loss of the intensity drop - off at the boundary which provides important information about the edge intensity .
therefore , we weigh all image pixels by a sigmoid function(4)isigx , y=11+eifx , y+ts.the value of t is set to f  f , and the parameter s depends on the pixel intensity range of the input image .
finally , isig is filtered using a derivative of gaussian ( dog ) filter to obtain the edge image idog .
the kernel size of the dog filter is set to a large value to get a smooth similarity measure ; see figure 3(e ) .
the projection of the 3d triangle mesh edges into 2d is done differently than in [ 10 , 20 ] .
we rendered the whole surface mesh and , depending on the viewing direction d and the surface normal n at a point , we set the opacity o of projected triangles to o = 1  |(dn)| ; see figure 3(f ) for an example . by doing so
, areas that are parallel to the imaging plane are rendered transparent while areas with a normal vector orthogonal to the viewing direction are rendered opaque .
the similarity between edges extracted from the fluoroscopic images and the edge images et , et rendered from the 3d model transformed by t is measured by(5)edgeidoga , idogb , t=nidoga , etanidogb , etb .
previous approaches [ 16 , 17 ] for la registration searched for a rigid transformation of the la such that either its projected shadow or its drr fit to the contrasted area in both fluoroscopic images .
3d information was taken into account insofar as the resulting projections came from the same 3d position of the model .
however , such an approach does not necessarily guarantee that corresponding objects in both fluoroscopic images are matched to the same 3d structure of the la .
more precisely , the registration result could be such that , in plane a , the contrast agent is located in a left pulmonary vein ( pv ) whereas , in plane b , the contrasted area corresponds to a right pv .
this is possible as for a given 2d registration in one plane the 2d registration in the other plane has one degree of freedom , which corresponds to an out - of - plane motion in the first plane .
an illustration of this problem can be found in figure 4 . to solve this problem , we compute for a given transformation t a cade inside the la using binary reconstruction . then , t is optimized such that the contrast agent distribution estimate is most consistent with the projection images .
more precisely , a voxel v is estimated as contrasted if it satisfies all of the following conditions : ( a ) the voxel v transformed by t is projected on a contrasted pixel in plane a , ( b ) v is projected on a contrasted pixel in plane b , and ( c ) v is part of the left atrium ( as contrast agent can only be found inside the la ) . to compute the cade
, we define the indicator function(6)v=1vr3  is  inside  the  left  atrium.given the binary images ithr and ithr with corresponding projection operators pa , pb and the indicator function (v ) , the cade ct for a transformed voxel , t(v ) , can be computed as(7)ct3dv = ithrapatvithrbpbtvvfor a given rigid transformation t. this product is the mathematical equivalent of the three conditions introduced above . if t is chosen suboptimally , the resulting 3d cade will be inconsistent with the ca observed in the 2d images .
that is , a pixel in the 2d image is contrasted but no corresponding voxel along its projection ray is estimated as contrasted .
this can be due to the following reasons as shown in figure 5 : ( a ) the projection ray from a contrasted pixel does not intersect the left atrium as the la has not been placed at the proper position yet ; ( b ) the projection ray hits the la , but all voxels intersected by this ray can not contain ca because their corresponding pixels in the other plane are uncontrasted .
additional inconsistencies are introduced by pixels which are erroneously labeled as contrasted , for example , due to motion artifacts . to verify the validity of the cade
, we compute binary 2d images ct , ct by forward projecting all contrasted voxels in ct using pa , pb . we assess the consistency of the cade for the given transformation t by computing the similarity between the fluoroscopic images and the projected cade by(8)cadeithra , ithrb , t=nithra , ctanithrb , ctb .
the set of potentially contrasted voxels is defined by the intersection of the projection ray bundles of all contrasted pixels from views a and b , respectively . maximizing the number of contrasted voxels would lead to a transformation such that as much as possible of this set is contained inside the la . if , for example , a pulmonary vein was filled with ca , the transformation would be chosen such that the set of possible contrasted pixels was located in the main body of the la which provides the most space to include most possibly contrasted voxels ; see figure 6 . to avoid this registration bias towards large structures inside the la , we decided to optimize the consistency to the 2d images . to exploit dependencies between successive frames of a sequence , we model the position of the la model in 3d as a time - dependent continuous markov chain of the first order as proposed by us previously .
the states are transformations , that is , a translation and a rotation of the model .
the parameter i denotes the actual transformation of the la in the ith contrasted frame , ti refers to the estimate of the transformation for the ith frame , and p(i = ti ) is the probability that for frame i the transformation ti is observed . for convenience
the transition probability from one state into another is independent of the frame number i. it is denoted as p(ti  ti+1 ) .
finally , a sequence of transformations t1 ,  , tn is to be determined such that the term(9)pt1,  ,tn = pt1t=2nptt1ttpttis maximized .
transition probabilities and state probabilities control the filtering differently : due to the transition probabilities , small motions of the la from a frame to its next frame are preferred .
the state probability determines the impact of the averaging , that is , how much the registration result is changed by the filtering .
we will model the state probability based on a confidence measure such that frames with high - confidence registration results are less subject to temporal filtering .
if the confidence value is low , that is , the estimated error of the registration result is higher , we want to rely more on the results of the previous and next frame .
the contrast agent visible in frame i determines the probability of the la for being transformed by ti in this frame . if the dependency on other frames is ignored , the most probable transformation ti is the transformation obtained by optimizing the similarity measures  defined in ( 3 ) , ( 5 ) , and ( 8) or a combination of them .
the similarity measure (i , i , ti ) depends on the contents of images i , i. zhao et al . suggested using the value (i , i , ti ) as a confidence measure : from all n registration results the frame i where (i , i , ti ) is maximum should be selected as registration result for the complete sequence .
the value of  can therefore be used as a confidence measure . by linear regression , a function e((i , i , ti ) ) can be determined to estimate the error based on the value of . with the transformation ti of the la found during optimization for frame i , the probability p(ti ) can be modelled as normal distribution(10)pti = nti;ti,i.the confidence range is incorporated by setting the covariance matrix i to 1  e((i , i , ti ) ) .
the transition probability p(tt1  tt ) states how likely a movement of the la is from frame t  1 to t. over multiple breathing cycles , the la moves about a mean position .
the larger the change in translation and rotation is , the less likely the transformation transition is . to account for different frame rates ,
we consider the translational and rotational velocity v   which comprises three translational velocities and three rotational velocities . in a training step ,
the covariance matrix v of the velocities gives an estimate for how likely a transition from one transformation to the next is .
we model the probability of a transition tt1  tt as a normal distribution(11)ptt1tt = ntttt1r;0,v.besides the frame rate r , the transition probability depends also on the current breathing phase . compared to breathing motion , cardiac motion can be neglected as it rather deforms the la . if information on the breathing phase can be estimated , superior motion should be , for example , more likely during the inhale phase . during exhalation phase
, inferior motion should have a higher probability . to account for breathing motion , for example
, the mean value and the covariance matrix could be estimated separately for each stage of the breathing cycle .
the most likely state sequence,(12)t1,  ,tn=arg  maxti,  ,tn  pti,  ,tn , can be found using a log - likelihood method : by applying the logarithm to ( 9 ) , we get ( 13)t1,  ,tn=arg  maxt1tn12t1t1t11t1t1+i=2ntititi1titi+rtitxi1tv1titi1r.this convex optimization problem can be solved using the bfgs method .
we retrospectively evaluated our method on 21 clinical biplane x - ray sequences from 10 different patients .
all of the patients provided their informed consent for the analysis of their clinical data .
for all patients , a segmentation of their left atrium from a preoperatively acquired mri scan was available as a triangle mesh .
the data set comprised 11 sequences showing an initial contrast agent injection where 15  ml ca was injected into the la center through a sheath to verify the success of the transseptal puncture .
there were 10 more sequences showing subsequent injections to verify catheter placement . in these cases ,
all x - ray angiography sequences were acquired during normal breathing using a standard acquisition protocol .
the first contrasted frame was determined manually . for 129 frames , reference registrations performed by three clinical experts
were available ; for the remaining four frames registrations performed by two clinical experts were available .
reference 3d registrations were established by manually shifting the mesh in each of two orthogonal views .
the resulting 3d translation was refined this way until a good match to the associated contrast distributions as seen in the two 2d x - ray projections had been found .
these reference registrations covered only 3d translation for the following reasons : first , patients were positioned head first , supine , during both preoperative and intraoperative imaging .
furthermore , given the small amounts of contrast injected in our cases , the remaining small rotations were very difficult to detect .
as initialization for optimization , the 3d model was placed at that 3d position which corresponded to the centers of both 2d images . in some cases ,
the initialization was more than 30  mm away from the correct solution and beyond the capture range for gradient - based methods .
therefore we applied an octree - like coarse - to - fine scheme where we evaluated several positions at a coarse resolution . at positions in space that yielded a good similarity value , we performed subsequent evaluations at an increasingly finer resolution .
the 3d translation t^=arg  maxt(t ) found by the optimization process of the respective objective function  was compared to the mean translation vector t of the three manual registration results .
the significance of the results was measured using a wilcoxon signed - rank test   and a significance level of p = 0.05 .
all sequences contained 12-bit images of size 1024  1024 pixels ; all image processing , including rendering from the 3d model , was performed on the full image size .
the kernel size for median filter was 30 pixels ; the value s of ( 4 ) was 0.1 .
in the evaluation , we compared the similarity measures shad , shad , cade , and edge and the combined similarity measures shad + edge , shad + edge , and cade + edge .
we investigated different weightings . giving both terms equal weights , that is , setting  = 1 , turned out to be a good choice .
we did two types of evaluation : an evaluation on all frames and an evaluation using only a single frame of each sequence , namely , the one which provides the best similarity measure .
we computed a registration for each contrasted frame and compared the result to the manual registration of the physicians .
these results are clinically relevant if the physician requires a registration for a desired frame of his choice , for example , depending on the breathing phase . as potentially every frame
the overall accuracy is also of importance if the results are postprocessed by temporal filtering .
the overall interuser variability observed in the manual registrations was 3.2  2.3  mm . considering all sequences ,
cade performed significantly better than shad , shad , and the state - of - the - art method by thivierge - gaulin et al . .
although edge gave significantly worse results than all other measures , a combination with edge could improve the results of shad and shad significantly . for subsequent injections , cade
+ edge performed significantly better than shad , shad , their respective combinations with edge , and the method by thivierge - gaulin et al . .
for shad + edge and cade + edge , the error distribution for each sequence is shown in figure 8 , first for all initial injections and then for subsequent injections . in figure 9 , the error distribution is given for each frame number as counted after the ca injection .
figure 8 indicates that many sequences have at least one frame with a registration error of less than 5  mm .
in fact , there exists such a frame for over 75% of all sequences when using shad or shad and for over 85% of the sequences when using cade , shad + edge , shad + edge , or cade + edge .
proposed to automatically find a good frame in each sequence and consider only these single good frames for evaluation .
in other words , from all frames of a sequence , a single frame was automatically chosen for registration . from a clinical point of view
, this would , however , only make sense if the physician accepted an automatically selected frame for registration or if further motion compensation steps for the other frames were done , for example , using device tracking .
suggested selecting this single frame as follows : for each frame i  [ 1 , m ] out of the m contrasted frames , the transform ti that maximizes the similarity measure  for this frame is computed . in a second step ,
going through all frames of a sequence , the frame with the highest overall similarity measure is picked .
the error to the average manual registration result ti is then computed as(14)t^iti2with  t^i = arg  maxi  ia , i , ib , i , ti ,  ti = arg  maxt  ia , i , ib , i , t . the evaluation results are presented in table 2 .
recalling section 3.1 , where we found that the cade method yielded good results for all frames , the cade performance for a single frame was , however , not that good .
this is why we investigated if a different single - frame selection strategy would lead to better results .
instead of using cade both for translation estimation and for best - frame selection , we used cade only for estimating the translation ti for each frame i. among the obtained translations ti , we selected the frame i with the corresponding translation t^i such that shad was maximized .
so ( 14 ) was modified to(15)t^iti2with  t^i = arg  maxi  shadthrithra , i , ithrb , i , ti ,  ti = arg  maxt  cadeithra , i , ithrb , i , t.the results of this frame selection method are denoted by cadeshad .
although only 10 sequences were available for initial contrast injections , shad , shad + edge , and ( cade + edge)(shad + edge ) gave significant better results when compared to shad . also the performance of edge
our proposed method cadeshad and the corresponding combination with edge outperformed the method by thivierge - gaulin et al .
the error estimate function e(( ) ) and the transition probability covariance matrix v were trained in a leave - one - patient - out cross - validation . the results for the temporal filtered frames are given in table 3 .
for a combination of the cade and edge similarity measure , markov filtering reduced the mean error to 5.7  4.6  mm for initial injections and 6.6  3.6  mm for subsequent injections .
the results obtained by the markov filtering were in all cases significantly better than the results without filtering .
the image preprocessing took 0.5  s on an intel xeon e3 with 3.4  ghz and 16  gb  ram .
the evaluations of the similarity measures were performed completely on the gpu . on an nvidia geforce gtx 660
the evaluation of shad , shad , or edge took 1.8  ms for a given translation and 13.4  3.7  ms for cade .
the whole registration for a single frame took 2.9  s for shad or shad and 21.5  5.9  s for cade .
for a combination with edge it took 5.8  s and 27.1  5.9  s , respectively .
our novel cade - based method outperformed the shadow based similarity measures shad and shad , especially for reregistration sequences where only a small amount of contrast agent was used .
the problem with nondistinct optima for shadow based similarity measures which was already sketched in figure 2 becomes also apparent in the sections of the optimization function in figure 10 for shad : for a small amount of contrast agent like in figure 10(a ) , the optimum is not at the position of the ground truth , but about 15  mm off . also for secondary injections into small structures such as the pulmonary veins ,
the objective function for shad in figure 10(c ) has a large plateau . in both cases , cade is more distinct , although clear extrema as for bones   are not obtained . especially for secondary injections where ca was often injected into pulmonary veins , registration errors leading to inconsistent results
the improvement by cade was less as inconsistencies played only a minor role and the shape of the contrasted region in the image was more similar to the projected shadow of the left atrium .
this becomes also apparent in figure 10(b ) where the shapes of the objective function of shad and cade look very similar .
we found that the similarity measure using explicit apparent edges , edge , yielded poor results when used on its own .
a possible reason is that the objective function of edge has several local optima and also the global optimum does not necessarily correspond to the ground - truth position .
it often has a more distinct local optimum at the position of the ground truth that facilitates fine registration .
this is important as the other similarity measures define a rather plateau - shaped optimal region . 

figure 9 shows that the accuracy of the registration depends on when the frame was recorded after contrast agent injection .
this was in many cases the frame before the ejection into the ventricle ; that is , it contains the most contrast agent . especially for the first frames , where only little ca was present , registration accuracy was lower
the first frame is an exception , as it has a lower mean error than the second frame .
this is probably because the previous frame , that is , the last uncontrasted frame , is used as mask frame for dsa computation which leads to low motion artifacts and a better extraction of the contrasted region .
we found that , for our registration to work best , a well opacified frame should be selected from the sequence .
this strategy was evaluated in the context of the single - frame evaluation by selecting the frame which had the best objective function value . for methods like shad and shad , which are based on the projection shadow ,
the measure cade , however , is not related to the amount of contrast agent , but it depends on consistency . when relying on cade for best - frame selection , we get the frame yielding the most consistent registration .
this is , however , not necessarily the frame with the most contrast agent . and
it is usually the frame with the most contrast agent which provides the most information for registration .
we believe that this is the reason why the errors in table 2 for cade are higher . if , however , from the translations estimated by cade + edge , the single frame was selected based on shad + edge , better results were achieved as now again the frames with the most contrast agent were selected .
sometimes , the automatic best - frame selection does not provide a frame with a satisfactory result .
still , 85% of all sequences contain at least one frame that is below a clinical relevant threshold of 5  mm [ 27 , 28 ] .
to benefit from the fact that at least one frame with a good registration result is likely to be found , the frame selection could be performed manually by stepping through the frames and the associated registration results .
although this implementation still involves user input , the required user interaction is less than for fully manual registration . for the remaining 15% of the cases or
if higher accuracy level , for example , 3  mm , is required , small manual adjustment may be necessary in these cases .
particularly high errors for frames at the beginning or at the end of the contrast injection were reduced .
if a registration is not desired for an arbitrary frame but rather for a frame determined by the physician , for example , based on the breathing phase , temporal filtering should be applied .
it is notable that the runtime for cade was considerably higher compared to , for example , shad .
this is because the contrast agent distribution needs to be updated in each iteration before projecting it .
although the fast runtime of , for example , shad will remain out of reach , we are confident that the runtime for cade could be reduced to a clinically acceptable level . in general , the performance for initial registration was better , as the amount of contrast agent was higher and fewer catheter artifacts were present .
we found by visual inspection that also the amount of breathing motion had an impact on the registration accuracy as it caused motion artifacts in the dsa computation .
as a consequence , if the patient was not anesthetized , acquisition of the ca injection should be performed under breath - hold or shallow breathing . for cases with intubation , also jet ventilation   or a small period of apnea could be applied to reduce breathing motion . since
our data was not acquired using a dedicated dsa program , the different brightness levels within a sequence changed .
although intuitively appealing , our data did not allow us to make a statement if the registration accuracy depends on the part of the la in which ca was injected . for initial contrast injections , an average accuracy of 4.6  4.0  mm and a median error of 3.7  mm could be achieved using cade + edge together with the best - frame selection approach .
this error is in the range of the interuser variability of 3.2  2.3  mm .
the faster registration method using shad + edge reached an accuracy of 4.8  4.6  mm .
these numbers are also similar to the accuracy reported when performing manual registration based on segmentations of the cs , the whole heart , and the spine   or a segmentation of the cs when 3d and 2d data are in the same breathing and cardiac phase   which was achieved by ventilation and rapid pacing of anesthetized patients . however
, our method does not require structures other than the la to be segmented and requires no anesthesia .
compared to a registration based on the cs alone , the error was reduced by over 50% .
the average accuracy for all frames after applying temporal filtering is 5.7  mm for cade + edge and 6.1  mm for shad + edge .
though the results of cade + edge are close to the 5  mm threshold [ 27 , 28 ] , it remains open if this is sufficient for a clinical application , but we believe that these results should at least provide users with an acceptable initial estimate for further manual adjustments . by now
, the performance of this approach to contrast - based registration of the right atrium should be evaluated . a registration based on the right atrium
would also provide a registration for the la which is available for transseptal puncture or one of the ventricles could be assessed . due to the lack of pulmonary veins and arteries , the ventricles have a more simple anatomical structure but are subject to stronger cardiac motion .
compared to the approach by zhao et al . , the use of special weights for different heart regions is not needed for any of the proposed approaches . in addition , for shad + edge , a time - consuming drr generation can be avoided . as a result , a registration approach based on a combination of shadow and edge features can be computed fast .
if sufficient computational power was available , the novel cade - based measure , which takes consistency into account , should be used as it improves results significantly , especially when very small amounts of contrast agent are injected .
 for augmented fluoroscopy during cardiac ablation , a preoperatively acquired 3d model of a patient 's left atrium ( la ) can be registered to x - ray images recorded during a contrast agent ( ca ) injection . 
 an automatic registration method that works also for small amounts of ca is desired . 
 we propose two similarity measures : the first focuses on edges of the patient anatomy . 
 the second computes a contrast agent distribution estimate ( cade ) inside the 3d model and rates its consistency with the ca as seen in biplane fluoroscopic images . 
 moreover , temporal filtering on the obtained registration results of a sequence is applied using a markov chain framework . 
 evaluation was performed on 11 well - contrasted clinical angiographic sequences and 10 additional sequences with less ca . for well - contrasted sequences , 
 the error for all 73 frames was 7.9  6.3  mm and it dropped to 4.6  4.0  mm when registering to an automatically selected , well enhanced frame in each sequence . 
 temporal filtering reduced the error for all frames from 7.9  6.3  mm to 5.7  4.6  mm . 
 the error was typically higher if less ca was used . 
 a combination of both similarity measures outperforms a previously proposed similarity measure . 
 the mean accuracy for well contrasted sequences is in the range of other proposed manual registration methods . 
1. Introduction
2. Registration Method
3. Experiments and Results
4. Discussion and Conclusions
5. Conclusions