cardiac anatomical heterogeneity amongst patients ( e.g. , left atrial ( la ) pulmonary vein ( pv ) anatomy , atrial septal defect ( asd ) variations ) can pose a challenge when performing minimally - invasive procedures .
the accuracy of navigation during cardiac procedures is limited in part by this patient - to - patient variability and in part by the operator 's ability to manipulate the catheter / guide wire manually to the intended intracardiac location .
precision , safety and a successful procedure hinge around better appreciation of structural anatomy and the relationship between catheters , guide wires and anatomical structures .
most interventional devices are designed to be x - ray visible and can be seen throughout the part of their length that lies in the x - ray field of view ( fov ) .
two - dimensional ( 2d ) x - ray imaging is the dominant imaging modality for guiding cardiac interventions .
typically , imaging can be performed at high frame rates ( up to 30 frames per second ) and therefore the cardiac motion and the motion of interventional devices do not cause significant motion artefacts in the acquired images .
in addition to x - ray , pre - procedural imaging of patients undergoing cardiac interventions is becoming routine and plays an important part in patient management .
imaging modalities such as computerised tomography ( ct ) , magnetic resonance imaging ( mri ) and three - dimensional ( 3d ) rotational x - ray angiography ( rxa ) or c - arm ct have been integrated into the cardiac catheterisation laboratory in a variety of ways  , with successful clinical cases reported in the literature .
this is especially common for cardiac electrophysiology procedures where pre - procedural images can be combined with electroanatomical mapping systems such as ensite ( st .
these approaches have been shown to reduce procedure time and radiation dose , and to improve outcome for some types of procedures , .
however , pre - procedural imaging represents a static roadmap image of the heart and can not monitor the cardiac intervention in real - time .
this can be important since tissue deformations do occur due to the presence of interventional devices and delivered treatments .
furthermore , elaborate techniques are required to compensate the static roadmaps for the contractile and respiratory motion of the heart .
in contrast to the relatively slow data acquisition of mri and ct , 3d echocardiography is a real - time imaging modality which is increasingly performed during cardiac interventions .
this technique visualises not only the relevant cardiac structures but also the interventional devices and delivery systems .
furthermore , 3d transesophageal echocardiography ( tee ) confers improved spatial and temporal resolution of the cardiac anatomy .
3d tee is now routinely used in a variety of cardiac interventional procedures including structural heart disease : with device closures , valvular heart disease : with mitral annulus resizing , mitral annulus repair , transcatheter aortic valve implantations ( tavi ) ; and in electrophysiology with trans - septal puncture guidance and percutaneous left atrial appendage device closures . more recently , intracardiac echocardiography ( ice ) has been used to enable real time visualisation of intracardiac chambers and adjacent structures and facilitate precise guidance of catheter based procedures .
however this imaging modality confers a smaller fov in comparison to tee , requires an additional femoral puncture and has cost implications with each ice catheter being non - reuseable in europe and being reusable only a few times in the united states .
real - time 3d tee   is presently used in combination with x - ray fluoroscopy and is particularly useful in defining location and morphology of the target cardiac structure and anatomical relation to neighbouring landmarks . whilst 3d tee allows for high quality visualisation of cardiac anatomy ,
x - ray has the potential to overcome the limitations of 3d tee by assisting in the optimum fov selection for the 3d ultrasound and allowing a clearer definition of the interventional device .
procedural guidance is thus dependent on visual information obtained from two separate screens showing the 3d tee and x - ray fluoroscopy .
this approach lacks the precise anatomical relationship between the device and the cardiac structure of interest .
in addition , continued x - ray fluoroscopy during device positioning and deployment exposes patients , especially paediatric patients with congenital defects , to ionising radiation which carries a significant risk , . a possible solution to this is integration of the real - time 3d tee and x - ray fluoroscopy to improve device to tissue spatial orientation .
one approach focused on the registration of transthoracic echocardiography ( tte ) and x - ray   using a mechanical tracking device .
reported a method to register tee volumes with x - ray by using an electromagnetic ( em ) tracking system .
alternatively , a fiducial - based approach can be used to track small , x - ray visible markers attached to the tee device .
we have previously developed an accurate and robust image - based algorithm for registration of 3d tee and x - ray fluoroscopy .
this image - based approach requires no modification of the existing hardware and so is more suited to clinical translation . in the current paper
, we develop this further into a real - time clinical system to carry out the image fusion . to our knowledge , this is the first real - time clinical platform developed using this concept .
while we have previously demonstrated successful registration of static clinical images in , we now perform a more extensive validation using the real - time implementation with sequences of images over time .
we perform pre - clinical experiments to validate the platform and clinical feasibility experiments to establish a workflow for the system and to measure clinical accuracy .
fig .  1 gives an overview of the prototype clinical system in which live x - ray and 3d echo are integrated to obtain the hybrid view .
the hardware comprises a philips allura xper fd10 c - arm x - ray system and a philips ie33 3d ultrasound system with an x72 t 3d tee probe .
data is streamed in real - time from each image source to a pc running the fusion and visualisation software .
the software displays the live x - ray images overlaid with a projection of the live 3d echo volume . 

1.both the x - ray fluoroscopic image and echocardiography image are acquired separately and streamed into the visualisation platform that allows for the real - time integration of the two modalities . 
 
both the x - ray fluoroscopic image and echocardiography image are acquired separately and streamed into the visualisation platform that allows for the real - time integration of the two modalities .
the two modalities are registered using our previously described method . in brief , this uses a previously acquired nano - ct image of the tee probe which is registered to the projection image of the probe in the x - ray .
the registration algorithm involves simulating the x - ray projection process to generate projections of the 3d probe image , called digitally reconstructed radiographs ( drrs ) . using these images
, the algorithm iteratively adjusts the position and orientation of the 3d probe image to obtain the best match between its drr and the x - ray projection .
2 shows a typical x - ray view , the probe model and a registration of the two .
2.automatic overlay registration . ( a ) tee probe model from a nano - ct scan of the probe head .
( b ) x - ray image with the projection of the tee probe clearly visible .
( c ) probe model registered to the x - ray by automatic 3d-2d registration . 
  automatic overlay registration .
( a ) tee probe model from a nano - ct scan of the probe head .
( b ) x - ray image with the projection of the tee probe clearly visible .
( c ) probe model registered to the x - ray by automatic 3d-2d registration . in the clinical prototype , this iterative process is initialised manually to approximately the correct position and orientation before running the automatic 3d-2d registration . following the initial alignment , the automatic registration is repeatedly rerun on the updating x - ray images to track changes in the probe position due to cardiac and respiratory motion .
this registration is gpu - accelerated ( graphical processing unit , nvidia geforce 8800gtx ) and updates in this way at a rate of 12 hz , sufficient for near - real - time tracking of the probe position .
this is a significant improvement on our previous implementation   in which an equivalent setup required an average of 8 s per registration .
the protocol was divided into 2 phases . the pre - clinical phase involved use of the system for data acquisition from a phantom model and during a porcine experimental study . the second , clinical phase involved use of the system for data acquisition during catheter ablation ( ca ) in 9 patients and tavi in 2 patients following approval from the local research ethics committee . in stage 1
, the prototype image - based tee probe localisation algorithm was validated using a phantom model .
the phantom comprised a water tank in which two wires were suspended forming a cross .
this was imaged by the tee probe with the ultrasound machine in full volume mode .
corresponding x - ray images were acquired at two different c - arm positions ( rao 54 and rao 24 views ) each at high and low dose ( 4 x - rays in total ) . from these 4 sequences ,
errors were measured between corresponding points on the x - ray and echo projection images of the crossed wires .
these error measurement points were defined by manually fitting straight - line models to the projected images of the crossed wires in the two imaging modalities .
the crossing points were then detected automatically from these fitted lines and corresponding measurement points were automatically defined in fixed steps along the lines from the crossing point .
this provided up to 13 measurements per overlay view ( minimum 10 where the crossing point was near the edge of the view ) .
2d projection errors were calculated between corresponding points and averaged to give an overall alignment error , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ e$\end{document } , for the overlay view as follows \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ $ e={{1}\over{n}}\sum_{i=1}^{n}{\left\vert{p_{i,{\rm xray}}-p_{i,{\rm echo}}}\right\vert}{{d_{{\rm source}\to{\rm target}}}\over{d_{{\rm source}\to{\rm detector}}}}\eqno{\hbox{(1)}}$$\end{document } where \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ n$\end{document } is the number of measurements , and \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ p_{i,{\rm xray}}$\end{document } and \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ p_{i,{\rm echo}}$\end{document } are the locations of corresponding points \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ i$\end{document } in the overlay view .
\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ d_{{\rm source}\to{\rm target}}$\end{document } and \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ d_{{\rm source}\to{\rm detector}}$\end{document } scale the errors measured in the magnified x - ray projection view to errors at the patient location .
errors were measured between automatically defined points on straight line models of the crossed wires .
errors are measured as the shortest distance from automatically defined points on the echo catheter image to a spline model of the x - ray catheter . 
 
errors were measured between automatically defined points on straight line models of the crossed wires .
errors are measured as the shortest distance from automatically defined points on the echo catheter image to a spline model of the x - ray catheter . in vivo data to measure accuracy were acquired from a live porcine experiment . following successful trans -
septal puncture , a circular mapping catheter ( \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ { \rm inquiry}^{\rm tm}~{\rm optima}^{\rm tm}$\end{document } , st .
jude medical inc . ) and an ablation catheter ( 3.5 mm irrigated tip catheter \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ { \rm navistar}\circit{\rm r}~{\rm thermocool}\circit{\rm r}$\end{document } , biosense webster inc .
five echo volumes were acquired in full volume mode with the probe in different positions . at each position ,
between one and three x - ray sequences were recorded ( rao 30 , lao 30 and pa positions ) with high x - ray dose setting ( 9 x - ray views in total ) . the catheters inside the heart during the acquisition provided convenient targets for measuring alignment errors . although real - time synchronised visualisation of the live data stream was possible during the procedure , the post - procedure analysis for this paper required that the recorded x - ray and echo data were synchronised manually , resulting in only approximately synchronised sequences .
automatic registrations were done at two separate frames in each x - ray sequence ( 18 overlay views generated ) .
corresponding catheters were manually defined in the echo and x - ray views using catmull - rom spline curves .
the corresponding x - ray point was defined as the closest point on the x - ray curve . the alignment error for each overlay view
was again taken as the average of the 2d errors between corresponding points , in the same way as in the phantom experiments .
average errors were measured using between 4 and 6 point pairs per overlay view , depending on the length of catheter visible in the echo image . in all cases ,
the mapping catheter was used for the measurements , because it was the most consistently visible catheter over all the echo images .
the aim was to assess the workflow of the platform during live clinical procedures and not to guide the interventional procedure at this early stage .
the operators performing the procedure relied on the conventional standard echo visualisation on one screen and the fluoroscopic image on another . patients
studied here were candidates undergoing an ablation for atrial fibrillation ( af ) under general anesthesia , requiring a tee , and patients undergoing a tavi procedure . during each case , x - ray and echo sequences were acquired independently and streamed into the visualisation software .
x - ray and echo sequences were also recorded for post - procedure analysis . during the procedure , real - time synchronised visualisation of the live data streams
was achieved automatically . however , the post - procedure study for accuracy required that the recorded x - ray and echo sequences were synchronised manually . the catheters used to measure accuracy were those that were visible in the echo image at the time of acquisition .
this varied depending on the stage of the procedure , but was often only one device at a time .
for the electrophysiology studies , catheter sheaths were used for error measurement , as well as mapping and ablation catheters ( see fig .  4 for example images ) .
for the tavi procedures , the guide wire was used for error measurement in both sequences ( fig .  5 ) . 

real - time hybrid views were generated during the trans - septal puncture and subsequent placement of circular mapping and ablation catheters within the left atrial chamber . 

real - time hybrid views were generated during the placement of the edwards core valve within the native aortic valve followed by the deployment of the prosthetic valve . 
  clinical study to document feasibility in an af ablation case .
real - time hybrid views were generated during the trans - septal puncture and subsequent placement of circular mapping and ablation catheters within the left atrial chamber .
real - time hybrid views were generated during the placement of the edwards core valve within the native aortic valve followed by the deployment of the prosthetic valve .
following establishing conscious sedation and peripheral access , a 6f decapolar catheter was placed in the coronary sinus ( as a reference during electroanatomic mapping and to enable la pacing ) and the xfr nih catheter was placed within the superior vena cava ( fig . 
simultaneously , a tee probe ( echo frequency 58 mhz ) was advanced to study the left atrium and exclude a left atrial appendage clot .
the probe was then rotated to visualise the interatrial septum , assessing for possible patent foramen ovale and establishing the most suitable site for trans - septal puncture ( fig . 
once established , the probe was held still in place and the image obtained was overlaid onto the fluoroscopy .
two trans - septal punctures were made and access to the left atrium was obtained using 8.5f non - deflectable long sheaths , ( st .
4(c ) and ( d ) ) . following the first trans - septal puncture , intravenous heparin was administered to achieve an activated clotting time of between 300 and 400 seconds .
a 3d geometry of the left atrium was created using either \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ { \rm navx}^{\rm tm}$\end{document } ( st .
paul , mn , usa ) or carto xp ( biosense webster inc . , diamond bar , ca , usa ) . a circular mapping catheter ( \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ { \rm inquiry}^{\rm tm}{\rm optima}^{\rm tm}$\end{document } , st .
was then placed in each pulmonary vein in turn while the corresponding la - pv antrum was targeted with wide area circumferential ablation ( fig . 
the procedure was performed under general anesthesia in the catheterisation laboratory under strict aseptic conditions , by a team comprising interventional and imaging cardiologists , cardiac surgeons and anesthetists .
the tee probe was positioned to view the left ventricular outflow tract and ascending aorta , aortic valve and left ventricle .
the stenosed aortic valve was crossed with a hydrophilic wire ( terumo medical , somerset ) supported by an al1 5f diagnostic catheter ( fig . 
a pigtail catheter was then advanced into the lv whilst simultaneous pressure recordings ( lv / aorta ) were performed . over the pigtail , a super stiff 260-cm long wire , with a floppy tip of
an 18f sheath was then advanced carefully over the amplatz catheter with a subsequent two - stage process : firstly , as the tip of the dilator entered the aorta , it was stabilised and the sheath was advanced over it until it rested just inside the aortic lumen .
next , over the stiff wire a nucleus balloon catheter ( numed , ny ) was advanced and valvuloplasty was successfully performed . following this ,
the corevalve prosthesis itself was advanced through the sheath and across the aortic valve ( fig . 
5(c ) ) and successfully deployed in its correct position in the patient ( fig . 
summaries for continuous variables are expressed as median and interquartile range ( iqr ) and mean \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ { \pm}~{\rm sd}$\end{document}. a student 's t - test was used to determine the differences between groups . a \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ p$\end{document } value of less than 0.05 was considered statistically significant .
in the phantom model , 12 synchronised echo - fluoro image pairs were overlaid . on each hybrid image ,
a mean of 11.5 measurements ( range 1013 ) between corresponding points on the crossed wires were made to assess the 2d projection error . from a total of 138 error measurements ,
a median alignment error of 4.5 mm ( iqr 3.05.9 mm ) was calculated . in the porcine experimental study ,
each had between 4 and 6 projection error measurements made between corresponding points on the mapping catheter ( 83 measurements in total ) .
6.the box plots display the overall alignment error measurements evaluated in the phantom model and the porcine experimental study . 
 
the box plots display the overall alignment error measurements evaluated in the phantom model and the porcine experimental study .
the 9 patients with symptomatic drug - refractory af ( paroxysmal / persistent ) undergoing catheter ablation had the following characteristics : age , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 59.2\pm 8~{\rm years}$\end{document } ; male / female , 6/3 ; weight , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 95.6\pm 18.2~{\rm kg}$\end{document } ( range 64110 kg ) ; mean duration of af , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 5.4\pm 3.8~{\rm years}$\end{document } ( range 111 years ) ; and left ventricular ejection fraction , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 50\pm { 10\%}$\end{document } ( range , 4070% ) .
the 2 tavi patients were both male aged \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 89\pm 1.4~{\rm years}$\end{document } , weight \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 82.5\pm 5.6~{\rm kg}$\end{document } ( range 8090 kg ) with moderate renal function and an ejection fraction of approximately 35% .
all 4 pulmonary veins were isolated and in the tavi group the corevalve ( edwards corevalve ) prosthesis was deployed with good positioning confirmed on echo . 

table ipatient characteristics and procedural data.catheter ablationtavin=9n=2gender ( male ) / % 67100age / years\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 59.2\pm 8$\end{document}\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 89\pm 1.4$\end{document}ejection fraction / % \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 50\pm 10$\end{document}\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 36 \pm 12$\end{document}weight / kg\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 95.6\pm 18.2$\end{document}\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 82.5\pm 5.6$\end{document}procedure time / mins\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 160 \pm 28.6$\end{document}\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 75\pm 14.6$\end{document}fluoroscopy time / mins\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 54.4\pm 12.2$\end{document}\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 34.8\pm 6.4$\end{document}fluoroscopy frame rate / fps\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 3 - 7$\end{document}\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 15 - 30$\end{document}radiation dose / \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ { \rm cgy}.{\rm cm}^2$\end{document}\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 842\pm 11.4$\end{document}\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ 640\pm 5.6$\end{document } 
  successful real - time integration was achieved in all 11 clinical cases .
4 and 5 show typical examples of overlays from the catheter ablation and tavi cases . from a total of 124 alignment error measurements made , a median error of 2.4 mm ( iqr 1.33.9 mm )
was achieved . in the catheter ablation cohort ( 100 measurements ) a median of 2.2 mm ( iqr 1.33.4 mm ) was calculated whilst among the tavi cohort ( 24 measurements ) a median of 3.4 mm ( iqr 1.54.7 mm ) was achieved . fig . 
median alignment difference between catheter ablation and tavi procedures was 1.2 mm \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } $ ( p=0.17)$\end{document}. 
 fig . 
the first nine box - plots were obtained from the af cases and the remaining two from tavi cases . 
 
the first nine box - plots were obtained from the af cases and the remaining two from tavi cases .
this is the first study to report a platform for real - time 3d tee and x - ray fluoroscopy image integration evaluated in a phantom model , porcine experimental study and in the clinical setting .
the main findings of this study are ( 1 ) successful real - time image integration and overlay of 3d tee and x - ray fluoroscopy , ( 2 ) a median real - time alignment accuracy of 4.5 mm and ( 3 ) a feasible workflow during the platform application within the clinical setting .
the main aim of the study was to examine the feasibility of overlaying 3d - echocardiogram volumes onto fluoroscopic images during real - time catheter / guide wire navigation .
this platform does not involve the use of any additional tracking devices and therefore can be easily integrated into the current workflow of the catheter laboratory .
the fundamental utility of this combination is the ability to better appreciate soft tissue anatomy in relation to the catheter / guide wire during navigation and device deployment .
one of the potential advantages of the hybrid echo - fluoroscopy imaging modality is increased and more precise control of positioning of the catheter / guide wire .
visualisation of catheter direction in relation to the anatomical structures facilitates faster and more accurate catheter movement to the intended anatomical site .
an echo volume on its own can be difficult to interpret because of its limited fov and lack of context for the echo coordinate system relative to the patient .
catheters and devices tend to produce artefacts in the ultrasound data reducing the clarity of the images . in the overlay view
, the echo volume is displayed in a coordinate system that can be more easily related to the patient .
also , the highly visible catheters in the x - ray image help with identifying the catheters in the echo and so can be related to the cardiac anatomy via the echo image . during an ablation procedure ,
the precise location of the catheter tip and electrodes on the mapping catheter in relation to atrial tissue permits better targeted lesion delivery than just visualising the catheter as a whole on fluoroscopy .
similarly during a tavi procedure , being able to visualise both the native valve and the prosthesis on a single image will facilitate delivery and deployment of the prosthetic valve .
the accuracy requirement for a clinically useful image guidance system depends on many factors including the patient and the type of procedure being performed , .
an accuracy of less than 5 mm is often deemed as a suitable target for many applications and this threshold is highlighted on the plots in figs . 
6 and 7 where most of the errors are below the 5 mm threshold . based on these results ,
it should be noted that the location of the catheter in the echo volume will affect the accuracy , as objects further from the probe will have a greater misalignment due to errors in the probe 's orientation .
the standard deviations noted within the figures are , in part , likely to arise from some data streaming delay in the visualisation software and , in part , time lag in x - ray registration .
the largest standard deviations are due to respiratory motion acting on the probe , which is not immediately compensated by the 12 hz registration updates .
while all sequences encounter respiratory motion , variations in the probe position and x - ray direction produce different scales of potential error in each sequence .
the median alignment error difference between the tavi and ablation procedures ( 3.4 mm vs 2.2 mm ) may be attributed to the rapid movement of the guide wire / device that is located near the aortic valve in contrast to the lasso and ablating catheter located within the left atrium . in comparison to the phantom data ,
. this may be due to the error measurement techniques : full 2d errors were measured in the phantom experiment whereas in the porcine and clinical experiments it was only possible to measure to the nearest point on the corresponding catheter .
other studies on echo - fluoroscopy overlay have also found clinically - useful accuracies . in
, the mean error was approximately 2.0 mm when imaging a heart phantom and tracking the tee probe with an electromagnetic tracking system .
.   found a mean error of approximately 1.2 mm in vivo , by tracking fiducial markers attached to the tee probe .
these are both better than our system 's average accuracy , but require devices to be attached to the probe .
the results therefore highlight a trade - off between accuracy and ease of clinical integration . in this study , we have presented a platform for real - time hybrid x - ray fluoroscopy and 3d echo visualisation and have successfully demonstrated a feasible workflow within the clinical setting .
we anticipate that by gaining familiarity and confidence within this hybrid viewing system , there will be a lesser need for repeated x - ray fluoroscopy use to establish catheter position .
on - going research is actively seeking to reduce the use of , or even replace , x - ray fluoroscopy in cardiac interventional procedures , especially for paediatrics . with increased usage over time , we predict a greater reliance on echo views to assist catheter / guide wire navigation . in the long run , this will reduce overall fluoroscopy time and patient exposure to ionising radiation
. ultimately , randomised comparisons of real - time hybrid x - ray fluoroscopy and 3d echo image - guided therapy with standard visualisation approaches are necessary to definitively establish its clinical utility and translate it from an experimental method to a clinical routine .
it is important to recognise that one of the main objectives of this study was to assess the clinical feasibility of the proposed method .
although all the porcine experimental and clinical data were acquired specifically for this study , the data analysis was performed off - line .
the porcine experiment in particular may suffer from artificially large errors due to the echo and fluoroscopy images not being acquired simultaneously .
our future work will focus on evaluating the accuracy of the live system and further improving the co - registration .
in addition , our approach of measuring to the closest point accounts for 2d in plane errors and does not necessarily capture the complete error as there can also be misalignment tangentially to the catheters .
however , many of the catheters were curved , which reduces the severity of this limitation . the clinical case numbers in this study at the present time are small .
while we have established the clinical feasibility of this system , further studies involving larger patient groups are required to demonstrate that this novel technique translates into reduced fluoroscopy time , reduced complications and better overall patient outcome .
in this study , we have developed a clinically - integrated system for real - time hybrid x - ray fluoroscopy and 3d echo visualisation .
the practicability of accurate real - time hybrid visualisation was demonstrated in both a phantom model and a porcine experimental study .
the clinical series demonstrated a feasible clinical workflow , with successful image fusion in all cases . further clinical validation and a randomised comparison between real - time hybrid echo - fluoro image - guided therapy versus a standard visualisation approach
is needed to translate this tool into routine clinical practice , where it may prove useful for real - time guidance of catheterisation interventions .
 real - time imaging is required to guide minimally invasive catheter - based cardiac interventions . 
 while transesophageal echocardiography allows for high - quality visualization of cardiac anatomy , x - ray fluoroscopy provides excellent visualization of devices . 
 we have developed a novel image fusion system that allows real - time integration of 3-d echocardiography and the x - ray fluoroscopy . 
 the system was validated in the following two stages : 1 ) preclinical to determine function and validate accuracy ; and 2 ) in the clinical setting to assess clinical workflow feasibility and determine overall system accuracy . in the preclinical phase , 
 the system was assessed using both phantom and porcine experimental studies . 
 median 2-d projection errors of 4.5 and 3.3 mm were found for the phantom and porcine studies , respectively . 
 the clinical phase focused on extending the use of the system to interventions in patients undergoing either atrial fibrillation catheter ablation ( ca ) or transcatheter aortic valve implantation ( tavi ) . 
 eleven patients were studied with nine in the ca group and two in the tavi group . 
 successful real - time view synchronization was achieved in all cases with a calculated median distance error of 2.2 mm in the ca group and 3.4 mm in the tavi group . 
 a standard clinical workflow was established using the image fusion system . 
 these pilot data confirm the technical feasibility of accurate real - time echo - fluoroscopic image overlay in clinical practice , which may be a useful adjunct for real - time guidance during interventional cardiac procedures . 
Introduction
Methods
Result
Discussion
Conclusion