image - guided neurosurgery relies on preoperative images to provide surgical visualization and navigation into the brain after registration of the images to the patient s physical space .
however , access to the brain subsequent to craniotomy often leads to deformation of the brain along with the movement of subsurface resection targets such as a tumor .
the amount of brain shift depends on a number of factors including the extent of the craniotomy , retraction , tumor resection  , drainage of cerebrospinal fluid ( csf ) , ,   and drugs administered during surgery , . as a consequence , cortical shifts of up to 20 mm ,   and subsurface shifts of up to 7 mm , , , ,   have been reported and result in fundamental misalignment between actual brain target positions and their counterparts as determined from registered preoperative images .
in addition , when one considers the abundance of preoperative image information ( e.g. , functional magnetic resonance ( mr ) data , positron emission tomography , mr diffusion tensor imaging , etc . ) that can be brought to bear on the care of patients during surgery , the need to re - establish alignment between preoperative and intraoperative states becomes even more critical .
one direct approach to achieving updated deformed brain images is to re - image the brain during surgery using intraoperative magnetic resonance ( imr ) imaging systems . to date , imr systems have been the only clinical solution that has been adopted to any extent . while these systems are similar to diagnostic ones , often due to the surgical environment and workflow , the quality of these images acquired is not the equivalent of their preoperative counterparts . in an effort to utilize the pristine preoperative anatomical images as well as other forms of imaging data
, preoperative images are deformed to match the intraoperative images using non - rigid registration techniques that are image - based   or physics - based  using the data - rich but albeit lesser quality intraoperative images to drive the registration .
while significant work has been produced in this direction , imr imaging systems are rather costly , occupy a significant portion of operating room ( or ) space and may not be available in every hospital .
a more cost - effective solution is to make use of the exposed cortical surface to record brain shift and use the subsequently measured surface displacements to drive a comprehensive biomechanical model of the brain .
once the model has computed a deformation field , it can then be used to update / deform the preoperative images ,   ( and other data consequently ) .
the difficulty with this approach is determining the extent of data necessary to produce a sufficiently accurate registration for intraoperative guidance while simultaneously trying to minimize the impact on operational work flow , i.e. , the sparse data extrapolation problem .
while there have been many proposed sparse - data solutions with encouraging results in phantom , animal and human studies , the work has largely reflected retrospective analysis , , . for practical use in the or , the updated preoperative images must be produced within a reasonable amount of time .
this time constraint means that the cortical brain data collection and processing must be executed quickly and with minimal interruption to the surgical workflow . a brain shift compensation system , which includes a preoperative biomechanical model development pipeline , a preoperative surgical planning graphical user - interface ( gui ) and two intraoperative guis , was developed to perform real - time brain shift correction in the or .
this study introduces this brain shift compensation system and presents a comprehensive evaluation of it in terms of the time taken for each processing step along with an analysis of possible areas for improvement .
a semi - automated , preoperative and intraoperative computational processing pipeline for brain shift correction was developed ( figure 1 ) .
briefly , preoperative magnetic resonance ( mr ) images are acquired a day or more prior to surgery ( diagnostic series can be used provided significant surgical changes have not ensued ) . from the images , the patient s brain , tumor and intracranial support structures , falx and tentorium cerebri , ,  
a patient - specific volumetric finite element mesh is generated from the segmented brain and tumor images with the structures of the falx and tentorium celebri having predefined boundary conditions .
a preoperative planning gui was developed for use by neurosurgeons to establish the approximate head orientation as well as size and location of the craniotomy .
based on the preoperative plan , remaining boundary conditions are generated using an automatic boundary condition generation algorithm . as the exact forcing conditions are difficult to know ( e.g. , level of csf drainage , gravitational direction , effects of hyperosmotic drugs and edema ) , a distribution of possible conditions is determined which generates an atlas of boundary conditions .
each boundary condition set is used to constrain a finite element deformation solution thus producing a distribution of possible deformation solutions or a 
the model used within this precomputation phase is a biphasic biomechanical model that takes into account may of the sources of brain shift , i.e. , hydration effects from drugs like mannitol , gravity - induced brain sag due to csf drainage , resection effects , and skull - tissue interactions , . 

figure 1a workflow illustrating the semi - automated preoperative and intraoperative computational processing steps involved in producing an updated brain shift image in real - time .
the inputs are preoperative mr images , face lrs scan for registration , and pre and post - resection cortical brain surface lrs scans to drive the inverse modeling .
a workflow illustrating the semi - automated preoperative and intraoperative computational processing steps involved in producing an updated brain shift image in real - time .
the inputs are preoperative mr images , face lrs scan for registration , and pre and post - resection cortical brain surface lrs scans to drive the inverse modeling . on the day of surgery ,
the deformation atlas is transferred to the intraoperative guidance system that performs an inverse model calculation driven by sparse cortical brain deformation measurements obtained from intraoperative laser range scanner ( lrs ) data .
the lrs records the cortical brain surface by sweeping a line of laser light across the surface while recording the laser line with a digital camera and by triangulation produces a 3-dimensional ( 3d ) point cloud of the surface geometry .
texture information is also recorded from the same digital camera by acquiring a 2-dimensional bitmap of the field of view ( fov ) .
other examples of lrs use in image - guided procedures include orthodontics , neurosurgery  , liver surgery , ,   and cranio - maxillofacial surgery , . in this work , a commercial lrs system ( pathfinder therapeutics , inc . ,
nashville , tn ) was integrated with an optical tracking system ( polaris spectra , northern digital inc .
after cortical brain measurements are made using the lrs , the optimum brain shift solution is determined from an inverse problem approach using the deformation atlas , .
once calculated , the patient s brain image data is subsequently deformed using the optimum solution to reflect the current state of the brain s shape
. a preoperative planning gui ( called surgical planner ) , a processing automatic pipeline , and two intraoperative guis ( called registration , and correction ) were developed for use before and during surgery to plan and process the collected data .
the custom software was written in c++ using open source insight segmentation and registration toolkit ( itk ) , visualization toolkit ( vtk ) and fast light toolkit ( fltk ) libraries .
matlab r2011b ( mathworks , natick , ma ) s parallelization and optimization toolboxes were also used .
methodologies used will be briefly discussed , followed by results concerning the full system performance . in this study , five patients were processed through the preoperative and intraoperative pipelines .
all patients provided written consent prior to imaging for this vanderbilt institutional review board approved study .
for each patient , two sets of t1-weighted mr image volumes , one gadolinium - enhanced and the other non - enhanced , were acquired from a conventional clinical mr scanner ( table 1 ) . 

patient demographics and mr image details . to streamline the preoperative pipeline and model generation , a semi - automatic segmentation of the brain was implemented .
briefly described , for each patient , a rigid alignment is performed between patient s enhanced and non - enhanced image volumes .
once complete , the patient s non - enhanced image volume is registered to an expertly segmented non - enhanced brain image volume ( i.e. , atlas volume ) first using a mutual information rigid registration followed by a custom - built adaptive bases nonrigid registration algorithm .
once complete , the atlas mask can be transformed such that the contrast - enhanced patient s can be automatically segmented .
we should note that the falx and tentorium have been expertly segmented in the atlas which serves as an automatic approach to deploying the dural septa within our model .
the atlas also provides an excellent reference surface set for finite element mesh generation once registered to the patient .
we should note that a visual confirmation is performed when complete , and some manual editing of the automatic segmentation is sometimes performed using the open source software itk - snap ( www.itksnap.org ) to correct small discrepancies . at this time
, itk - snap is also used to manually segment the contrast - enhancing tumor region .
we should note that manual methods of tumor segmentation are the standard in commercial guidance systems .
finally , we should note that in , a sensitivity study was performed which compared our brain shift correction results based on models built from our semi - automatic segmentation approach versus those coming an expert manual segmentation approach and no statistical difference between results was found .
the direction and degree of brain shift are in part dependent on how the head is oriented with respect to gravity , as well as location and the size of the craniotomy .
a priori knowledge of these 3 variables helps to limit the size of the atlas of possible deformation solutions and can be provided by the neurosurgeon during preoperative planning .
a user - interactive gui was developed to assist the neurosurgeon in quantifying those variables .
brain and tumor surface meshes were converted from segmented brain and tumor images respectively by using marching cubes and smoothing algorithms .
both surface meshes were rendered in the gui and the neurosurgeon rotated the brain into the planned position and recorded the transformation .
the center of the craniotomy was selected by picking a point on the brain surface and the craniotomy size was determined using the slider tool to adjust the sphere ( figure 2 ) .
these 3 variables were used later in defining the boundary conditions of the computational model . 

figure 2screenshot of the surgical planner gui with the brain and tumor surfaces loaded and oriented to the same position as in the or . the center and size of the craniotomy
are represented by the green sphere selected by the neurosurgeon screenshot of the surgical planner gui with the brain and tumor surfaces loaded and oriented to the same position as in the or .
the center and size of the craniotomy are represented by the green sphere selected by the neurosurgeon based on the observation that the brain acts similar to fluid saturated poroelastic medium , biot s theory of biphasic consolidation was used to represent the deformation behavior of brain tissue , . according to biot s theory ,
the mechanical behavior of a poroelastic medium such as the brain can be described using equations of linear elasticity for solid porous matrix and darcy s law for describing the flow of fluid through the porous matrix .
equation ( 1 ) represents equilibrium whereby the gradient in interstitial fluid pressure can cause shape change to the solid matrix .
in addition , changes in the buoyancy of the surrounding fluid can generate gravity - induced deformations . in equation
( 2 ) , we see a conservation of fluid mass relationship whereby changes in hydration can effect the the time rate of change of the volumetric strain of the solid matrix . in addition , we also allow for dilatation effects as exchange with capillary beds can occur in response to drugs like mannitol .
the model can be described as , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \begin{align } \nabla \cdot g\nabla \vec { u}+\nabla \frac { g}{1 - 2\textit{v}}(\nabla \cdot \vec { u})- \alpha \nabla p=&-(\rho
\\ \alpha \frac { \partial } { \partial t}(\nabla \cdot \vec { u})+k_{c}(p - p_{c})=&\nabla \cdot k\nabla p\end{align } 
 \end{document } where \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(g\ ) 
 \end{document } is the shear modulus defined by \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(\frac { e}{2\left ( { 1+\nu } \right ) } \ ) 
 \end{document } with \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(e\ ) 
 \end{document } as young s modulus and \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(\nu \ ) 
 \end{document } is the poisson s ratio .
\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(\vec { u}\ ) 
 \end{document } is the displacement vector , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(\alpha \ ) 
 \end{document } is the ratio of fluid volume extracted to volume change of the tissue under compression , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(p\ ) 
 \end{document } is the interstitial pressure , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(\rho _ { t}\ ) 
 \end{document } is the tissue density , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(\rho _ { f}\ ) 
 \end{document } is the surrounding fluid density , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(g\ ) 
 \end{document } is the gravitational unit vector , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(k_{c}\ ) 
 \end{document } is the capillary permeability , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(p_{c}\ ) 
 \end{document } is the intracapillary pressure and \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(k\ ) 
 \end{document } is the hydraulic conductivity .
this constitutive model is a common model and has been used successfully to describe brain shift , , .
for each patient , a patient - specific finite element volumetric mesh was generated from the mr images .
briefly , once the patient s images have been segmented , a marching cubes algorithms can be used to generate bounding surface . a custom - built mesh generator is then used to generate a volumetric tetrahedral mesh   with parenchyma , and tumor designated .
parenchyma can be discretized further into white and gray matter elements using an image - to - grid methodology whereby the average image - intensity of voxels within a tetrahedral element can be determined and then used to threshold tissue type .
typically , a brain mesh consists of approximately 100,000 tetrahedral elements ( figure 3 ) . 

figure 3brain ( in white ) and tumor ( in yellow ) mesh overlaid with mr images .
brain ( in white ) and tumor ( in yellow ) mesh overlaid with mr images .
the boundary conditions applied were determined according to observed conditions commonly associated with brain shift from previous studies , , .
the boundary conditions associated with displacement that were found to produce good estimates of brain shift were as follows : ( 1 ) the brainstem area was typically found to be very stable and as a result represent a fixed , i.e. , no displacement condition  figure 4 , left , red region , ( 2 ) in the region of the craniotomy and surrounding area where the brain can often sag away or shift laterally , the surface was designated as stress free allowing for the brain to fall away from cranial wall  figure 4 , left , green region , ( 3 ) the remaining brain surface is bound by the skull such that movement is limited to tangent - to - the - skull motion along the cranial wall only , i.e. , a freedom to slip boundary condition  figure 4 , left , black region , ( 4 ) slip boundary conditions were also designated for the internal rigid dural septa structures 
figure 4 , left , magenta region . as equations ( 1,2 ) state , gradients in interstitial pressure can induce deformations and do embody the transient effects of the model .
boundary conditions were either designated at an atmospheric reference pressure in elevations above cerebrospinal fluid ( csf ) drainage levels  figure 4 , right , blue region , or non - draining surfaces ( i.e. no flux ) below drainage levels  figure 4 , right , red region . 

figure 4(left ) mesh of the brain with the fixed brain stem nodes in red , stress - free nodes in green , slippage nodes in black , dural septa nodes also defined with slip boundary conditions in magenta and tumor nodes in blue .
( right ) mesh of the brain with dirchlet boundary conditions for pressure set on blue nodes at a baseline reference pressure and neumann boundary conditions set on the red nodes indicating non - draining surfaces .
( left ) mesh of the brain with the fixed brain stem nodes in red , stress - free nodes in green , slippage nodes in black , dural septa nodes also defined with slip boundary conditions in magenta and tumor nodes in blue .
( right ) mesh of the brain with dirchlet boundary conditions for pressure set on blue nodes at a baseline reference pressure and neumann boundary conditions set on the red nodes indicating non - draining surfaces .
while the above provides a good reference for a single boundary condition set , the surgical environment is quite dynamic . as a result ,
our strategy is to generate a distribution of possible boundary conditions based on reasonable surgical presentation , a so - called  atlas of deformations. as a result , the boundary conditions in the previous section have been parameterized such that based on minimal preoperative planning a complete deformation atlas can be constructed .
the distribution of boundary conditions is based on three mechanisms of brain shift that we have observed to be important : gravity - induced brain shift , brain volume reduction due to administration of hyperosmotic drugs like mannitol , and brain swelling due to edema around the tumor , . for gravity - induced deformation
, we have varied the atlas to express three different levels of csf drainage which influences the deployment of pressure - related boundary conditiozns ( figure 4 , right ) .
we should also note that with each drainage level , we also account for a distribution of possible head orientations around the estimate from the preoperative plan .
while this accounts for inaccuracies in the preoperative plan , it also helps to account for surgical table adjustments during surgery ( typically involves a distribution of + /20 degrees from preoperative estimate , leads to approximately 60 different head orientations ) . we should note that with each orientation
, the boundary condition distribution in figure 4 , left change , i.e. , our displacement boundary conditions are parameterized as a function of head orientation .
with respect to the influence of hyperosmotic drugs , we have chosen to exploit the second term in equation ( 2 ) .
our atlas allows for 3 different capillary permeability , i.e. , varying \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(k_{c}\ ) 
 \end{document } , with a fixed intracapillary pressure .
similarly , swelling variations were simulated with 3 different capillary permeability values , and positive intracapillary pressures , however , we did allow for 3 different craniotomy sizes ( 75% , 100% and 125% of planned size ) to account for any deviations from the plan .
we should note that boundary swelling , the boundary conditions associated with figure 4 , left , green region are modified to slip - based boundary conditions with stress free in the craniotomy region . for the work here , there were 729 total brain shift solutions contained within our deformation atlas .
the different material properties and their varying levels are tabulated in table 2 and are based on sensitivity studies performed within in vivo porcine brain experiments which we have found to be quite satisfactory for predicting bulk brain shift . 

the 729 finite element models were solved by spreading the computation across an 8-node computing cluster to ensure the atlas of solutions was built in time for the day of surgery .
the biphasic brain model was solved for displacement and pressure using an open source portable extensible toolkit for scientific computation ( petsc ) package .
a highly automated process of computational model generation , boundary conditions creation and solving for the atlas were developed to streamline and ensure minimal user error .
once complete , the deformation atlas is transferred to the intraoperative data collection and processing computer used in the or on the day of surgery . to facilitate real - time brain shift correction in the or
, an intra - operative pipeline was developed with 2 simple user - friendly guis to process the collected data from the lrs along with the precomputed atlas ( figure 1 , right side ) .
the registration gui developed is a registration and visualization utility that registers the patient s physical space to image space using an lrs scan of the face and the corresponding surface from the mr image volume .
the face lrs scan was acquired by positioning the lrs directly over face of the patient , making sure to include all if possible , the nose , eye and ear as these structures serve as good landmarks for use in registration .
the manual segmentation tool in the lrs acquisition software ( pathfinder therapeutics , inc . ,
nashville , tn ) was used to remove extraneous points in the face scan , such as hair , intubation tubes and drapes , that will unnecessarily slow down the registration computation ( figure 5 ) .
once the segmented face lrs scan data is complete , a smoothing process using a commercially available radial basis fitting ( rbf ) procedure is performed ( farfield technology , ltd . , christchurch new zealand ) .
to initialize , three surface fiducials are selected on the lrs data of the patient s face and the corresponding points are designated on the mr surface counterpart and a rigid registration using horn s method   is executed .
verification of the registration is confirmed by visual inspection of the overlay of both 3d objects ( figure 6 ) .
if the alignment of the two objects is not satisfactory , the user may select new points and execute another registration . 

figure 5the face rbf before ( left ) and after ( right ) manual segmentation to remove extraneous points . 

figure 6a screenshot of the registration gui with the face rbf on the right overlaid onto the mr image - based head surface mesh on the left .
the 3 homologous points on both surfaces are in green and they are used for the initial alignment .
the face rbf before ( left ) and after ( right ) manual segmentation to remove extraneous points .
a screenshot of the registration gui with the face rbf on the right overlaid onto the mr image - based head surface mesh on the left .
the 3 homologous points on both surfaces are in green and they are used for the initial alignment .
after the craniotomy and dura is opened , the lrs was moved into place to acquire the exposed cortical brain surface .
care was taken to ensure a direct line of sight between the brain surface and the laser .
once acquired , a simple manual segmentation tool is used to remove extraneous points , isolating just the brain surface ( figure 7 ) .
similar to the face scan , an rbf surface is fit and rigid transformations can be applied to transform the surface to image space . to confirm the positioning accuracy of the optically - tracked lrs
, the transformed pre - resection rbf scan was automatically overlaid onto the head surface mesh for visual inspection ( figure 8) . 

figure 7the pre - resection rbf before ( top ) and after ( bottom ) manual segmentation to remove extraneous points . 

figure 8a screenshot of the registration gui with the transformed pre - resection rbf overlaid over the mr image - based head surface mesh set to be semi - transparent .
the pre - resection rbf before ( top ) and after ( bottom ) manual segmentation to remove extraneous points .
a screenshot of the registration gui with the transformed pre - resection rbf overlaid over the mr image - based head surface mesh set to be semi - transparent .
multiple lrs scans may be taken during the course of the surgery depending on neurosurgeon s request to track the updated position of the tumor since the amount of brain shift is a function of time .
however , only one final cortical brain surface was acquired after tumor resection was thought to be complete and an image update would be useful to confirm complete removal of the tumor in the presence of brain shift .
since time is critical at this juncture as the entire surgical team is waiting for updated images , processing steps were specifically developed to minimize user intervention and computation time . instead of a full manual segmentation as done previously with the pre - resection lrs scan ,
a mask of the pre - resection lrs was applied to the post - resection lrs to remove points outside of the craniotomy region .
the fully segmented post - resection scan was also automatically fitted with an rbf surface , transformed to image space , and displayed along with the pre - resection rbf for visualization ( figure 9 ) . 

figure 9a screenshot of the registration gui with transformed pre- and post - resection rbf overlaid onto the mr image based head surface that has been made less opaque .
the post - resection rbf is below that of the pre - resection rbf illustrating brain shift .
a screenshot of the registration gui with transformed pre- and post - resection rbf overlaid onto the mr image based head surface that has been made less opaque .
the post - resection rbf is below that of the pre - resection rbf illustrating brain shift .
once the pre- and post - resection lrs cortical surfaces were spatially transformed to image space , the correction gui is used to determine the driving shift measurements for correcting the image volume for brain shift . to accomplish his , the 2d pre - resection and post - resection bitmaps , i.e. , texture information acquired by the lrs unit , were visualized side - by - side .
homologous points were then selected using blood vessel bifurcations as landmarks ( figure 10 ) .
figure 10a screenshot of the correction gui with homologous points in green selected at blood vessel bifurcations on the pre- and post - resection bitmaps .
a screenshot of the correction gui with homologous points in green selected at blood vessel bifurcations on the pre- and post - resection bitmaps .
once homologous points are selected from the texture information provided by the lrs , they can be related directly to their 3d coordinate positions .
we should note that brain shift is possible from the very instant the dura is opened . to accommodate for this initial shift
, a correspondence between brain mesh and intraoperative pre - resection lrs - defined features is determined using a closest point operator . once determined , the shift from pre - resection to post - resection lrs is appended and an entire deformation is complete . in the event that a very limited number of homologous points can be determined , the platform does allow for the calculation to be driven by closest point operators solely with the possibility of weighting from any homologous points that can be determined . for one patient in this study
, this feature was used due to a lack of homologous points . with a field of displacements describing cortical surface deformation defined
, the correction algorithm is employed that uses a constrained least squares inverse modeling approach based on the atlas and constrained by the measured displacement shift vectors of the cortical surface as well as added constraints to the coefficients of reconstruction .
briefly , the least - squared errors between the measured shift vectors and predictions from the deformation atlas were minimized by solving the following equations for the weighting coefficients , \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(w\ ) 
 \end{document}.\documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \begin{equation * } \min \big \|m\textit{w}-u\big \|^{2}\exists \textit{w}_{i}\geq 0~{\rm and}~\sum ^{m}_{i=1}\textit{w}_{i}\leq 1\end{equation*}\end{document } where \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(u\ ) 
 \end{document } are the measured shift vectors on the brains surface as determined by the above methods , and \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(m\ ) 
 \end{document } is the atlas matrix containing the pre - computed deformation solutions at the selected measurement points on the computer model boundary mesh .
the first constraint ensures only positive regression coefficients and the second constraint prevents extrapolations of the solution .
the implementation of the method of lagrange multiplier in the optimization toolbox in matlab was used to solve the linear optimization problem , along with the use of the parallelization toolbox to improve input / output function speeds .
we should note that while other optimization approaches with less constraint can lead to better objective function results , we have found that constraints such as the above are necessary to maintain physically realistic deformations , i.e. , a real safety constraint consider the dramatically underdetermined nature of this problem .
once the inverse solution is achieved , a quantitative report is automatically generated based on the optimum solution for assessment , specifically the amounts of shift based on measurements , and remaining error after correction . as equation ( 3 ) is solved within the context of matching the sparse measurements at the surface , the coefficients determined are then used to construct a full volumetric deformation field which is subsequently used to deform the preoperative mr images , thus providing an updated image of the deformed brain for use within the neuronavigation system . with respect to image deformation ,
nodal displacements from the undeformed finite element mesh were trilinearly interpolated onto a regular grid at the same resolution as the preoperative mr images .
to ensure there was no extrapolation of displacements outside the brain , the grid of interpolated displacements were multiplied with the binary brain mask .
each undeformed pixel was then transformed to their respective deformed pixel and filled with a trilinearly interpolated pixel intensity value from the undeformed mr images . since not every deformed brain pixel will be filled , the missing pixels intensity values were interpolated from its surrounding neighbors . the parallelization toolbox in matlab
was used to parallelize and speed up the interpolations of 3 cartesian components of displacements . a computer with intel quad core i5 with 16 gb of ram running windows
7 64bit was used to compute the intraoperative steps . the same computer was also used to acquire the lrs scans .
the computational timing to process each of the steps in the preoperative part of the pipeline for all 5 patients is tabulated in table 3 .
the model and atlas creation steps were significantly longer for patients # 2 , 3 and 4 because more extensive mesh refinement was needed to resolve tumors .
the total preoperative processing time ranged from 7 to 17 hours , with the majority of the time spent on creating the atlas . 

time taken to run the preoperative steps in the pipeline . the computational costs to register the patient space to mr image space using the registration gui for all 5 patients are listed in table 4 .
the lrs acquisition time was up to 4 minutes , including positioning of the apparatus , for all 5 patients . 

table 4time taken for registration using the registration gui from intraoperative steps in the pipeline .
the computational costs to produce the updated deformed brain image from when the post - resection lrs scans were taken for all 5 patients are listed in table 5 .
the maximum time reported for an updated deformed brain image to be computed during post - resection , including post - resection lrs acquisition times , was approximately 13 minutes and the fastest time was about 11 minutes ( table 5 ) . from the perspective of surgical workflow , the most prominent 
waiting period would likely be experienced during the computation of the updated image after homologous point picking .
the average wait time during this period would be approximately 4.5 minutes . in a realistic workflow setting , it is likely that the surgeon would be engaged during homologous point picking .
once this task was complete , the surgeon would be effectively waiting for an image update . summing across columns 5 , 6 , and 7 , in table 5 and taking the average ,
table 5time taken to run the post - resection lrs scan segmentation part of the registration gui and the correction gui from intraoperative steps in the pipeline .
time taken to run the post - resection lrs scan segmentation part of the registration gui and the correction gui from intraoperative steps in the pipeline .
the performance of the predictive computational model for all 5 patients is summarized in table 6 .
it includes the number of homologous points used in calculating the measured brain shifts between the pre- and post - resection lrs scans , percentages of shift corrected and the magnitude of corrected position errors .
the percent shift corrected follows the formula , ( 1  corrected error magnitude / measured shift magnitude ) \documentclass[12pt]{minimal } 
 \usepackage{amsmath } 
 \usepackage{wasysym } 
 \usepackage{amsfonts } 
 \usepackage{amssymb } 
 \usepackage{amsbsy } 
 \usepackage{upgreek } 
 \usepackage{mathrsfs } 
 \setlength{\oddsidemargin}{-69pt } 
 \begin{document } 
 } { } \(\times 100\ ) 
 \end{document}% , where corrected error magnitude is the error between measured and model predicted points , . despite the variability in magnitude of brain shift between the 5 patients ,
the updated images of the shift - compensated brains for all 5 patients are illustrated in figure 11 . 

table 6the measured and predicted brain shift correction results ( mean + standard deviation with maximum in parenthesis ) for all 5 patients . 

figure 11.original ( left ) and model updated shifted ( right ) brain images for all 5 patients .
the measured and predicted brain shift correction results ( mean + standard deviation with maximum in parenthesis ) for all 5 patients .
original ( left ) and model updated shifted ( right ) brain images for all 5 patients .
the objective of this study is to evaluate a preoperative and intraoperative processing pipeline developed for real - time brain shift correction using cortical brain surface deformation data only .
the complete process beginning with the positioning of equipment into the surgical field , data acquisition , inverse model processing , and image deforming was approximately 11  13 minutes across the five patients . within that time , the actual wait time to compute an updated image volume , where the neurosurgeon is not actively engaged in the workflow , is approximately 5.5 minutes .
the current workflow has been developed to be minimally cumbersome but better or design is very achievable and will serve to reduce total process time .
as hardware and software techniques continue to evolve , computation times are likely to improve . as a comparison to intraoperative imaging systems , including movable magnetic systems like the visius ( imiris , inc . ,
chanhassen , mn ) , the fixed magnetic systems like the brainsuite imri miyabi ( brainlab , inc . ,
westchester , il ) , and portable systems like the polestar n20 ( medtronic , inc . , minneapolis , mn ) , these systems all require at least the same amount of time if not more to position the magnet or patient , taking extra care to ensure anesthesia and ventilation are not interrupted .
in addition , after the deformed images are acquired , more computation time is required to nonrigidly register the new images to the preoperative images , , , , , .
even with the employment of imr techniques , one should expect algorithmic times similar to our approach to still be needed to align other forms of data .
time is also spent moving the magnet or patient back to the original surgical positions . breaking down the total 11 - 13 minutes of intraoperative setup and correction time yielded some interesting observations .
the tasks that took the longest time were the manual selection of the homologous points ( up to 2.25 mins ) , acquiring the lrs scan ( up to 4 mins ) and computing the deformed image ( up to 4 mins ) .
the length of time needed to deform the images was proportional to the number of slices and in - slice resolution of the patient s brain volume . to improve image deformation times , the computation was divided to run in parallel on 4 computer processing units ( cpu ) using matlab s parallelization toolbox . although cpu / gpu parallelization does improve computation time , there are alternative strategies to how deformation correction should be implemented within guidance systems . already some advances have been made . in , an alternative strategy where non - rigid deformations are compensated for in the localization of digitizers has been proposed which would eliminate the need for deformed image volumes .
that alone would reduce the wait time for surgeons by approximately 3 - 4 minutes .
the positioning and acquisition of the lrs scans took up most of the time during registration , pre - section and post - resection .
improvements can be made to the workflow by mounting the lrs scanner on the overhead articulating arms in the or , thereby allowing the scanner to be positioned and also withdrawn from the field quickly , resulting in less disruption to the surgical workflow .
alternate to lrs methodologies for point cloud generation , the use of stereo - pair reconstructed surfaces from the optics of the surgical microscope data would be an excellent way to reduce workflow problems , .
another opportunity to reduce interruption and reduce time would be to eliminate homologous point picking .
since video is available throughout the resection process , blood vessels on the cortical surfaces could be continuously tracked .
developed such a tracking feature although in   it was used in a retrospective analysis . combining vessel - tracking feature with stereo pair reconstruction from microscope data   could in effect
generate the same type of data as with the lrs but yet remain integrally contained within the microscope environment .
the preoperative processing time to create the atlas from mr images took from 7 to 17 hours in this study .
approximately 2 hours were spent on generating the patient - specific brain models with the majority of the preoperative processing time ( 6 to 14 hours ) spent on creating the 729 solutions in the atlas .
since preoperative mr images are typically acquired days before the surgery , there is clearly enough time to compute the atlas .
however , a recent sensitivity study on the size of the atlas solutions found that instead of 729 solutions used in this study , only a fraction , approximately 123 solutions could produce results with the same accuracy ( effectively a sparser sampling of the atlas ) .
the smaller atlas size means that construction time could be reduced to 2 hours , .
the biphasic biomechanical model - based brain shift correction accounted for 60%-88% of the shift , with a mean correction error of about 3 mm .
sources of error may be from image segmentation , finite element meshing , material properties , boundary conditions and registration .
additionally , the lrs scanner has a geometric error of 0.25 + / 0.40 mm and a tracking error of 2.2 + / 1.0 mm .
despite all these sources of possible error , the mean error of 3 mm is remarkably small .
although the majority of brain shift was accounted for in the biphasic biomechanical model , even higher accuracy could likely be achieved if the collapse of the tumor resection cavity could be included in the model .
the homologous points selected for use in the error analysis were from the cortical brain surface .
there is a lack of subcortical surface validation of the biphasic biomechanical model used . in a previous study by dumpuri et al .
, postoperative mr images were used with preoperative images to provide both surface and subsurface homogolous points to drive the same biomechanical model .
about 85% of the brain shift was recaptured in that 8 patient study , with remaining shift error less than 1 mm . while suggesting submillimetric correction accuracy , it must be noted that significant brain deformation recovery had taken place prior to post - operative imaging in this study ( up to 40% recovery in some instances ) .
nevertheless , the results from this study were promising and demonstrated the applicability of the biphasic biomechanical modeling approach .
the above system represents a cohesive approach to collecting , segmenting , and processing data with the result producing a  computationally altered image for improved navigation in image - guided procedures .
there are clearly limitations to the approach and room for improvement . in no area of imaging and image processing
has there been more development than that of the neurosurgical domain . the opportunity to develop sophisticated computer models with not only general anatomical information but also with complex structural information ( e.g. diffusion tensor imaging and elastography ) is attainable .
in addition , it is important to recognize that more sophisticated platforms for modeling are being developed that incorporate a variety of constitutive laws as well as interactive simulation conditions that include nonlinear effects ( e.g. sofa ) .
while we have chosen a linear platform here based on acceptable performance levels within the localization limitations of today s igs systems , this will undoubtedly change in the future with the evolution of more precise surgical systems ( e.g. robotic platforms ) . in this paper however , the work presented represents a baseline  systems level realization from which enhanced innovation can be realized . for example , challenges in space - occupying lesions and removal of tissue still persist and solutions are needed . while new data streams ( e.g. lrs and surgical microscope ) and interventional diagnostics ( e.g. optical spectroscopy , and fluorescence ) are on the horizon , new minimally invasive neurosurgical techniques will continue to provide challenges to presentation .
hardware and software developments bring enormous processing speed and enhanced computational architecture to the or , but workflow requirements and the ever - increasing wealth of preoperative information continue to expand and require improvements . as one looks at this contribution , it undoubtedly represents a 
snapshot of technology in time but is an important contribution emphasizing the characteristics that serve as constraints to data acquisition and guidance procedure execution , while also highlighting the potential for computation within the or .
it embodies the problem of extrapolating cost - effective relevant information from distinctly finite or sparse data while balancing the competing goals between workflow and engineering design , and between application and accuracy , a term we have called  sparse data extrapolation problem .
this paper clearly demonstrates that deformation compensated images can be computed intraoperatively using sparse data and biomechanical model approaches in near real - time for use in the or without the need for whole intraoperative imaging systems .
it also suggest that intraoperative computing is less significant than the workflow of equipment and data acquisition .
the work demonstrates the fabrication of logical and systematic preoperative and intraoperative pipeline that is robust , simple , and minimally disruptive ; perhaps in some cases , significantly less wieldy when compared to setup times for intraoperative imaging systems .
lastly , while a great deal of work towards these computational approaches has been achieved , more validation using 
gold standard imr measurement methods is needed , as well as long - term patient outcome studies .
 conventional image - guided neurosurgery relies on preoperative images to provide surgical navigational information and visualization . 
 however , these images are no longer accurate once the skull has been opened and brain shift occurs . to account for changes in the shape of the brain caused by mechanical ( e.g. , gravity - induced deformations ) and physiological effects ( e.g. , hyperosmotic drug - induced shrinking , or edema - induced swelling ) , 
 updated images of the brain must be provided to the neuronavigation system in a timely manner for practical use in the operating room . in this paper 
 , a novel preoperative and intraoperative computational processing pipeline for near real - time brain shift correction in the operating room was developed to automate and simplify the processing steps . 
 preoperatively , a computer model of the patient s brain with a subsequent atlas of potential deformations due to surgery is generated from diagnostic image volumes . in the case of interim gross changes between diagnosis , and surgery when 
 reimaging is necessary , our preoperative pipeline can be generated within one day of surgery . 
 intraoperatively , sparse data measuring the cortical brain surface is collected using an optically tracked portable laser range scanner . 
 these data are then used to guide an inverse modeling framework whereby full volumetric brain deformations are reconstructed from precomputed atlas solutions to rapidly match intraoperative cortical surface shift measurements . 
 once complete , the volumetric displacement field is used to update , i.e. , deform , preoperative brain images to their intraoperative shifted state . in this paper , 
 five surgical cases were analyzed with respect to the computational pipeline and workflow timing . with respect to postcortical surface data acquisition , 
 the approximate execution time was 4.5 min . 
 the total update process which included positioning the scanner , data acquisition , inverse model processing , and image deforming was  1113 min . 
 in addition , easily implemented hardware , software , and workflow processes were identified for improved performance in the near future . 
Introduction
Methods
RESULTS
DISCUSSION
CONCLUSION