in this paper , we proposed a novel graph - based registration framework , namely gm - icp , to align pairwise retinal images .
first , the retinal vessels are automatically detected and represented as vascular structure graphs ; a graph matching ( gm ) is then performed to find global correspondences between vascular bifurcations ; finally , the icp algorithm is used at fine level to register vessel shape models .
the image registration techniques become increasingly important in clinical human retina diseases diagnosis and treatment .
there are large demands for fast and fully automatic algorithms to register two or more retinal images which were taken at different times , from different views , and by different sensors .
register a sequence of images of the same retina can give a complete view of the retinal fundus region which may lead to early diagnosis and treatment of aids - cmv retinopathy . in laser retinal surgery ,
the treatment of exudative amd may cause highly 50% recurrence rate of cnv ( choroidal neovascularization ) due to inaccurate localization of the laser in fovea area .
real - time image registration method can be used to assist ophthalmologists with avoiding possible damages during the surgery .
retinal images registration is a challenging task . first , the retina is a curved surface ; nonlinear deformation may occur when using a weak - perspective uncalibrated camera .
third , retinal images may have large textureless regions with uneven illumination which make the extraction of retinal vessels very difficult .
fourth , two images taken long time apart or from diseased eyes can have physical changes in both structure and color of the retina . in the past ten years , many techniques
generally , these methods can be classified into two categories : vessel - based and nonvessel - based methods .
the nonvessel - based methods refer to a class of techniques that do not make use of retinal vessels directly during image registration .
these methods can be further subdivide into two classes : intensity - based and image descriptor - based methods .
the intensity - based techniques usually rely on image intensities and gradients , the registration is carried out through optimizing a certain similarity function , such as mean - square error , mutual information , and cross - correlation of images .
various of optimization method can be used for finding global optimum of the cost function , including downhill simplex , simulated annealing [ 4 , 5 ] , genetic algorithms , and so forth . in
, a multiscale elastic registration scheme was proposed to take into account retinal intensity variants based on optical flows .
the main limitations of intensity - based methods lie in two aspects : ( 1 ) these methods are highly reliant on the consistent intensities in two images and tend to fail due to the presence of nonuniform illumination and large textureless regions ; ( 2 ) the optimization may have huge searching space so that the computational cost becomes a bottleneck for intensity - based methods to be applied in clinic .
the registration methods with image descriptors [ 79 ] are feature - based and become popular recently . instead of optimizing a similarity function with whole image intensities ,
then , image registration is equivalent to finding correspondences between two feature sets . in , a generalized dual - bootstrap iterative closest point ( gdb - icp ) was proposed , in which sift local descriptor is used and the alignment process is driven by two types of keypoints : corner points and face points . in , to deal with multimodal registration problem , an edge - driven db - icp algorithm was developed by enriching the keypoint descriptor with shape context using edge points . in
, a new salient feature region descriptor was proposed for low quality retinal images registration .
the vessel - based methods utilize retinal vascular features as a basis for image matching .
currently , most of the retinal image registration approaches are vessel - based [ 3 , 1015 ] , because the retinal vessels are believed to be the most appropriate representation for retina .
in general , the vessel - based methods consist of two independent steps : vessel segmentation and vascular feature - based registration . once vessels were extracted
a dual - bootstrap iterative closest point ( db - icp ) algorithm was introduced to match vascular centerlines .
it started with some initial low - order estimations and iteratively refined the results by expending the bootstrap regions with higher - order transformation model . in order to make the matching robust to mismatches between feature points ,
hierarchical transformation models were employed in . in , a hybrid retinal image registration approach is proposed by combining both intensity - based and vessel - based methods . to achieve both global and local alignments ,
usually , the vessel - based method are supposed to be more reliable than nonvessel - based method because the vascular features are sufficient ( but not too many ) and accurate as landmarks and preferable spread over whole image region .
the main drawback of vessel - based methods is the local convergence problem when initial misalignment is large and mass of segmentation noises exist . in this paper
inspired by the fact that a retinal image may contain some unique geometric structures within its vascular trees .
we consider that the correspondence problem is not simply a one - to - one mapping problem between point sets but rather a pairwise ( structure ) matching problem between two separate graphs .
a brief overview of our approach is illustrated in figure 1 , in which the retinal feature extraction is preformed at first to generate a graph representation of the vessel networks .
afterward , a graph matching - based registration is performed to achieve both globally and locally alignment of the retinal images . to our knowledge ,
the graph - based approaches are relatively rare in the field of retinal image registration .
related work proposed recently were reported in [ 16 , 17 ] , where a graph transformation matching ( gtm ) algorithm was developed for retinal image registration and vascular characterization .
however , their method is limited in three aspects : ( 1 ) a good initial guess of correspondences is crucial to gtm ; ( 2 ) the number of graph nodes must be equal ; and ( 3 ) the vessels are detected in a semi - automatic way .
firstly , we demonstrated that the retinal fundus image registration problem can be efficiently solved within a graph - matching framework .
accordingly , we innovatively represent the retinal vascular features as an undirected graph model and a normalized vessel path distance measure is defined to characterize local similarities between two graph edges .
secondly , we proposed a hybrid registration framework that integrates graph matching and the icp algorithm together to work with a hierarchy of transformation models .
thirdly , we proposed a structure - based sample consensus ( struct - sac ) algorithm to eliminate incorrect correspondences from graph matching results .
this section presents the feature extraction methods used in our registration framework , including vessel centerline detection and graph - based representation .
retinal vessels are the most unique and prominent anatomical structure in the retina and provide rich information for the localization of distinct features .
a variety of techniques has been introduced for retinal vessel extraction , such as the hessian method , the gabor filter , matched filter , and parallel edge method . in our implementation , a single - scale matched filter is employed to detect retinal vessels because it provides better responses to thin and low - contrast vessels . although some multiscale techniques   can be used to achieve better vessel segmentation results .
however , those methods are quite time consuming and not necessary for the image registration task . blood vessels in retinal images usually have poor local contrast so that the matched filter is designed to detect and enhance the local piecewise vessel segments through image convolution . to approximate the vessel profile , gaussian function
is commonly adopted as the kernel of matched filter . however , in some cases , the intensity profile of some wide vessel segments is not gaussian due to vessel central light reflection effect .
hence , we use a gaussian - hermite mixture model described in   to approximate local vessel segment structures .
the 1-d second - order gaussian - hermite kernel is defined as 
 ( 1)h(x)=(1+a(x21))122e(x2/22 ) , 
							 where a is a weight parameter , that when a = 0 , the kernel is gaussian . in order to detect a vessel oriented in any directions , a set rotated kernels
because of symmetry , only 0 ~ 180 possible directions are required for computation . at each orientation
, the two - dimensional kernel can be represented as two one - dimensional kernels applied in succession according to cartesian separability rule .
let i(u , v ) be the image intensity at position ( u , v ) , a matched - filter response ( mfr ) of the input image at orientation  is defined as 
 ( 2)mfr(u , v,)=hvv(vx)h(uy)i(x , y,)dx  dy , 
							 where h(x ) is gaussian - hermite kernel in normal direction of  , hvv(x ) is second - order derivative of the gaussian - hermite kernel along the direction of . to make the implementation easier , the image coordinate is rotated at each orientation instead of rotating convolution kernels . finally , the matched filer is implemented by maximizing the responses over all orientations at each pixel , which can be expressed as 
 ( 3)m(u , v)=max  {mfr(u , v, ) } ,  0<180. 
   in our implementation , 12 matched filter orientations are used with a 15-degree increment . in order to avoid the influence of image noise , a histogram - match image filtering and a median image filtering
after the main vascular structures were detected , we used a threshold probing technique proposed in   to distinguish between enhanced vessels and the image background .
finally , vessel thinning is performed to the binary vessel image so that the resulting patterns consist of lines and curves with one pixel wide only .
once the retinal vessel centerline ( vessel shape ) model is calculated , the transformation into a graph is straightforward .
we construct a undirected vascular structure graph ( vsg ) g(v , e , a ) , where each graph node vi  v(1  i|v| ) represents a vascular bifurcation point or a vessel segment end point ; each graph edge ei  e(1 
the attributes set a carries some scalar numbers or vectors to characterize the local vessel segments . in our implementation , two distance measures were assigned to each graph edge : vessel path distance dvp and euclidean distance de between two linked graph nodes .
the vessel path distance measures the vessel path length between two vascular feature points in a binary vessel centerline image . to make the proposed method invariant to scale ,
we normalize dvp in the following way : 
 ( 4)a(i)=dvp(i)(1/|e|)k=1|e|dvp(k ) , 
							 likewise for de . in vsg ,
all the graph nodes are treated uniformly , we do not distinguish between the type of vertices on anatomical aspects .
first , the bifurcations of vessels are extracted using a template matching technique described in . then , we performed a parallel breadth first search ( bfs ) on the mono - pixel binary retinal vessel network to build edges among those bifurcation points and vessel segment end points
in this section , we will introduce the proposed gm - icp registration framework in details .
as mentioned above , we solve retinal image registration problem through a feature - based approach .
hierarchical features were utilized in our method , including retinal vessel shape models , vascular bifurcations and the underlying vascular topological structures , which were extracted previously by using the methods described in section 2 .
the key problem to register two retinal images in different coordinate positions is to establish correspondences between two feature sets .
then , a closed - form solution   can be applied to estimate the transformation parameters directly .
the simplest approach to the one - to - one correspondence problem is to define some similarity measure between two feature points , for example , the euclidean distance between sift descriptors .
and then the matching is carried out in a high - dimensional feature space following the nearest neighbor rule .
this approach will fail when the ambiguities exist , such as nondiscriminative local appearance or repeated patterns . however , these situations are quite common to retinal images .
another way to match two sets of points is to employ some heuristic techniques , such as icp   or em   algorithm , to iteratively align two feature sets until it converges .
these methods could be robust if the initial misalignment is small and the noise ratio is not high .
however , the features extracted by using current automatic retinal image segmentation techniques will always contain a mass of noises and outliers .
our method was motivated by the observation that a retinal image may contain some unique geometric structures within its vascular trees .
indeed , we are trying to make use of those distinctive structures and to enforce some geometric consistency between pairs of feature correspondences .
furthermore , we find that the graph - matching approach is a natural and powerful way for doing this task .
thus , we want to obtain points correspondence by taking advantage of structure information and the edge - to - edge comparison mechanism with graph matching .
more local invariant and expressive features with relatively low computational costs can also benefit from this .
in our implementation , the graduated assignment algorithm is employed to match vascular structure graphs extracted from two retinal images . in order to eliminate wrong matches from global correspondences
set obtained via graph matching , a structure - based sample consensus ( struct - sac ) algorithm is proposed .
this algorithm is more efficient and reliable than ransac   by replacing the random sampling procedure with a sequential local structure sampling .
once the global transform is known , we use a kdtree - based icp algorithm to register two vessel shape models to achieve better registration precision .
we use similarity transformation model for global correspondence calculation under the assumption which the apparent motion of the retina is generally rigid . to correct some nonlinear motions in the boundary area with low overlapping images , a quadratic polynomial model is adopted for vessel shape model registration .
we consider two vessel structure graphs g1(v1 , e1 , a1 ) and g2(v2 , e2 , a2 ) , which were extracted from reference image i1 and floating image i2 , respectively .
we do not assume that n1 = n2 because there may be different numbers of features to be matched .
the graph - matching problem is to find an optimal assignment between v1 and v2 that maximizes the sum of pairwise affinities between edges e1(i , j )  e1 and e2(i , j )  e2 .
this is equivalent to seek a n1  n2 assignment matrix m to maximize the objective function 
 ( 5)e = max  min1in2jn1jn2miimjjwiijj , 
							 where mii is equal to 1 if node vi in g1 corresponds to node vi in g2 .
wiijj carries potential matching weights corresponding to the pairs of points ( vi , vj ) of g1 and ( vi , vj ) of g2 .
a rectangle rule may better illustrate this edge - matching mechanism , which is shown in figure 4 . the problem ( 5 )
let us consider m as a n1  n2 binary vector x = { x | xii = 1  if  ii  m}. then , ( 5 ) may take the form as 
 ( 6)e = max  x  xtwx . 
							 in order to achieve one - to - one correspondences , an affine constraint is enforced to assignment matrix m as well as to assignment vector x 
 ( 7)imii=1 ,  imii=1 ,  mii{0,1}. 
  due to its combinatorial nature , to solve the graph matching problem in an exact way could be np - hard and it is not applicable to our vsg matching problem .
there are many approaches to inexact graph matching in the literature , such as the graduated assignment ( ga ) algorithm , spectral relaxation ( sm ) method , graph edit distance - based method , and shock graph grammar - based method . among these methods ,
the continuous relaxations approaches were thought to be the most successful ones . in this paper
, we employed a graduated assignment algorithm originally proposed by gold and rangarajan in   as our graph - matching framework .
our empirical evaluation shows that the method is efficient and reliable with vsg matching problem and even when outliers are present .
the ga method relaxes the discrete iqp into a continuous nonconvex quadratic program ( qp ) .
it optimizes a quadratic cost function through minimizing its low - order taylor expansion around previous solution .
one - to - one correspondence for graph nodes can be guaranteed via sinkhorn two - way constraint procedure .
thus , ( 5 ) can be reformulated as in following expressions under the ga framework 
 ( 8)e(m)=min  m12in1in2jn1jn2miimjjwiijj  + 1in1in2mii(log  ( mii+)1 )  s.t . 
ii=1n1mii1 ,  ii=1n2mii1 ,  mii0 , 
							 where m is a doubly stochastic matrix , in which the maximum element in each row and column represents a correspondence between two graphs nodes .
the xlog  x term in ( 8) is a smoothing function which can push the minimum of the objective away from the local discrete points .
we define the measure of compatibility wiijj between the links of two graphs as follows : 
 ( 9)wiijj={0 ,  e1(i , j )  or  e2(i,j)is  null,r=12r(11lr|a1r(i , j)a2r(i,j)|)otherwise , 
							 where l is the maximum edge weights in both graphs .
two edge weights are involved for pairwise affinity calculation , where a = dvp and a = de as defined previously .
empirically , we set  =  = 0.5 
 ( 10)lr = max  { max  ( a1r(i , j)),max  ( a2r(i,j))}. 
  from our experiments , we found that the ambiguities in the compatibility matrix may bring down the quality of graph matching in finding global correspondences .
an edge with small group of potential matches may help to clarify the optimal matching . to this end
, we employ a bistochastic normalization technique described in   to balance the compatibility matrix ( to enhance correct correspondences and to suppress spurious correspondences ) .
first , the compatibility matrix w is represented as a m1  m2 edge similarity matrix s , where m1 = |e1| and m2 = |e2| .
then , the edge similarity matrix s is normalized iteratively by using following expressions until convergence 
 ( 11)sij , ijt+1=sij , ijtklsij , klt , sij , ijt+2=sij , ijt+1klskl , ijt+1 . 
							
after the normalization was finished , the edge similarity matrix s is converted back to the compatibility matrix w   the global correspondences obtained by graph matching may contain a certain number of incorrect matches . in some extreme cases , the ratio of wrong matches can be greater than 80% .   to eliminate the wrong matches from global correspondences set
the ransac algorithm   is a well - known method to deal with outliers and has become a standard in the field of computer vision .
it first fits model parameters with randomly selected subsets of input data and then evaluates the quality of the parameters on the whole input data .
the process is performed repeatedly and terminated when the probability of finding a better model becomes lower than a user - specified threshold .
the main drawback of ransac is that most of the verified model parameters are incorrect under arbitrary random sampling .
this is inefficient and time consuming when dealing with large data sets .   in this paper
, we proposed a local structure - based sample consensus ( struct - sac ) algorithm , based on two simple observations :   correct matches between graph nodes are not isolated ;   most of the correctly matched graph edges are found next to each other .  
					
the phenomenon described above can be interpreted by the property of graduated assignment that it trends to approximately find the maximum common subgraph ( mcs ) during graph matching . as a result
, a good matched edge may have some good matched neighbors to support it . on the other hand
furthermore , we thought that the type of correctly matched structures in graph - matching results can be divided into two categories : stable structures and unstable structures ( as illustrated in figure 5 ) .
thus , the goal of our algorithm is simplified to find some stable graph structures .  
the struct - sac algorithm was motivated by the hypothesize - and - verify idea from ransac .
however , instead of random sampling , we select subsets of input data in a deterministic way by utilizing the local structures of a sample point .
find their corresponding ( the correspondence was obtained via graph matching ) nodes in other graph as the object subset .
( as an option , the ransac strategy can be used here to further distinguish the subsets from noisy structures when the number of neighbors is large . )
compute the transformation parameters determined by the selected couples and verify it first with a local geometric consistency test using partial data and then with global model consistency test using whole input data points .
the process is terminated when all the potential graph nodes were traversed and the best model parameter with lowest quality score is returned . in our implementation , the search is performed symmetrically and only on the nodes with 2 or 3 degrees . since these nodes are stable in vsg and provide sufficient information for the estimation of model parameters .
the effect of strac - sac is demonstrated in figure 6 . in local geometric consistency test
, we utilize pairwise edge similarity and triplewise orientation similarity of graph nodes to distinguish local structure samples . in global model consistency test ,
we use sum of squared distance ( ) to measure the quality of model parameter  between two point sets , as below : 
 ( 12)()=(p , q)c2(v1(p)t(v2(q), ) ) , 
							 where t denotes the similarity transformation function , c is a temporary correspondence set determined by transformation parameter .  is a huber kernel   to limit the influence of outliers 
 ( 13)(t)={12t2,if  ||t||,||t||122,otherwise . 
  since the retina is spherical and the retinal images captured with weak - perspective cameras may have nonrigid distortion in localized regions , higher - order transformation model is needed for accurate registration .
figure 7(a ) shows two fused retinal images under linear registration where some misalignment vessels were indicated with arrows . in figure 7(b ) , these vascular segments are well matched after a quadratic registration .
our registration algorithms employ a hierarchy of transformation models , which is also known as the global to local strategy , to account for the nonlinearities described above .
one advantage of hierarchical transformation is that it makes the algorithm robust to the mismatches caused by large interframe motions . in our method , the global registration is achieved via graph matching and struct - sac with similarity transformation model . at fine ( local ) level , we use a revised icp algorithm to register vessel shape ( centerline ) models .
a 12-dimensional second - order polynomial transformation model is employed in the local registration , that it maps a source point ( x , y ) in floating vessel model onto the target point ( x , y ) in reference vessel model as follows : 
 ( 14)[xy]=[a11a12a13a21a22a23][x2y2xy]+[a14a15a24a25][xy]+[a16a26 ] , 
							 where aij are transformation parameters .
once the correspondence is obtained at each iteration of icp , the quadratic motion can be estimated in closed - form by using a linear regression method described in .
in addition , several improvements are made to the standard icp algorithm in order to achieve better performance .  
( 1 ) to accelerate the closest point indexing , a kd - tree data structure is applied for underlying point sets modeling ; ( 2 ) the registration is performed in a multiresolution scheme ; ( 3 ) a point pair weighting strategy   is used to reject inconsistent matches .
in this section , we will evaluate the performance of the proposed gm - icp registration algorithm in two aspects : ( 1 ) saliency of the vascular structure graph , and ( 2 ) the overall performance of gm - icp in retinal image registration .
a total of 48 pairs of retinal images collected from clinical patients were involved for the algorithm evaluation . in order to make the evaluation consistent with images in different size , we resampled all test images to approximately 700  600  pixel using bicubic image interpolation .
our retinal image registration program was designed with matlab and some core algorithms ( such as graph matching and vascular bifurcations extraction ) were implemented in c - mex .
all the results reported in this paper were obtained on a pc with intel core 2 duo 2.33  ghz cpu and 3  g ram running under windows 7 professional .
the aim of our first experiment is to show the saliency and performance benefits of vascular structure graphs presented in this paper .
based on the vascular bifurcation points extracted with our methods , several spatial topological graph structures that commonly used in computer vision and graph - matching research were investigated in this study , including delaunay triangulation graph ( dt ) , minimum spanning tree of dt graph ( dt - mst ) , k - nearest neighbor graph ( knn ) , and minimum spanning tree of knn graph ( knn - mst ) .
the major difference between vsg and other graph representations is the anatomical properties of retinal vessels that vsg characterizes while the other graph representations do not provide .
three evaluation criterions are presented in our results : the average number of correctly matched graph nodes , the recall rate and the success rate of registration .
the recall value is defined as the number of correct matches with respect to the number of correspondences .
our experiment results denotes that the vsg representation for retinal images substantially outperforms other topological graph structures in graph matching .
in addition , we also plot recall rate against the percentage of structure ( graph node ) noise with gaussian curve fit of data in figure 9 .
as can be seen , the vsg has obtained relatively high matching rate even with greater than 80% of outliers . evaluating the accuracy of the retinal image registration is not an easy task because of lack of ground truth . in this section ,
we evaluate our algorithm with three assessment criteria : normalized correlation coefficient ( ncc ) ,   normalized mutual information ( nmi ) ,   vessel centerline error measure ( cem ) .  
					 the first two terms measure appearance difference between reference and floating retinal images based on intensity statistics .
the third term is a standard evaluation criteria that commonly adopted in the field of retinal image analysis [ 3 , 13 ] .
the cem is defined as median distances over all corresponding points between two vessel centerline models , expressed as 
 ( 15)median(pi , qi)c|pit(qi,)| , 
							 where pi and qi are two matched vessel points determined by correspondence set c. t( ,  ) denotes the final transformation function .
in addition , the success rate and run time performance was investigated in our experiments as well . in our settings ,
a manual validation was also performed in order to ensure the correctness of our results . in the second experiment
, we compare the performance of gm - icp with other two registration algorithms : the standard icp and gdb - icp .
the standard icp was implemented in matlab and we try to use it to register two - vessel shape models without initial alignment .
the gdb - icp   is a generalized version of db - icp which was originally proposed for retinal image registration .
we had obtained a public copy of its binary program from the internet and tested it under the quadratic transformation parameters setting . in order to assess the reliability of the graph matching algorithm
, we also performed some image registration tests using the graph matching ( gm ) alone without icp .
the results of comparative experiments on 48 retinal image pairs are listed in table 2 .
as can be seen , the registration accuracy of gm - icp ( 0.81 pixels in cem ) is very close to gdb - icp ( 0.79 pixels in cem ) according to three error assessment criteria .
both of the two algorithms have achieved subpixel accuracy . in absence of fine level alignment , the registration with only graph - matching is still robust and obtained acceptable results ( 1.00 pixels in cem ) .
the gm - icp successfully registered 95.83% of all pairs of test retinal images , two pairs failed due to poor quality of vessel segmentation from pathology images whereas gdb - icp registered 97.92% and only one pair failed .
it takes about 8 seconds on average to register a pair of images with gm - icp , including the time cost in both feature extraction and registration , while gdb - icp needs more than 20 seconds .
in contrast with above three methods , the performance of standard icp is not satisfactory in both registration accuracy and success rate . besides , it is a little surprise to see that the ncc and nmi assessment of standard icp is even better than the other three methods .
it denotes that the intensity - based global similarity measures for retinal images are nonconvex and sometimes become unreliable as a result of the presence of large - scale misalignment and inconsistent patterns . in most of the cases ,
the performance of registration will be gradually degraded with less overlap of pair of images .
we plot the vessel centerline error measure against the percentage of image overlap , as shown in figure 10 .
it is observed that gm - icp is accurate and robust with medium overlapped image pairs . due to the fact that the graph matching scheme does not rely on direct distance metric between two feature sets ,
the gm - icp registration framework is actually invariant to any linear geometric transformations , such as large - scale initial translations and rotations .
however , our method may fail with some low overlapped ( under 40% ) image pairs due to lack of enough consistent vascular segments for matching .
one difficulty to all vessel - based registration methods is that the reliability of registration is highly influenced by the vessel segmentation results . in our method , there are serval ways to deal with segmentation noise and structure inconsistency .
first , small edges were removed during vascular graph representation according to some vessel network topological modification rules .
second , a structure - based sample consensus algorithm was proposed to eliminate incorrect matches in global registration .
third , a point pair weighting strategy is used to reject inconsistent matches during fine level registration . the performance of gm - icp for the images with high structure noise and pathologies is visually demonstrated in figure 11 .
in this paper , we proposed a graph - based algorithm framework for retinal image registration .
hierarchical retinal features were utilized in the framework , including vessel shape models , vascular bifurcations and the underlying vascular topological structures .
the core idea behind our method is to obtain the point correspondences by taking into account structure information provided by human retina and retinal vessels .
thus , the image registration problem is transformed into a pairwise ( edge - to - edge ) correspondence problem .
accordingly , we have employed a robust graph matching framework , the graduated assignment with normalization on compatibility matrix , to match two retinal vascular structure graphs . in order to make our method robust to structure noise
, the struct - sac algorithm is proposed to eliminate incorrect matches from graph matching results .
the effectiveness of our method has been demonstrated experimentally with 48 pairs of real retinal fundus images .
the vsg representation for retinal vessel network is distinct , preferable spread and low computational cost .
additionally , it reflects the anatomical characterizations contained within a retinal image , which is prominent and almost unchangeable .
our experiments shows that the vsg model is robust even with high structure noise and substantially outperforms other regular topological structures for graph matching .
the most significant distinguishing property of our method is the structure matching and verification scheme for retinal image registration .
although graph matching have proven to be useful in many applications , such as character recognition , shape analysis , however , it is relatively rare for medical image registration .
the reasons are twofold : ( 1 ) structure representation from image is not straightforward ; ( 2 ) structure matching is not robust under high noise level . in our method ,
sufficient consistent structures are extracted from retinal images with proper selection of vessel segmentation and graph representation methods .
the adopted graph matching framework incorporating with struct - sac works fine with retinal vascular structure graphs .
heavy local feature descriptors are not required ( actually , only two simple distance measures are involved in the computation ) .
furthermore , it gives a convenient and practical way to distinguish between correct and incorrect structure correspondences .
the main limitation of the proposed method is the assumption that there are enough consistent vascular segments ( edges ) for matching .
however , with some low overlapped or poor quality pathology image pairs , this assumption is not satisfied . as a result
, the performance becomes poor ( the number of stable graph nodes is less than 3 ) and even fails . in this paper , we only addressed pairwise structure matching problem with retinal image registration .
it would be interesting to cope with some higher - order graph matching methods   in our framework . as a further development
, the registration accuracy can be improved by using more efficient and reliable vessel segmentation techniques .
also , our future work will focus on developing more robust graph representation for retinal features , which can accommodate both geometric and photometric invariants in an image .
 motivated by the observation that a retinal fundus image may contain some unique geometric structures within 
 its vascular trees which can be utilized for feature matching , in this paper , we proposed a graph - based registration 
 framework called gm - icp to align pairwise retinal images . 
 first , the retinal vessels are automatically detected and 
 represented as vascular structure graphs . a graph matching 
 is then performed to find global correspondences between 
 vascular bifurcations . 
 finally , a revised icp algorithm incorporating with quadratic transformation model is used at 
 fine level to register vessel shape models . in order to eliminate the incorrect matches from global correspondence 
 
 set obtained via graph matching , we proposed a structure - based sample consensus ( struct - sac ) algorithm . 
 the 
 advantages of our approach are threefold : ( 1 ) global optimum solution can be achieved with graph matching ; ( 2 ) 
 our method is invariant to linear geometric transformations ; and ( 3 ) heavy local feature descriptors are not required . 
 
 the effectiveness of our method is demonstrated by the experiments with 48 pairs retinal images collected from 
 clinical patients . 
1. Introduction
2. Feature Extraction
3. The GM-ICP Algorithm
4. Experiments
5. Conclusion