liver fibrosis is an important cause of mortality and morbidity and contributes substantially to increase health care costs in patient with chronic liver diseases .
fibrosis can lead to cirrhosis , for which the complications such as hepatic decompensation , hepatocellular carcinoma , and portal hypertension involve growing public health concerns .
cirrhosis and chronic liver disease were the 10th leading cause of death for men and the 12th for women in the united states in 2001 , leading to the death of about 27,000 people each year .
cirrhosis was first considered as an irreversible process , but , with the growing understanding of hepatic fibrogenesis mechanisms , more effective treatments have been developed [ 3 , 4 ] .
however , the latter must be initiated at a specific and early stage in fibrous development , and their administration requires regular clinical followup . while histological analysis after liver biopsy is the gold standard for the diagnosis , inherent risk of a recognized morbidity and mortality renders this method unsuitable for clinical monitoring [ 5 , 6 ] .
it has been demonstrated that perfusion imaging has the potential to detect and assess vascular modifications   associated with liver fibrosis .
several studies , using magnetic resonance dynamic contrast - enhanced imaging ( mr - dce ) to quantify liver perfusion , have shown that some perfusion parameters were relevant indicators for liver fibrosis assessment [ 1012 ] . in a previous work ,
an mri protocol associated to a dedicated processing step to quantify liver perfusion was developed [ 12 , 13 ] .
results demonstrated that mr perfusion imaging could be a noninvasive method for the clinical followup in patient with chronic liver diseases .
nevertheless , the evaluation was restricted to an roi , and regional variations often met in diffuse liver diseases could not be observed .
roi - based perfusion quantification already requires heavy processing methods such as image registration , denoising , and data fitting .
processing time drastically increases and becomes really prohibitive for clinical application for 2d or 3d mapping . in this context ,
parallel computing on distributed infrastructures such as clusters , grids , or clouds proves to be an interesting solution .
such infrastructures can bring significant speedup for a large spectrum of applications from various scientific domains .
they have already been used for medical imaging as described in [ 14 , 15 ] but never before for 3d - liver perfusion mapping .
nevertheless , new applications still require extra work for adapting ( porting ) them on the considered infrastructure .
this work describes an mr acquisition protocol and a processing method using distributed computing on the european grid infrastructure ( egi ) to allow 3d liver perfusion parametric mapping after mr - dce imaging with the ms-325 blood pool agent .
processing speed , reproducibility , and accuracy were assessed and adequate acquisition requirements were defined .
the study protocol was approved by the local experimentation ethics committee , and informed consent was obtained from each patient .
seven subjects ( 4 women , 3 men ; average age , 40  12 years ; mean weight , 75   8  kg ) were enrolled . among this group ,
one healthy subject was used as control and six patients with chronic liver diseases were prospectively enrolled ( maximum prospective period of one month ) after having had a liver biopsy .
tissue samples were fixed in buffered formalin and embedded in paraffin . 4  m - thick sections were stained with hematoxylin - eosin - saffron , iron stain , and masson trichrome reagents and evaluated by two pathologists .
fibrosis was evaluated on trichrome - stained slides according to the metavir classification   ( score f0 : absence of fibrosis ; score f1 : portal fibrosis ; score f2 : portal fibrosis with isolated bridges score f3 : fibrosis with numerous bridges without cirrhosis ; score f4 : cirrhosis ) .
acquisitions were performed using a siemens magnetom symphony maestro class 1.5 t imaging system ( siemens medical solutions , erlangen , germany ) .
a t1-weighted vibe 3d sequence with a parallel imaging technique was used ( grappa , r - factor = 2 ) .
the sequence parameters were as follows : te / tr/ , 1.22/2.87  ms/12 ; k - space partial filling , 6/8th according to slice and phase direction ; reduction of the slice and phase encoding step , 63 and 50% , respectively .
the plane was coronal oblique with a rectangular fov ( 400  300  mm ) for a rebuild matrix of 256  192 pixels with right / left phase - encoding direction .
the rationale behind the use of coronal imaging was to minimize the flow - related enhancement of the aorta .
the exploratory volume was acquired with a 1-sec temporal resolution with 6.4  cm slab thickness ( 16 slices of 4  mm ) .
the signal was collected using two circularly polarized phased array coils ( cp body array and cp spine array ) with a bandwidth of 650  hzpixel .
acquisition has begun at the time of injection of the contrast medium ( ms-325;epix pharmaceutical , inc . ,
lexington , ma , usa ) , and it continued for 2 minutes [ 12 , 13 ] .
injection was performed with an injection rate of 1  mls , a posology of 0.03  mmkg , and flushed with 25  ml of physiologic saline injected at the same rate .
finally , sixteen 2d + t volumes with t = 120 were acquired leading to 1920 images per examination .
images were first imported on a personal computer running an in - house developed application written in matlab r2010a ( the mathworks , natick , ma , usa ) . due to free - breathing acquisition , spatial shifts linked to motion had to be corrected .
this method consisted in the estimation of the transformation vector needed to register each moving images in relation to a static reference image .
for each image from 2d  +  t  volumes , a pixel - based method was used ( iconic approach ) to control the transformation of an input image .
an error measure was used to measure the registration error between the moving and static image .
the reduced - memory broyden - fletcher - goldfarb - shanno ( bfgs ) quasi - newton algorithm was used to move the control points to achieve affine rigid registration between both images with a minimal registration error
native 2d  +  t volumes were converted in to 2d +  t ms-325 mass concentration maps from a pixel - by - pixel operation based on the relationship between signal intensity and ms-325 concentration .
thirdly , native arterial and portal input functions , ca(t ) and cp(t ) , were measured using squared rois of 25 pixels placed by an experienced radiologist ( f.p .
12 years of postgraduate experience in digestive imaging ) at the level of the abdominal aorta close to the cliac trunk and the main portal vein . finally , definitive arterial and portal input functions were converted into continuous form ( function of the time ) instead of vectorial form ( discrete form ) , by an interpolation using spline curves .
hepatic capillary system was modeled by a 3-parameter one - compartment pharmacokinetic model adapted to hepatic dual supply ( portal and arterial ) .
the leakage of tracer through venous washout is carried out exponentially over time and the inverse of the constant of elimination is the mean transit time ( mtt ) . the equation describing this model is as follows : 
 ( 1)c(t)=[ca(ta)a+cp(tp)p]et / mtt , 
 where   designates the convolution product and  the volemic mass considered to be equal to 1  gml .
the parameters ,  a and p , are the arterial and portal perfusion , respectively , expressed as ml100  gmin .
ca and cp , are respectively , the arterial and portal input functions . the two delays , a and p , take into account the temporal offset between central compartment input and measured input from arterial and portal rois .
while  a ,  p , and mtt are model parameters , delays are independent of the fit procedure . the hepatic perfusion index ( hpi ) , defined as the arterial perfusion to total perfusion ( arterial + portal perfusion ) ratio , was also calculated . for each part of the image , pixel - by - pixel tissular time activity curves were obtained and a nonlinear least - square fit was performed according to the model previously described(1 ) using the levenberg - marquard algorithm . because some coefficients are closely connected , in particular portal perfusion and arterial perfusion with arterial and portal delay , the results of optimization were strongly influenced by the choice of starting coefficients , and algorithm may converge to local minima . in order to improve the robustness and reliability of optimization , but also to avoid any convergence to local minima , the algorithm needed to be started with a grid of pseudorandom starting points generated within two bounds ( multistart technique ) .
so , each fit procedure was done two - hundred - fold , with two - hundred different initializations .
for each fit procedure , delays were determined as the time between the beginning of tissular enhancement and the beginning of arterial enhancement in celiac trunk .
these starting points are chosen as the maximum of second - order derivative of tissue time activity curve and arterial input function . from this step ,
three perfusion parametric maps were obtained , one for each parameter of the model used .
the processing step was parallelized and executed on egi within the biomed virtual organization ( vo ) .
the parallelization was handled at the input data level , by splitting each volume into several pieces .
each piece was processed by independent jobs running in parallel on multiple grid resources and eventually merged .
the whole processing operation was modeled and implemented as a grid workflow using the gwendia language   and the moteur workflow engine .
the splitting and merging algorithms were developed in c++ , while the processing algorithm was developed in matlab .
all three programs were compiled on a grid compliant operating system ( centos ) and deployed on the fly on the grid nodes . for the matlab code
the interface with the grid resources was provided by the vip web platform ( https://vip.creatis.insa-lyon.fr/ ) .
a specific cartography workflow was developed for this application and integrated into the vip platform .
the user uploaded the input volumes on the grid and launched the processing workflow from a web portal . in order to evaluate the speedup provided by our parallel approach , the total cpu time to make span ratio was determined .
the makespan was defined as the time elapsed between the launch and the completion of the workflow , and the total cpu time as the sum of cpu times of all jobs in a workflow . in order to evaluate the reproducibility of our distributed computing algorithm ,
relative standard variation ( coefficient of variations ) was then mapped for each parametric map for all patients and defined as the standard deviation to arithmetic mean ratio .
next , to evaluate the accuracy of our method , results between roi - based quantification method described in   and the method presented in this paper were compared .
the difference between methods was evaluated using the bland - altman representation for each perfusion parameters , the spearman 's coefficient calculation , and the nonparametric wilcoxon test .
among the 6 biopsied patients , histological results were as follows : 2 patients were scored f0 , 3 patients scored f2 , and 1 patient scored f4 .   a representative set of 2d parametric maps extracted from 3d volumes on the healthy patient ( metavir f0 ) is shown in figure 1 .
then , parameter values were stratified according to the fibrosis severity . results corresponding to the advanced stage ( metavir stage  f2 ) and early stage ( metavir stage < f2 ) are presented in table 2 .
a significant correlation was observed between roi - based method and distributed method for each parameter .
spearman 's coefficients (  ) were 0.86 , 0.92 , and 0.80 ( p < 0.01 ) for arterial perfusion , portal perfusion and mtt , respectively .
regarding the wilcoxon test and the bland - altman representations ( figure 3 ) , significant difference was shown between compared methods .
however , bland - altman representations showed a systematic decrease of mtts values calculated with distributed method compared to roi - based reference method .
the major drawback of the perfusion - based method , its prohibitive computing time , has been overcome with the help of the egi . by using the resources of distributed european infrastructure , 1 cpu year ( corresponding to twenty - one 3d mapping procedures )
the speedup varied among the 21 workflows from 20 to 94 with an average value of 48 . the average error ratio for the experiments presented here was of 18% , with a maximum of 43% for one of the workflows .
as shown in figure 2 , the late completion of the last jobs significantly increases the makespan .
the workflow needs to wait for the completion of all jobs in order to produce the final result .
thus , for the experiments presented here , the makespan was almost tripled ( from roughly five to fifteen hours ) because of these late jobs .
similar results have been reported in other studies such as , where the authors propose a dynamic load balancing approach in order to improve performance .
nevertheless , the dynamic load balancing approach proposed in   only works for monte - carlo - based simulations .
the presented method was reproducible and results were correlated with roi - based reference method run locally on a personal computer .
the quantified parameters were found to be in the same range as those obtained with the reference roi - based method and those related in the literature[9 , 10 , 12 ] .
nevertheless , patient - wise , quantified values were slightly modified even if the shift was not found significant .
indeed , compared to the roi - based method , blood flow quantified with the presented 3d method is overestimated whereas ; on the contrary , mtts are underestimated . additionally , for each parameter , standard deviation observed with 3d methods run on egi was found lower compared to the roi - based method . when results are globally stratified according to fibrosis severity , the difference between mean values for each parameter computed with presented method is systematically lower than with roi - based method .
these findings confirm the smoothing effect induced by 3d quantification algorithm . indeed , in roi - based estimation method , arterial and portal delays are optimally set by user .
however , these delays depend on spatial location and take into account the time shift between the measured input functions and the position where modeling takes place in parenchyma .
hence , manual setting is not possible in the 3d case , and an automatic estimate of both delays was mandatory . due to relatively low image signal - to - noise ratio ( snr ) of about eighteen , this step requires hard smoothing filtering , affecting quantification results with the acquisition data currently available .
another limitation is the restricted exploration volume . indeed , to keep an acceptable snr acquired with a high temporal resolution of 1  sec , the number of encoding steps in the slice - encoding direction was limited and the whole liver volume was not covered .
these restrictions ( snr and coverage ) can be overcome with the latest imaging mr systems with improved acquisition capabilities using 32 receiver channels with multiple element array coils .
an snr value of 80 was measured based on preliminary test performed at our institution with a 3 t gehc mr 750 ( gehc , milwaukee , wi , usa ) with 32  ch body coil .
the parallelization of the method brings significant speedup and renders it feasible despite its prohibitive computing time .
currently the poor scheduling of the last tasks is largely due to platform heterogeneity and multiple task resubmissions caused by high error ratios .
data transfers account for most of the errors , while the rest are mostly application failures due to improper grid node configuration .  
as future work , scheduling will be improved by taking into account these considerations .   to conclude , this preliminary study demonstrated that the described method allows 3d liver perfusion quantification within a reasonable processing time .
it is now suitable to be used for similar clinical studies in a research context .
while the distributed processing method was validated compared to the roi - based quantification , such fully automatic processing requires high - quality images .
the required snr , together with a high temporal resolution and large volume exploration , can now be achieved on the latest 3 t mri systems available .
further work will have to demonstrate the interest of parametric 3d perfusion maps for fibrosis assessment on a larger number of subjects with chronic liver disease .
 an mr acquisition protocol and a processing method using distributed computing on the european grid infrastructure ( egi ) to allow 3d liver perfusion parametric mapping after magnetic resonance dynamic contrast enhanced ( mr - dce ) imaging are presented . 
 seven patients ( one healthy control and six with chronic liver diseases ) were prospectively enrolled after liver biopsy . 
 mr - dynamic acquisition was continuously performed in free - breathing during two minutes after simultaneous intravascular contrast agent ( ms-325 blood pool agent ) injection . 
 hepatic capillary system was modeled by a 3-parameters one - compartment pharmacokinetic model . 
 the processing step was parallelized and executed on the egi . 
 it was modeled and implemented as a grid workflow using the gwendia language and the moteur workflow engine . 
 results showed good reproducibility in repeated processing on the grid . 
 the results obtained from the grid were well correlated with roi - based reference method ran locally on a personal computer . 
 the speed - up range was 71 to 242 with an average value of 126 . in conclusion , distributed computing applied to perfusion mapping brings significant speed - up to quantification step to be used for further clinical studies in a research context . 
 accuracy would be improved with higher image snr accessible on the latest 3 t mr systems available today . 
1. Introduction
2. Materials and Methods
3. Results
4. Discussion