epilepsy is a neurologic disorder that affects ~70 million people worldwide ( ngugi et al . , 2010 ) .
typically , seizures originate in a focal brain area , called the seizure onset zone ( soz ) , and epilepsies are eventually classified based on the location of soz in the brain .
the most common type of epilepsy is temporal lobe epilepsy , accounting for about 65% of focal epilepsy cases ( tle ; engel , 2001 ; tellez - zenteno and hernandez - ronquillo , 2012 ) .
about one third of tle patients do not respond to medication , and become candidates for surgical resection of the epileptogenic tissue .
the success of surgery depends on the correct identification of the soz in each patient . besides eeg and neuropsychologic evaluation
, a repertoire of neuroimaging tools are frequently used to improve soz detection , among which mri , pet , and diffusion imaging ( bonilha and keller , 2015 ; sperling et al . , 1992 ;
the various neuroimaging modalities are reviewed by expert radiologists who visually inspect the images in search of pathology . when found , asymmetric pathologies are a good indicator of the presence of the soz , providing a good prognosis for improved seizure control once the pathologic area is resected ( blum et al . , 1998 ; englot and chang , 2014 ; gok et al . , 2013 ;
the most common pathologies observed in tle are : an asymmetric reduction of glucose metabolism ( known as hypometabolism ) in one of the temporal lobes , an asymmetric atrophy of mesial temporal lobe structures ( known as mesial temporal , or hippocampal , sclerosis ) , or other local morphologic anomalies such as dysplasias and heterotopias .
besides pathologies related to gray matter ( gm ) , white matter ( wm ) anomalies have also been reported in tle , some of which are asymmetric and can help localize the soz ( ahmadi et al . , 2009 ; concha et al . , 2012 ; thivard et al . ,
the ability to identify pathologies from neuroimaging data requires many years of training and additional dedicated time in routine clinical practice .
even with the right training and expertise , the accuracy of human experts may depend on other subjective factors , such as the amount of clinical knowledge the expert has about the case ( thivard et al . , 2011 ) .
recent developments in neuroimage analysis have shown that automated quantification and prediction can be a powerful and promising tool in medical practice ( for a review , see gabrieli et al . , 2015 ) . when applied to epilepsy
, early studies showed that automated neuroimage analysis and patient classification can match or exceed the accuracy of visual inspection of human experts ( ferrie et al .
more recently , several research papers have built predictive models that can distinguish among subtypes of epilepsy patients based on pet data ( didelot et al .
2012 ) , mri data ( duchesne et al . , 2006 ; moser et al . , 2000 ) , or a combination of pet and mri ( lopez - acevedo et al
the power of dti has also been investigated for automated classification either as a single modality ( ahmadi et al . , 2009 ; concha et al . ,
2012 ) or in combination with pet and mri ( thivard et al . , 2011 ) .
this knowledge , however , has not been readily translated into tools of clinical utility . moreover , the availability of many neuroimaging protocols raises the question to whether the combination of multiple imaging modalities into a single predictive model can improve accuracy rates above and beyond what is obtained from each modality separately .
the aim of this study was to provide a comprehensive investigation of data obtained from three imaging modalities with the intent of creating an automated classification algorithm with clinical applicability , capable of distinguishing left from right temporal lobe epilepsy ( ltle and rtle , respectively ) .
we aimed at creating a predictive tool that does not simply classify patients , but also provides a probabilistic index of the classification .
in addition , we investigated whether the relationship between gm , wm , and metabolism , is the same in ltle and rtle patients .
we utilized asymmetry measures from all modalities for three main reasons : ( i ) human experts frequently look at degrees of asymmetry when inspecting neuroimaging data , ( ii ) asymmetries use each subject as a control for itself , thus decreasing confounds coming from differences in scanner parameters , time of scan , metabolic factors , etc .
, and ( iii ) asymmetries have been shown to be more sensitive to the pathology than raw voxel values ( didelot et al . , 2010 ;
a total of 58 patients with refractory , unilateral and focal temporal lobe epilepsy were recruited from the thomas jefferson university comprehensive epilepsy center ( 28 left tle ; 30 right tle ) .
all patients had solely unilateral temporal pathology , and all were evaluated for surgical resection of the epileptogenic area as treatment for their intractable temporal lobe epilepsy .
details of the thomas jefferson comprehensive epilepsy center algorithm for surgical decision making are described in sperling et al .
( 1992 ) . a combination of eeg ( at least 96  h ) , video recording , mri , pet , and neuropsychological testing , was used to localize the seizure focus .
expert board - certified neuroradiologists , epileptologists , and neuropsychologists , by consensus decision , classified patient neuropathology and seizure type .
when the data from the various tests was not sufficiently informative , intracranial electrodes were implanted to determine the seizure focus through electrocorticography ( 12 patients , 5 ltle ) .
participants were excluded if they were missing data from one of the modalities ( mri , dti , or pet ) .
other exclusion criteria consisted of medical illness with central nervous system impact other than epilepsy ; prior or current alcohol or illicit drug abuse ; extratemporal epilepsy ; present or past neoplasia ; contraindications to mri ; psychiatric diagnosis other than an axis - i depressive disorder ; or hospitalization for any axis i disorder listed in the diagnostic and statistical manual of mental disorders , iv .
depressive disorders were allowed in the patient sample , given the high co - morbidity of depression and epilepsy ( tracy et al . , 2007 ) .
patients with mental retardation ( full - scale iq  <  70 ) who were likely to be unable to cooperate with the mri examination were also excluded .
the study was approved by the thomas jefferson university institutional review board for research with human subjects , and consent was obtained from all patients .
table 1 displays the mean values and frequencies for the relevant demographic and clinical characteristics of our sample .
images were acquired in a philips achieva 3 t scanner ( amsterdam , the netherlands ) using an 8-channel sense head coil . the mp - rage and the dti sequence
no single group was scanned on a schedule or time different than the other groups , thereby avoiding bias related to temporally dependent scanner calibration .
the mp - rage volume was collected in sagittal orientation with in - plane resolution of 256    256 and 1  mm slice thickness ( isotropic voxels of 1  mm ; tr  =  650  ms , te  =  3.2  ms , fov 256  mm , flip angle 8 , sense factor  =  1 ) .
the diffusion data were acquired using a single - shot spin - echo epi pulse sequence ( te  =  90  ms , tr  =  8609  ms , sense factor  =  2.5 , 5  min acquisition ) with 32 diffusion weighted ( b - factor  =  850  s / mm , anterior  posterior fold - over direction ) and three non - diffusion volumes ( b - factor  =  0  s / mm ; averaged in scanner to a single b0 volume ) .
each volume contained 66 slices ( thickness  =  2  mm , gap  =  0  mm ) acquired in the axial plane , with a reconstructed matrix size of 128    128 and fov 230  mm , resulting in voxel size 1.8    1.8    2  mm .
fat suppression was achieved using a standard spir ( spectral pre - saturation with inversion recovery ) technique .
the sequence was repeated twice for each subject and the datasets were averaged to increase signal to noise ratio .
pre - injection blood glucose level was below 150  mg / dl for all patients ( range : 61128  mg / dl ) .
an intravenous catheter was inserted under local anesthesia and a dose around 5.9  mci ( std 1.42 ) of radioactive 100  mg / l fludeoxyglucose ( fdg ) was injected .
the scan was initiated about 42  min ( std 14.8 ) after the injection .
the eyes were open , the ears were non - occluded , and ambient noise and light was kept to a minimum .
forty - seven patients ( 81% of patients ; 24 ltle , 23 rtle ) were scanned on a siemens biograph 1080 camera ( siemens medical solutions , erlangen , germany ) , with data consisting of 109 axial slices , 3  mm thick , and 1    1  mm in resolution ( scan dates 20082013 ) .
the remaining 11 subjects were scanned on two different cameras ( scan dates 20062008 ) , a siemens biograph cps 1080 with voxel size 1    1  mm ( 109 slices , 2  mm thick ) , and a philips allegro pet body ( philips medical systems , best , the netherlands ) with voxel size 2    2  mm ( 90 slices , 2  mm thick ) .
note , we obtained asymmetry indices that use the same subject as reference , therefore reducing potential scanner specific bias .
this procedure also avoids confounds related to demographic factors , such as , age , medication history , epilepsy history , etc .
nevertheless , a cross validation of results between different scanners was performed ( results section ) .
the rationale for considering cortical thickness as a potential predictor of soz laterality followed the reports of several groups who found more widespread atrophy in ltle than in rtle ( bonilha et al . , 2007 ; keller and roberts , 2008 ; kemmotsu et al . , 2011 ) .
two popular software were used to estimate cortical thickness in order to cross - validate potential findings and to investigate differences arising from estimation methods : freesurfer v.5.3.0 ( http://freesurfer.net ; fischl and dale , 2000 ) and advanced normalization tools v.2.1.0 ( ants , http://stnava.github.io/ants ; avants et al . , 2011 ) .
in freesurfer , a fully automated thickness estimation was obtained from the mp - rage volume using default parameters and the  recon - all command with the desikan atlas as a parcellation reference ( desikan et al . , 2006 ) .
average thickness was then obtained from the lateral temporal lobe ( superior , middle , and inferior temporal gyri ; parcels 1030/1015/1009 for left and 2030/2015/2009 for right ) and the mesial temporal lobe ( parahippocampal gyrus and parahippocampal cortex ; parcels
the parcels were combined using a weighted mean depending on area size ; parcels with larger surface area contributed more to the average . the hippocampal volume constituted a third measure of interest .
asymmetries were computed with the formula ( right    left )  /  ( right  +  left ) .
thick - lat - frees  ,  thick - mes - frees  , and
thickness maps were obtained using an established pipeline that included bias correction , multivariate segmentation , and thickness estimation with the direct algorithm ( all incorporated in the script antscorticalthickness.sh provided with the software ; tustison et al . , 2014 ) .
note that ants uses a voxel based method of thickness estimation , while freesurfer uses a surface based method ( avants et al .
after obtaining the thickness map with default parameters , which weighted segmentations priors at   =  0.25 , regional parcellation was performed using a multi - atlas parcellation algorithm .
this procedure calculates the parcellation based on the similarity of multiple atlases ( nki - trt-20 dataset , http://mindboggle.info ) with the patient 's brain (  antsmalflabeling.sh ; wang et al . , 2012 ) .
thickness values for lateral and medial temporal lobe were obtained in ants from the same regions used in freesurfer ; only voxels that were classified as gray matter during segmentation were included in the computation .
while hippocampal volume was not available in ants , we obtained an additional measure of entire temporal lobe asymmetry by combining mesial and lateral temporal regions .
this measure matched a similar pet measure obtained from the entire temporal lobe ( see section 2.3.3 pet  metabolism asymmetries ) .
thick - lat - ants  ,  thick - mes - ants  , and  thick - entire - ants  . prior to processing ,
diffusion data were visually inspected to identify and potentially remove volumes with excessive motion ; no such volumes were found .
subject motion , eddy currents , and epi distortions were corrected in a single step using cubic spline interpolation in exploredti ( leemans et al . , 2009 ) .
tensor estimation was performed with the non - linear restore algorithm , a procedure that is robust to outlier data points ( chang et al . , 2005 ; chang et al . , 2012 ) .
maps of fractional anisotropy ( fa ) were exported for later computation of tract asymmetries .
data preprocessed in exploredti were imported in fsl for tractography reconstruction ( diffusion toolbox , www.fmrib.ox.ac.uk/fsl ; behrens et al .
a monte carlo sampling procedure initially estimated the diffusion properties in each voxel , accounting for two tensors per voxel (  bedpostx command ) .
afterward , a probabilistic algorithm was used (  probtrax2 command ) to initiate 5000 fiber samples per seed voxel with the default angular threshold of 78. the first author ( dp ) manually drew seed and waypoint masks using anatomical landmarks visible in the colorized fa map . after obtaining the tractography map , voxels with less than 10% of the maximum visitation count
were set to zero to remove spurious streamlines , and a visual inspection in 3d was performed in search of potential failures ( see dti tractography examples in supplementary information ; see also pustina et al .
, eight tracts were reconstructed for each subject , four in each hemisphere : ( i ) the fornix , ( ii ) the parahippocampal fasciculus , ( iii ) the uncinate fasciculus , and ( iv ) the temporo - occipital connections consisting mostly of the inferior longitudinal fasciculus . for each tract ,
the fa value was computed as a weighted average of the probability mask , i.e. voxels with high visitation counts influenced the average more than voxels with low visitation counts .
asymmetries were calculated with the formula ( right    left )  /  ( right  +  left ) .
the final dti asymmetries were labeled  dti - fa - fx  ,  dti - fa - phip 
,  dti - fa - unc  , and  dti - fa - ilf  . the pet volume was coregistered to the respective mp - rage volume of each subject using spm8 ( university college london , http://www.fil.ion.ucl.ac.uk/spm/software/spm8/ ) .
four regional masks were utilized to extract pet values for the lateral , mesial , and entire temporal lobe , as well as the hippocampus .
the masks were obtained from the freesurfer parcellation map and contained , respectively , ( i ) the gray matter of the lateral temporal lobe ( parcels 1030/1015/1009 for left and 2030/2015/2009 for right ) , ( ii ) the gray matter of mesial temporal lobe including amygdala and hippocampus ( parcels 17/18/1006/1016 for left and 53/54/2006/2016 for right ) , ( iii ) only the hippocampus ( parcel 17 for left and 53 for right hemisphere ) , and ( iv ) the mesial and lateral masks combined together .
note , the four regional masks were partially overlapping in order to determine which combination of lateral and mesial areas yielded the best discriminatory power . during the preliminary phase of the study we noticed that the cortical mantel zigzagged around the pet blobs , and
we resolved this issue by processing the masks to reflect the smoothness of pet data ( see example in supplementary information : pet masking ) . to understand this procedure ,
consider that the metabolic signal is emanated from glucose uptake occurring mostly in gray matter , which is represented by the binary masks obtained from freesurfer .
this signal , however , is distorted due to limitations of pet imaging ( i.e. , the finite size of detector crystals , detector physics , travel distance before annihilation , poisson count statistics , and reconstruction methods ) .
the final pet image can be theoretically approximated by assuming that the true metabolic signal is smoothed with a gaussian smoothing kernel ( the point spread function of the scanner ; greve et al . , 2014 ) .
once smoothed , values will remain closer to 1 in areas of densely packed gm , and will fade closer to 0 in areas of thin gm .
the metabolic signal undergoes a similar distortion such that pet values near densely packed gm are closer to the true metabolic signal , while pet values of thinner gm are farer from the true metabolic signal .
thus , we applied 8  mm fwhm smoothing to the binary masks , and used the resulting continuous masks as weights to compute weighted average of pet for each region .
note , the pet volume itself was not smoothed , only the mask was smoothed .
voxels below 0.35 in the continuous mask were set to 0 to minimize the influence of non - gm signal .
the final asymmetry values were computed with the formula : ( right    left )  /  ( right  +  left ) .
 pet - entire  ,  pet - lateral  , and  pet - mesial  .
in epilepsy , the seizure focus may manifest as a small region with low metabolism , abnormal gm thickness or volume , or abnormal wm structure .
this pattern may not be detected when averaging the signal from large regional masks . instead
, local anomalies may manifest as increased variance in the data , or skewed and abnormally distributed data .
we investigated the value of adding variables that describe the shape of the data distribution by creating an extended dataset , where each asymmetry involving a mean was complemented with additional asymmetries of variance , skewness , and kurtosis ( fig .
a weighted variance algorithm was applied ( variance in regions of high probability counted more than variance in regions of low probability ) .
the asymmetries of skewness and kurtosis were computed with a modified formula to account for the presence of negative values : ( right    left )  /  abs(right )  +  abs(left ) .
analyses were performed using spss ( ibm spss statistics for windows v.20 , armonk , ny ) , and r ( r foundation for statistical computing , http://www.r-project.org ) .
demographic profiles ( i.e. , age , iq , scan intervals , etc . ) were compared between left and right tle patients with t - test or chi - square , as appropriate , and a correction for variance inhomogeneity was applied when necessary .
shapiro - wilk tests on neuroimaging variables revealed non - normal distribution on some asymmetries .
consequently , asymmetry differences between groups were compared with the non - parametric wilcoxon test . the maxt procedure with 100 million permutations
was used to correct for multiple comparisons (  multtest package in r ; pollard et al . ,
the maxt procedure extracts the dependency structure of statistical tests by running permutations on raw data rather than considering tests as independent and correcting only p - values . in general , maxt is considered more conservative than the false discovery rate and less conservative than bonferroni ( ge et al . , 2003 ; westfall and young , 1993 ) .
logistic regression was used to discriminate the left and right tle groups and predict group membership .
in contrast with linear discriminant analysis , logistic regression does not assume normal data distribution or equal variance between groups .
a stepwise procedure was used to select relevant variables that can distinguish ltle from rtle (  forward conditional 
after building the regression models , their reliability was tested with bootstrapped and split - sample procedures . for bootstraps run on the full sample ,
the group size was kept identical to the original dataset ( 28 and 30 , respectively ) , while 10,000 random samples were drawn with replacement . for bootstraps run in split samples ,
the size of the training group varied progressively from 97% to 52% , reducing the size in steps of 3% ( approximately removing one patient per group in 16 steps ) .
the process was repeated 10,000 times at each step to provide a reliable estimate of the success rates of the model with unseen data .
all bootstraps were performed with a penalized regression algorithm , which avoids infinite beta values when complete separation occurs ( firth , 1993 ) .
the  brglm package in r was used for this purpose ( kosmidis , 2013 ) .
the output of logistic regression models is in the range 01 , with 0 representing ltle and 1 representing rtle .
two variants of predictive models were attempted : using each modality separately or using all three modalities .
when combined modalities were utilized , the candidate predictors were either all the 14 available asymmetries of the main dataset , or those that were significantly different between the two groups after multiple comparison correction ( to be referred to as the unthresholded and thresholded models , respectively ) .
differences in average inter - modality correlations between groups were investigated with independent t - tests . for this purpose ,
all correlation coefficients between a modality pair ( i.e. , pet - dti ) obtained from one group were compared with those of the other group ( see pustina et al . , 2015 for a previous application ) . in total
, three t - tests were run , one for each modality pair , with results thresholded at   =  0.05/3  =  0.0167 to correct for multiple comparisons .
a qualitative inspection of the links between asymmetries was also performed using dendrogram trees from bootstrapped hierarchical clustering ( suzuki and shimodaira , 2006 ) .
all 58 patients included in this study were pet - positive ( a pathology was detected in pet by expert visual inspection ) , while 35 of them were also mri - positive ( a pathology was detected in mri by expert visual inspection ) . the lack of pet - negative patients derived from the need for data in all three modalities and the other exclusion criteria . the most common pathology found in mri was mts ( 22 patients , 40% of all patients , 63% of mri - positive patients )
the typical pathology found in pet was hypometabolism in one of the temporal lobes , none of the patients had hypermetabolism in the temporal lobe .
the female / male ratio was 15/13 in the ltle group and 13/17 in the rtle group .
the two tle groups did not differ in age ( t  =  .767 , p  =  .446 ) , gender ( chi - square  =  .608 , p  =  .436 ) , handedness ( t  =  1.98 , p  =  .070 ) , age of epilepsy onset ( t  =  .903 , p  =  .717 ) , epilepsy duration ( t  =  .903 , p  =  .717 ) , or intracranial volume ( t  =  .554 , p  =  .582 ) . fig . 1 displays the boxplots of each asymmetry variable in ltle and rtle .
1a ) , none of the dti or mri asymmetries survived the correction , while all four pet asymmetries were significantly different between ltle and rtle . the extended dataset
( including variance , skewness , and kurtosis ) showed ten asymmetries to be different between groups after maxt correction , all of which were derived from pet : pet - mesial ( w  =  6.32 , p - adj  <  0.001 ) , pet - entire ( w  =  6.30 , p - adj < 0.001 ) , pet - lateral ( w  =  6.18 , p - adj  <  0.001 ) , pet - hippo ( w  =  5.98 , p - adj  <  0.001 ) , pet - entire - variance ( w  =  4.01 , p - adj  =  0.001 ) , pet - hippo - variance ( w  =  3.97 , p - adj  =  0.002 ) , pet - entire - skewness ( w  =  3.90 , p - adj  =  0.002 ) , pet - hippo - skewness ( w  =  3.69 , p - adj  =  0.006 ) , pet - lateral - skewness ( w  =  3.44 , p - adj  =  0.016 ) , and pet - mesial - variance ( w  =  3.28 , p - adj  =  0.028 ) . the stepwise regression of pet asymmetries selected pet - mesial and pet - lateral as significant predictors of group membership ( accuracy  =  96.6% , chi - square  =  70.95 , p  <  .001 , nagelkerke r2  =  .941 , hosmer & lemenshow p - value  =  1 ) . the stepwise regression of mri asymmetries selected only thick - entire - ants as a significant predictor of group membership ( accuracy  =  67.2% , chi - square  =  6.67 , p  =  .01 , nagelkerke r2  =  .145 , hosmer & lemenshow p - value  =  .698 ) .
in an attempt to match the variables obtained from freesurfer and ants , we also obtained the same thick - entire asymmetry from freesurfer , but a t - test showed no difference between tle groups ( p  >  .1 ) and the freesurfer variable did not enter the predictive equation .
the stepwise regression of dti symmetries selected only dti - fa - ilf as significant predictor of tle group membership ( accuracy  =  70.7% , chi - square  =  6.84 , p  =  .009 , nagelkerke r2  =  .148 , hosmer & lemenshow p - value  =  .642 ) .
a stepwise regression with all 14 multimodal asymmetries as candidate predictors selected two pet ( pet - mesial and pet - lateral ) and one mri ( thick - lateral - ants ) asymmetries as significant predictors ( accuracy  =  100% , chi - square  =  80.34 , p  <  .001 , nagelkerke r2  =  1 , hosmer & lemenshow p - value  =  1 ) .
therefore , there was no chance for other modalities to enter the model created by thresholded data , and the same model as with pet alone emerged ( accuracy  =  96.6% , chi - square  =  70.95 , p  <  .001 , nagelkerke r2  =  .941 , hosmer & lemenshow p - value  =  1 ) .
besides the existing asymmetries of mean , the extended dataset included the asymmetries of variance , skewness and kurtosis , across all three modalities .
a stepwise regression with all 47 asymmetries selected four significant predictors : pet - lateral , pet - mesial , dti - fa - phip , pet - entire - variance ( accuracy  =  100% , chi - square  =  80.35 , p  <  .001 , nagelkerke r2  =  1 , hosmer & lemenshow p - value  =  1 ) . as mentioned , ten asymmetries survived multiple comparison adjustment from the extended dataset , all of which were pet related . a stepwise regression with these asymmetries selected three variables ( pet - mesial , pet - hippo - variance , and pet - entire - variance ) as the best predictors of the tle group ( accuracy  =  100% , chi - square  =  80.34 , p  <  .001 , nagelkerke r2 
=  1 , hosmer & lemenshow p - value  =  1 ) .
the above analyses showed that three models were the most successful in predicting group membership , all reaching 100% accuracy .
the first was obtained from unthresholded data in the main dataset and included two pet and one mri variable ( model 1 ) , the second was obtained from the thresholded extended dataset and included three pet variables ( model 2 ) , the third was obtained from the unthresholded extended dataset and included three pet and one dti variable ( model 3 ) . in selecting the most appropriate model we considered three factors : ( i ) the amount of erroneous classifications from a full - sample bootstrapped procedure , ( ii ) the accuracy rates obtained from the split - sample validation procedure , and ( iii ) whether a bias existed in predicting one group better than the other .
success rates from the full - sample bootstrapping procedure were slightly lower for the first model ( 95% ci [ 96% 100% ] ) compared to the other two ( [ 98% 100% ] and [ 100% 100% ] , respectively ) .
however , the bootstrap on the full sample showed a largely unbalanced prediction with the third model , in which ltle patients carried more certain posterior probability values than rtle ( lower right panel in fig .
consequently , the second model was deemed as the winning model as it satisfied all three criteria .
given that the winning predictive model was derived from three pet asymmetries , we tested whether data coming from different scanners can be used to predict group membership .
for this , we ran a bootstrapped cross - validation with the patients scanned with the main siemens camera as training group ( n  =  47 , resampling with replacement ) and the remaining patients as test group ( n  =  11 ) .
the average success rate was 99.4% with 95% ci  =  [ 91% 100% ] , indicating a slight drop of the lowest confidence interval , but yet good average accuracy with cross - scanner data .
one of the limitations of our patient sample is that it contains data from both operated and non - operated patients . despite the efforts of the clinical team ,
the identification of the seizure focus in non - operated patients remains inferential ; only patients who underwent surgery with good clinical outcome can be considered to have confirmed seizure lateralization . to investigate the quality of the current classifications attributed by the clinical team , we trained the predictive algorithm on operated patients with good seizure outcome and applied the prediction to the 23 patients who did not yet receive surgery .
of the 35 patients who underwent surgery , 29 ( 83% ) had good seizure control and were used for training ( 24 engel class i and 5 engel class ii ) .
the prediction of the 23 non - operated patients showed 100% correspondence with the current classification assigned by the clinical team .
this analysis provided further evidence that , despite the inferential nature of the soz in non - operated patients , their clinical classification is in line with what can predicted based on available patients with good surgery outcome . our sample consisted in patients who were all pet - positive .
consequently , the main reason why the winning model included only pet asymmetries could be that pathologies in the other modalities were under - represented . to put both mri and pet modalities at a comparative level
, we built a predictive model only within the 35 mri - positive patients ( 19 ltle , 16 rtle ) .
unthresholded asymmetries from the main dataset were introduced as potential predictors in stepwise logistic regression .
two variables entered the equation , pet - hippo and thick - mes - frees , reaching an average bootstrapped accuracy of 99.7 with 95% ci  =  [ 97% 100% ] .
this finding confirmed that mri can be a valid predictor of seizure laterality when mri pathology is present .
however , we also applied on these patients the winning algorithm obtained from the three pet asymmetries .
average bootstrapped accuracy of this model was 99.9% with 95% ci  =  [ 100% 100% ] . in sum , success rates obtained with pet asymmetries alone were comparable , if not better , than the success rates obtained with the two pet / mri predictors above .
the applicability of the predictive algorithm was tested on a series of patients recently admitted and reviewed by the comprehensive epilepsy board at thomas jefferson hospital .
data from new patients followed the same processing pipeline used for the other patients in the study , though the predictive algorithm was enhanced to provide a probabilistic classification of each new patient .
in short , for each new patient , 10,000 predictions were obtained with random training ratios between 52% and 97% .
each prediction was complemented with a goodness of fit ( or generalizability score ) of the model that produced it .
this score consisted in the classification accuracy of the patients that were left out during training ( i.e. , accuracy of classifying 48% when the other 52% was used for training ) .
finally , the 10,000 predictions of the new patient were weighted by the goodness of fit of each prediction ; i.e. , a model that predicted 100% of the non - trained patients was counted 100 times , a model that predicted 50% of the test patients was counted 50 times .
this procedure outputs the ratios of times the patient was classified as ltle or rtle out of the 10,000 predictions , which consists in a probabilistic classification of the patient .
in addition , the confidence intervals of the regression output provide a measure of fluctuations ( variability ) in the probability obtained across the bootstrap .
the toolkit and data used in this study are available at http://dorianps.github.io / tleprediction/. there were seven new patients reviewed in the past months by the epilepsy board ( table 2 ) .
their pet reports indicated three cases with left temporal hypometabolism , one bilateral hypometabolism slightly biased to the left , two right temporal hypometabolism , and one mild right temporal hypometabolism . after reviewing all the clinical information , the epilepsy board classified the first four as ltle and the remaining three as rtle .
the prediction algorithm correctly classified all of them over 95% of the 10,000 bootstraps ( range 96.33%100% ) .
six of the seven patients had confidence intervals of the posterior probabilities that did not include the uncertainty threshold of 0.5 , while one patient slightly crossed the threshold ( 95% ci  =  [ 0.44 0.99 ] ) .
this patient crossed the 0.5 threshold between 2.5% and 5% of the 10,000 bootstraps , and was rated with mild hypometabolism in visual inspection reports .
the above new patients had lateralized hypometabolism and convergent data from other modalities , which helped the clinical team in establishing the soz . in other cases ,
the neuroimaging profile may not be clear ; i.e. , data may be discordant or pet visual inspections may not be informative .
these are cases in which pet is rated as normal or bilaterally hypometabolic , or cases in with hypometabolism in one hemisphere and mri pathology in the other hemisphere . to test the predictive model with such cases we searched our database for patients who had surgery with good surgery outcome , thus , providing a well - grounded confirmation of soz laterality .
we found five cases that were initially excluded from the study because of missing dti , and obtained the probabilistic classification from each .
patient 1-rd had discordant mri  pet findings , with mri revealing t2 hyperintensities in right frontal , right periventricular region , left caudate , and a mild right hippocampal atrophy .
two pet examinations were obtained from this patient , the first rated as unremarkable , the second rated with mild left temporal hypometabolism .
the seizure focus was finally established through intracranial electrodes implantation , which showed the patient was ltle .
the patient underwent left anterior temporal resection and has been seizure free for 30  months .
we predicted this patient as ltle 99.5% of the time , with 95% ci of posterior probability [ 0.075 0.389 ] .
patient 2-vo was found with left mts in mri and was rated with bilateral temporal hypometabolism in pet .
she was classified as ltle by the clinical team and underwent laser ablation of the left hippocampus and amygdala , after which has been seizure free for 28  months .
we predicted this patient as ltle 99.9% of the time , with 95% ci of posterior probability [ 0.001 0.157 ] .
patient 3-vd had left mts in mri and was rated with bilateral temporal hypometabolism in pet .
he was classified as ltle by the clinical team and underwent left anterior temporal lobe resection , after which has been seizure free for 34  months .
we predicted this patient as ltle 100% of the times , with 95% ci of posterior probability [ 0.000 0.004 ] .
she was implanted with intracranial electrodes , where seizures were found to originate in the left temporal lobe .
the patient underwent left anterior temporal lobe resection and has been seizure free for 7  months .
we predicted the patient as ltle 100% of the times , with 95% ci of posterior probability [ 0 0.01 ] .
she was implanted with intracranial electrodes , where seizures were found to originate from the left temporal lobe .
she underwent left anterior temporal resection and has been seizure free for 53  months .
we predicted this patient incorrectly as rtle 90.5% of the time , but this was the only patient to show uncertain posterior probability ( 95% ci  =  [ 0.289 0.981 ] ) . in sum , four of the five patients who had discordant or negative pet findings ,
were categorized with high certainty both in classification ratios and in the posterior probability intervals .
a comparison of average correlations between each modality pair revealed closer pet  dti relationship in ltle compared to rtle ( t  =  3.06 , p  =  .005 , 95% ci  =  [ 0.06 0.32 ] ) , and closer mri 
pet relationship in rtle compared to ltle ( t  =  3.65 , p  <  .001 , 95% ci  =  [ 0.42    0.12 ] ) . the mri 
dti correlations did not differ between groups ( t  =  1.87 , p  =  .07 ) .
the qualitative inspection of hierarchical clustering dendrograms showed that pet was first linked with dti , with mri asymmetries located on a different branch in ltle patients .
vice versa , pet was first linked to mri and later merged with dti asymmetries in rtle patients ( supplementary fig .
this study investigated the predictive power of asymmetry indices from three different modalities , pet , mri , and dti , with the aim of building a clinically applicable predictive tool capable of distinguishing the laterality of the seizure focus in temporal lobe epilepsy . to achieve the necessary confidence required in clinical settings , we performed split - sample validations using robust statistical methods ( i.e. , bootstrapping ) , and tested the predictive model on several new patients . among all the asymmetries considered , pet asymmetries stood out as the most informative with respect to the lateralization of the soz .
these asymmetries also survived multiple comparison correction , while none of the asymmetries from mri and dti survived the correction .
for example , reports have shown that good surgical outcome is best predicted by the presence of hypometabolism in pet images , while the presence or absence of mri pathology adds no further value to the predictive value of pet ( carne et al . , 2004 ; choi et al . , 2003 ;
feng et al . , 2014 ; gok et al . , 2013 ; kuba et al . , 2011 ; lopinto - khoury et al . , 2012 ;
) . when comparing the predictive value of pet , mri , and dti , thivard et al .
( 2011 ) found that pet carries the most useful information to identify the soz , while dti added predictive value only for cases with pet - negative findings . in our study , pet had top predictive power not only in the total sample of mixed mri - positive / mri - negative patients , but also in the restricted group of mri - positive patients .
thus , our data , and the above mentioned studies , highlight the importance of using metabolism as a predictor of soz laterality in tle patients .
note , hypometabolism is not directly related to structural pathologies found in mri ( lamusuo et al .
, 2001 ; o'brien et al . , 1997 ; theodore et al . , 2001
) , though recent findings suggest that even mri - negative patients carry some structural pathology at the microscopic level ( deep et al . , 2012 ) .
these findings suggest that subtle structural gm pathologies may frequently be present in tle , and , while not visible on mri , might still be detected at the metabolic level . differently from previous studies that reported success rates in the range
85%92% when classifying tle patients with pet data ( didelot et al . , 2010 ; kerr et al . , 2013 ; li et al . , 2000
; muzik et al . , 1998 ; soma et al . , 2012 ) , we were able to achieve 100% success rates on the full dataset , and above 95% in split - sample validation .
we took extra care in verifying this result with thousands of bootstraps and 16 split - sample steps , up to a split - half validation . while such rigorous validation has not been reported in previous studies , this is an important step for obtaining reliable and replicable classifications .
indeed , several authors have warned against the widely used leave - one - out validation , as a suboptimal method with unstable prediction error estimates ( gabrieli et al . , 2015 ; see also http://www.russpoldrack.org/2012/12/the-perils-of-leave-one-out.html ; kohavi , 1995 ; rao and fung , 2008 ) .
interestingly , we observed such instability as an increase in confidence interval range when the split sample approached the leave - one - out procedure ( i.e. , training sample  =  97% in fig .
2 ) . this result supports the previous warnings that a reliable estimation of prediction error requires split samples larger than one or two subjects . among the reasons that allowed us to create a highly accurate predictive model
first , we used a new masking procedure based on smoothed cortical mantel masks , similar to probabilistic rois .
van bogaert et al . , 2000 ) or region - wise asymmetries ( muzik et al . , 2005 ) , but not asymmetries from probabilistic rois .
the results obtained in our study suggest that probability masks of smoothed cortical mantel can be a simplistic , yet powerful , solution to automatize pet analyses .
second , we included not just asymmetries of mean , but also asymmetries of variance , skewness , and kurtosis .
importantly , results from cancer research suggest that these measures improve significantly the predictive power ( nair et al . , 2012 ) .
third , all pet asymmetries were obtained without smoothing or normalizing the data , thus , keeping the pet image as close as possible to the original scanner output .
this choice seems to be advantageous for conserving subtle variance changes that can dissociate ltle from rtle .
pet was followed by dti as the second best predictive modality at the individual modality level ( 71% ) , while mri was the third best ( 67% ) . even when multimodal asymmetries were combined , the combination of asymmetries of pet with dti yielded higher accuracy rates than the combination of pet with mri ( lower ci 100% and 96% , respectively ) .
however , in our study mri was an important predictor within mri - positive patients , where mesial temporal lobe thickness was combined with hippocampal metabolism to produce a highly successful model .
( 2012 ) , who studied only patients with hippocampal sclerosis , and found that t1-weighted mri is better able to distinguish ltle from rtle compared to dti .
together , our findings and the above literature indicate that mri measures can be more sensitive than dti in the restricted group of mri - positive patients , while dti can be slightly superior when all tle patients are considered .
success rates with mri data have been in the range 86%100% ( duchesne et al . , 2006 ; focke et al . , 2012 ; li et al . ,
our mri - based prediction model was less successful ( 67% ) than the above studies , a finding that might be related to our choice of using thickness asymmetries .
previous literature has reported more widespread atrophy in ltle than in rtle ( bonilha et al .
, 2007 ; keller and roberts , 2008 ; kemmotsu et al . , 2011 ) , suggesting that thickness might be a good variable for distinguishing the groups .
the ltle  rtle difference , however , has not been consistently reported in all studies ( bernhardt et al . , 2011 ;
. when inspecting the maps of thickness anomalies published by some of these studies ( kemmotsu et al . , 2011 ; lin et al . , 2007 )
it appears that gm thinning may occur bilaterally in both ltle and rtle patients , reducing the power of thickness to distinguish ltle vs. rtle .
another factor that may limit the use of cortical thickness is the unreliable estimation of thickness in areas of mri pathology ( i.e. , dysplasias ) .
however , we expected these anomalies to be captured in asymmetries of variance , which would have been positive from a detection perspective .
nevertheless , attempts have been made to use dti data for distinguishing ltle from rtle .
our success rate of 71% when using only dti asymmetries is lower than these reports , a finding that may again be related to dti processing steps ( i.e. , use of tbss instead of regular tractography , tracts under investigation , etc . ) , or the selection of variables ( i.e. , use of raw fa instead of fa asymmetry , use of mean diffusivity , etc . ) .
a new finding not previously reported is that ltle patients carry a closer relationship pet 
previous studies have shown that the relationship of metabolism with structural integrity can break down in proximity of the epileptogenic area ( theodore et al . , 2001 ) .
this indicates that the coupling of structure with metabolism is not static , and might be interrupted by the presence of pathology . on the other hand ,
numerous studies have shown that ltle carry a more severe form of wm pathology than rtle ( ahmadi et al . ,
2009 ; lu et al . , 2013 ; pustina et al . , 2014a ; pustina et al . , 2015 ) , and that wm is more closely associated with the clinical course ( kemmotsu et al . , 2011 ) .
therefore , one interpretation of our findings might be that the extensive wm pathology in ltle biases the relationship of metabolism more toward the status of wm than to that of gm . note ,
metabolic activity is not only related to gm structural integrity ( volume , thickness , etc . ) , but also to functional integrity , which is based on the inputs sent and received from other brain regions . while this hypothesis is speculative , an appropriate way to investigate it in future studies is the comparison of multimodal relationships in tle patients to the same relationships existing in the healthy population . on the technical side ,
we found a higher concordance of thickness estimated in the lateral temporal lobe ( r  =  0.673    0.08 ) than the mesial temporal lobe ( r  =  0.3    0.03 ) .
the discrepancy between the two software asymmetries became evident also in the relationship of thickness with other modalities , i.e. , the difference in pet 
1 ) . on the other hand , freesurfer asymmetries entered the prediction model of the mri - positive patients , but could not make it on the prediction models of the full sample , where ants asymmetries were more successful .
these findings suggest that the choice of software may have a strong impact on the results , raising concerns about the stability of the findings when different methods are used in different studies . in considering the conclusions drawn from this study , several limitations should be noted .
our findings should not be erroneously interpreted as evidence that the other modalities besides pet are not needed . in this regard ,
the conclusions we draw about the accuracy rates of each modality pertain only to the specific measures used in this study ( i.e. , fa from dti , thickness from mri ) .
there are several other measures that can be obtained from mri and dti , which we did not investigate ( i.e. , mean diffusivity , jacobian determinants , segmentation probability , etc . ) , not to mention recent new acquisition protocols and new analysis techniques ( i.e. , hardi , spherical deconvolution of diffusion data , etc . ) .
moreover , we built the model on pet - positive patients , with limited and inconclusive results from pet - negative examples . a proper development of models to apply to the full range of clinical findings requires a more heterogeneous sample that includes more pet - negative and ambiguous cases .
in this study we used a relatively small sample of 58 patients , and , despite using rigorous validation methods and finding robust effects , the lack of bias can not be entirely excluded with such small sample .
the goal of this paper was to develop a quantitative formula that would match the clinical team classification .
however , a better standard for accurate prediction would be to train the model on seizure outcome .
we had a limited number of patients with surgical outcome , and few of them had bad seizure outcome , providing insufficient information for training purposes .
lastly , while we used three variables from a single modality to build our prediction model , recent studies have shown high lateralization accuracy with models based on clinical / cognitive variables ( armananzas et al . , 2013 ) ,
fmri ( chiang et al . , 2015 ) , or hippocampus architectural properties ( nazem - zadeh et al . , 2014 ;
future work can build upon this knowledge to combine the predictive value of all predictive variables into a single model for routine clinical use .
to conclude , we investigated the separate and combined power of asymmetries from three different modalities , and found that metabolism is the single most powerful predictor of soz laterality .
the crucial elements of pet 's predictive success were the introduction of a new masking method and the inclusion of variance asymmetries .
our findings suggest that statistical processing of neuroimaging data can provide accurate classification of patients that match or exceed the detection ability of human experts , a conclusion that has been advanced by several authors ( ferrie et al .
the implementation of automated procedures of classification not only allows for an objective estimation of anomalies present in the data , but also offers a probabilistic range of confidence , which is not obtained from qualitative descriptions . ultimately , automatic predictions can be combined with visual inspections to improve detection of subtle patterns of anomaly . in the spirit of open source science ,
we have made available the tools and data necessary to lateralize seizure onset in new tle patients : http://dorianps.github.io / tleprediction/. the following are the supplementary data related to this article.supplementary material 1.supplementary material 2.supplementary fig . 
1asymmetry correlations ( spearman s rank method ) in ltle ( panel a ) and rtle ( panel b ) .
correlations that did not pass an uncorrected  = 0.05 threshold are crossed out , while those that are significant show the spearman s rho value .
panel c shows the difference in correlation between ltle and rtle , along with the confidence interval of this difference .
the ` cocor ` r package ( http://comparingcorrelations.org ) was used for ci computations.supplementary fig .  2 supplementary material 1 .
asymmetry correlations ( spearman s rank method ) in ltle ( panel a ) and rtle ( panel b ) .
correlations that did not pass an uncorrected  = 0.05 threshold are crossed out , while those that are significant show the spearman s rho value .
panel c shows the difference in correlation between ltle and rtle , along with the confidence interval of this difference .
 pre - surgical evaluation of patients with temporal lobe epilepsy ( tle ) relies on information obtained from multiple neuroimaging modalities . 
 the relationship between modalities and their combined power in predicting the seizure focus is currently unknown . 
 we investigated asymmetries from three different modalities , pet ( glucose metabolism ) , mri ( cortical thickness ) , and diffusion tensor imaging ( dti ; white matter anisotropy ) in 28 left and 30 right tle patients ( ltle and rtle ) . 
 stepwise logistic regression models were built from each modality separately and from all three combined , while bootstrapped methods and split - sample validation verified the robustness of predictions . among all multimodal asymmetries , three pet asymmetries formed the best predictive model ( 100% success in full sample , > 95% success in split - sample validation ) . 
 the combinations of pet with other modalities did not perform better than pet alone . 
 probabilistic classifications were obtained for new clinical cases , which showed correct lateralization for 7/7 new tle patients ( 100% ) and for 4/5 operated patients with discordant or non - informative pet reports ( 80% ) . 
 metabolism showed closer relationship with white matter in ltle and closer relationship with gray matter in rtle . 
 our data suggest that metabolism is a powerful modality that can predict seizure laterality with high accuracy , and offers high value for automated predictive models . 
 the side of epileptogenic focus can affect the relationship of metabolism with brain structure . 
 the data and tools necessary to obtain classifications for new tle patients are made publicly available . 
Introduction
Material and methods
Results
Discussion