the world health organization ( who ) has defined osteoporosis as a skeletal disorder characterized by diminished bone strength resulting in increased fracture risk .
bone strength is determined by interacting somatic and genetic factors . reported somatic factors include aging [ 25 ] , menopause [ 5 , 6 ] , and body mass index ( bmi ) [ 4 , 5 , 7 , 8 ] . to identify the genetic determinants of osteoporosis , an earlier study by the first author of this paper   investigated how the incidence of low bone mineral density ( bmd ) in taiwanese women is affected by interactions among eleven single nucleotide polymorphisms ( snps ) in nine genes known to be involved in osteoporosis [ 1016 ] , including tumor necrosis factor - alpha ( tnf )
, transforming growth factor - beta 1 ( tgfb1 ; tgf1 ) , osteocalcin , parathyroid hormone ( pth ) , interleukin 1 receptor antagonist ( il1_ra ) , heat shock 70  kda protein 1-like ( hspa1l ; hsp70 hom ) , heat shock 70  kda protein 1b ( hspa1b ; hsp70 - 2 ) , calcitonin receptor ( ctr ) , and bone morphogenetic protein-4 ( bmp-4 ) .
generally , several hormones , cytokines , and cell signaling - related proteins were chosen .
for example , ctr , which is a receptor for the linear polypeptide hormone calcitonin , reduces blood calcium and suppresses the effects of pth .
the cytokine family includes tnf , tgf- , bmp4 ( protein of tgf-superfamily ) , and il-1ra ( protein of interleukin 1cytokinefamily ) whereas cell - signaling proteins include hsp70 hom and hsp70 - 2 .
studies of the interactions among these hormones ( e.g. ,   and references therein ) indicate that osteoporosis is an endocrinological problem .
accumulating evidence reveals that snps are potential genetic markers for predicting osteoporosis outcome in taiwanese women .
chang et al .   also proposed a novel odds ratio - based genetic algorithm ( or - ga ) method of using odds ratios for quantitatively measuring the disease risk associated with various snp combinations to determine the susceptibility to osteoporosis in taiwanese women .
taiwanese women who are carriers of risk alleles in two or more of these snps are likely to be at increased risk of osteoporosis because several partial deficiencies in these pathways may severely diminish bone density
. therefore , snps may indicate risk of osteoporosis in taiwanese women and may be useful in clinical association studies to determine the genetic basis of disease susceptibility .
the risk of osteoporosis is likely to be higher than normal in carriers of risk alleles in two or more of these snps because several partial deficiencies in these pathways may substantially decrease bone density .
therefore , interacting polymorphisms may affect osteoporosis risk . in , the effects of age , bmi , and
eleven interacting polymorphisms in nine genes were studied in terms of their effects on the incidence of low bmd ( table 1 ) .
the findings showed that specific snp combinations may be risk factors for postmenopausal osteoporosis in taiwanese women .
in addition to these specific snp combinations , bmi and age also showed independent associations with bmd in postmenopausal taiwanese women .
although an apparent association between snps and osteoporosis has been identified in taiwanese women , a continuing challenge in genomics studies of taiwanese women populations lies in identifying significant genes .
exhaustive computation over the model space is infeasible if the model space is very large , as there are 2 models with p snps [ 20 , 21 ] .
feature selection techniques are designed to find responsible genes and snps for certain diseases . by selecting a small number of snps with significantly larger effects compared to other snps and by disregarding snps of lesser significance
, researchers can focus on the most promising candidate genes and snps for use in diagnosis and therapy [ 21 , 22 ] . in , combined polymorphisms in different genomic regions
the findings showed that a combination of several gene polymorphisms contributes to the development of osteoporosis in taiwanese women .
however , that study did not report a subset of snps that can be used to predict osteoporosis outcome in this population .
therefore , the current study used the same dataset used in   to elucidate the relationship between osteoporosis and snps in taiwanese women in a performance comparison of three different classification algorithms with wrapper - based feature selection : multilayer feedforward neural network ( mfnn ) [ 2428 ] , naive bayes , and logistic regression .
the mfnns have proven particularly effective for nonlinear mapping based on human knowledge and are now attracting interest for use in solving complex classification problems .
an mfnn containing layers of simple computing nodes , which is analogous to brain neural networks , has proven effective for approximating nonlinear continuous functions and for revealing previously unknown relationships between given input and output variables [ 25 , 26 ] .
the unique structure of mfnns enables them to learn by using algorithms such as backpropagation and evolutionary algorithms [ 31 , 32 ] .
potential medical applications of mfnns include solving problems in which the relationship between independent variables and clinical outcome are poorly understood . because mfnns are capable of self - training with minimal human intervention , many studies of large epidemiology databases have , in addition to conventional statistical methods , used mfnns for further insight into the interrelationships among variables .
a naive bayes classifier assumes that the presence ( or absence ) of a particular feature of a class is unrelated to the presence ( or absence ) of any other feature , given the class variable .
depending on the precise nature of the probability model , naive bayes classifiers can be trained very efficiently in a supervised learning setting .
the classifier obtained by using this set of discriminant functions and by estimating the relevant probabilities from the training set is often called the naive bayesian classifier because , if the the attributes are  naively  assumed to be independent given the class , direct application of the bayes theorem easily confirms that this classifier is optimal in terms of minimizing the misclassification rate or zero - one loss [ 34 , 35 ] .
logistic regression is a statistical method of predicting the outcome of a variable that is categorical ( i.e. , it can have several different categories ) and is dependent on one or more predictor variables .
a logistic function can be used to model the probabilities describing the possible outcome of a single trial as a function of explanatory variables .
logistic regression is typically used to measure the relationship between a categorical dependent variable and one or more continuous independent variables by converting the dependent variable to probability scores . the wrapper - based feature selection method , in which the feature selection algorithm acts as a wrapper around the classification algorithm ,
was also used to identify an snp subset with sufficient predictive power to distinguish between high- and low - risk alleles . in the wrapper - based approach ,
the function used to evaluate feature subsets uses the classification algorithm itself to perform a best - first search for a good subset .
starting from an empty feature set , it searches forward for potential feature subsets by performing greedy hillclimbing augmented with a backtracking technique .
.   showed that it may be superior to hybrid approaches combining chi - square and information - gain methods reported in the literature .
a comprehensive literature review shows no attempts to predict osteoporosis outcome in taiwanese women using genetic factors ( snps ) and the three above mentioned classification algorithms with wrapper - based feature selection method .
this study therefore compared performance in three classification algorithms : mfnn , naive bayes , and logistic regression , with and without wrapper - based feature selection techniques .
identifying the genes and snps associated with taiwan population of women with osteoporosis would enable researchers to focus on the candidate genes and snps that are most promising for use in diagnosis and therapy .
the results of our studies could be generalized to snp searches in genetic studies of human disorders and to development of new molecular diagnostic / prognostic tools .
however , before routine application of genomic analysis in clinical practice , genetic markers must be validated in prospective clinical trials .
the dataset in this study , which included snps , age , menopause , and bmi , was the same dataset used in a previous study by the first author of this paper .
the t - score was calculated according to who classifications using a locally derived reference range provided by the manufacturer .
the subjects were divided into two bmd groups according to t - score [ 3840 ] .
subjects with t - score > 1 were enrolled in the high bmd group , and those with t - scores  1 were enrolled in the low bmd group .
the overall dataset was derived from 295 cases , including ( i ) 247 postmenopausal cases ( 83.73% ) and 48 prepremenopausal cases ( 16.27% ) ; ( ii ) 112 high bmd cases ( 37.97% ) and 183 low bmd cases ( 62.03% ) .  
post - menopause was defined as the absence of menstruation for > 6 months or age  50 years .
clinical data used for diagnosis were further converted into numerical form , that is , 1 for  high bmd  and 0 for  low bmd .  

table 1 shows the 22 snps analyzed in this study , which were the same as those analyzed previously by the first author of this paper .
table 1 shows that the nine candidate genes included tnf , transforming growth factor - beta 1 ( tgf1 ) , osteocalcin , parathyroid hormone ( pth ) , interleukin 1 receptor antagonist ( il1_ra ) , hsp , calcitonin receptor ( ctr ) , bone morphogenetic protein-4 ( bmp-4 ) , and three genotypes per locus .
the three families of classification algorithms used as the basis for comparisons in this study were mfnn , naive bayes , and logistic regression .
these classifiers were implemented using the waikato environment for knowledge analysis ( weka ) software .  
an mfnn is an artificial neural network ( ann ) model in which connections between the units do not form a directed cycle [ 2428 , 30 ] .
from an algorithmic perspective , the underlying process of an mfnn can be divided into retrieving and learning phases .
assume an l - layer feedforward neural network with nl units at the lth layer .
in the retrieving phase , the mfnn iterates through all layers to produce the retrieval response { ai(l ) , i = 1 , 2 ,  , nl } at the output layer based on test pattern inputs { ai(0 ) , i = 1 , 2 ,  , n0 } , the known weights wij of the network , and the nonlinear activation function fi ( e.g. , sigmoid function ) . in the learning phase of this mfnn ,
the backpropagation algorithm   and evolutionary algorithms [ 31 , 32 ] are used in the learning scheme .
the weight updating mechanism is a backpropagation of corrective signals from the output layer to the hidden layers .
the goal is iteratively selecting a set of weights wij(l ) for all layers such that the squared error function e can be minimized by a pair of input training patterns { ai(0 ) , i = 1 , 2 ,  , n0 } and target training patterns { tj , j = 1 , 2 ,  , nl}.   mathematically , the iterative gradient descent formulation for updating each specific weight wij(l ) can be expressed by the following equation : 
 ( 1)wij(l)wij(l)ewij(l ) , 
 where  is the learning rate and e/wij(l ) can be effectively calculated through a numerical chain rule by backpropagating the error signal from the output layer to the input layer . structurally , however , an mfnn is a spatial and iterative neural network with several layers of hidden neuron units between the input and output neuron layers .
the basic function of each neuron is the linear basis function , and activation is modeled with a non - decreasing and differentiable sigmoid function .
inputs contain the information about clinical factors , for example , snps , that are needed for the database .
the mfnn is trained first by repeatedly providing input - output training pairs and by executing the backpropagation learning algorithm . after this training process is complete , the mfnn is tested by sending testing data inputs ( i.e. , snps ) to the network .
the forward propagation of the mfnn reveals the osteoporosis outcome for a specific case so that causes can be inferred from effects . here ,
the default weka parameters were used , that is , hidden layer neurons = 6 , learning rate = 0.3 , momentum variable = 0.2 , and training time = 500 .
second , all features in naive bayes , which is the simplest bayesian network , are assumed to be conditionally independent .
let ( x1 , x2 ,  , xp ) be features ( i.e. , snps ) used to predict class c ( i.e. , disease status , 1 = high bmd or 0 = low bmd ) . given a data instance with genotype ( x1 , x2 , 
, xp ) , the best prediction of the disease class is given by class c , which maximizes the conditional probability pr(c = c | x1 = x1 , x2 = x2 ,  , xp = xp ) .
bayes theorem is used to estimate the conditional probability pr(c = c | x1 = x1 ,
x2 = x2 ,  , xp = xp ) , which is decomposed into a product of conditional probabilities .  
third , the logistic regression generates the coefficients for the following formula used for logit transformation of the probability of a patient having a characteristic of interest : logit(p ) = b0 + b1x1 + b2x2 + +bkxk .
the formula used to calculate the probability of the characteristic of interest in this study is p
= 1/(1 + e ) , where 1 = high bmd and 0 = low bmd .
the wrapper - based feature selection approach , in which a feature selection algorithm acts as a wrapper around a classification algorithm , was used to find a subset of snps that maximizes the performance of the prediction model .  
figure 1 shows that , in the wrapper approach , the feature subset is selected by using a black box classification algorithm ( i.e. , selection is performed using the interface alone and does not require knowledge of the algorithm ) . to search for a good subset ,
the search space is organized such that each state represents a feature subset . for n features ,
each state has n bits , and each bit indicates whether a feature is present ( 1 ) or absent ( 0 ) . to determine the connectivity between the states , this study used operators that add or delete a single feature from each state , where the states correspond to the search space commonly used in stepwise method .  
figure 2 shows an example of the state space and operators obtained by stepwise method in a four - feature problem .
the classification algorithms are used to calculate a performance measure for each of 16 different subsets .  
therefore , the wrapper - based approach conducts a best - first search for a good subset by including the classification algorithm itself ( mfnn , naive bayes , or logistic regression ) in the feature subset evaluation .
to search for potential feature subsets , the best - first search starts from an empty feature set and searches forward by greedy hillclimbing augmented with a backtracking technique .  
figure 3 shows how mfnn , naive bayes , and logistic regression were applied in the wrapper - based approach .  
the performance of the prediction models was measured in terms of receiver operating characteristic ( roc ) and area under the roc curve ( auc ) .
the auc of a classifier can be interpreted as the probability of the classifier ranking a randomly chosen positive example higher than a randomly chosen negative one .
most researchers have now adopted auc for evaluating the predictive capability of classifiers since auc is a better performance metric compared to accuracy .
this study used the auc value for performance comparison of different prediction models using the same dataset . the higher the auc , the better the learning performance .
other calculations included sensitivity , the proportion of correctly predicted responders out of all tested responders , and specificity , the proportion of correctly predicted nonresponders out of all tested nonresponders . to investigate the generalization of the prediction models produced by the above algorithms , the repeated 10-fold cross - validation method was used
the model was then trained with nine - tenths of the data and tested by the remaining tenth of data to estimate its predictive performance .
each time , a different tenth of the data was used as testing data , and a different nine - tenths of the data were used as training data . finally ,
the average estimate over all runs was reported by running the above regular 10-fold cross - validation 100 times with different splits of data . in repeated
10-fold cross - validation testing , the performance of all models was evaluated with and without feature selection .
tables 3 and 4 summarize the results of the repeated 10-fold cross - validation experiments for mfnn , naive bayes , and logistic regression using snps with and without feature selection .
first , the auc , sensitivity , and specificity were calculated for the three predictive models without wrapper - based feature selection .  
table 3 shows that the average auc values for the mfnn , the naive bayes , and the logistic regression prediction models were 0.489 , 0.462 and 0.485 , respectively . in terms of auc ,
the the mfnn model ( auc = 0.489 ) outperformed the naive bayes ( auc = 0.462 ) and logistic regression ( auc = 0.485 ) models .  
a repeated 10-fold cross - validation experiment was performed to compare performance in the three wrapper - based predictive algorithms .  
table 4 shows that the mfnn , the naive bayes , and the logistic regression models had average auc values of 0.631 , 0.569 , and 0.620 , respectively . in terms of auc , the mfnn model ( auc = 0.631 ) outperformed both the naive bayes model ( auc = 0.569 ) and the logistic regression model ( auc = 0.620 ) .
out of 11 snps , the wrapper - based mfnn model identified only 4 : rs1800469 ( tgf1 - 509 ) , vntr ( il1_ra ) , rs2227956 ( hsp70 hom ) , and rs1801197 ( ctr ) .  
feature selection using the wrapper - based approach clearly improved performance in the mfnn , the naive bayes , and the logistic regression .
overall , the mfnn classifier with the wrapper - based approach demonstrated superior prediction performance ( auc = 0.631 ) compared to the other models . additionally , the mfnn classifier with wrapper - based feature selection required fewer snps ( n = 4 ) compared to the mfnn classifier without feature selection ( n = 11 ) . 

table 4 shows that the aucs did not significantly differ between the mfnn model with wrapper - based feature selection ( auc = 0.631 ) and the logistic regression model with wrapper - based feature selection ( auc = 0.620 ) .
however , the mfnn classifier with wrapper - based feature selection required fewer snps ( n = 4 ) compared to the logistic regression classifier with wrapper - based feature selection ( n = 8) , that is , by selecting a small number of snps with significantly larger effects compared to other snps and by disregarding relatively insignificant snps , the mfnn model with wrapper - based feature selection successfully identified a subset of four major snps that could be used to predict osteoporosis outcome in the study population ( rs1800469 ( tgf1 - 509 ) , vntr ( il1_ra ) , rs2227956 ( hsp70 hom ) , and rs1801197 ( ctr ) ) . after confirming that the mfnn model outperforms the logistic regression model ,
the next objective was finding the candidate genes and snps that are most promising for diagnosing osteoporosis , designing therapies , and predicting outcome in the studied population of taiwanese women with osteoporosis .
this study compared three classification algorithms , including mfnn , naive bayes , and logistic regression with and without feature selection in terms of accuracy in predicting osteoporosis outcome in a population of taiwanese women .
accounting for models is not a trivial task because even a relatively small set of candidate genes obtains a large number of possible models .
the three classifiers were chosen for comparison because they cover varying techniques with different representational models such as probabilistic mfnn , naive bayes , and logistic regression models .
the proposed procedures can also be implemented using the publicly available software weka   and are thus easily applicable in genomic studies . to the best of our knowledge
, this study is the first to propose the use of three classification algorithms , including mfnn , naive bayes , and logistic regression , and wrapper - based feature selection method for modeling osteoporosis outcome in taiwanese women based on genetic factors such as snps . in this paper ,
the wrapper - based feature selection approach was used to find a subset of snps that maximizes the performance of the prediction model according to how feature selection search is incorporated in the classification algorithms .
the results showed that the mfnn classifier with wrapper - based approach was superior to the other tested algorithms and achieved the greatest auc with the smallest number of snps when distinguishing between high and low bmd in taiwanese women .
these results suggest that mfnn model is a good method of modeling complex nonlinear relationships among clinical factors and the responsiveness of osteoporosis outcome in taiwanese women
. the wrapper - based approach does not require knowledge of the classification algorithm used in the feature selection process , in which features are optimized by using the classification algorithm as part of the evaluation function [ 21 , 23 ] .
another advantage of the wrapper - based method is its inclusion of the interaction between feature subset search and the classification model .
however , the risk of over - fitting is high when using the wrapper - based method [ 21 , 45 ] . in the current study ,
use of the wrapper - based feature selection approach to assess high and low bmd individuals revealed a panel of genetic markers , including tgf1 - 509 , il1_ra , hsp70 hom , and ctr , which were more prominent compared to other markers observed in the examined taiwanese women population with osteoporosis .  
a noted limitation of this study is that , due to the small sample size , the auc values were too low ( < 0.7 ) to obtain good dataset classifications .
therefore , further prospective clinical trials are recommended to determine whether the observed outcome associations with these candidate genes are reproducible in a larger population of taiwanese women with osteoporosis .
this study used an mfnn methodology with wrapper - based feature selection method to predict osteoporosis outcome in taiwanese women based on clinical factors such as snps .
the findings suggest that patients and doctors can use the proposed tool to enhance decision making based on clinical factors such as snp genotyping data .
however , genetic markers require validation in further prospective clinical trials before routine clinical use of genomic analysis for predicting osteoporosis outcome .
 an essential task in a genomic analysis of a human disease is limiting the number of strongly associated genes when studying susceptibility to the disease . 
 the goal of this study was to compare computational tools with and without feature selection for predicting osteoporosis outcome in taiwanese women based on genetic factors such as single nucleotide polymorphisms ( snps ) . to elucidate relationships between osteoporosis and snps in this population , 
 three classification algorithms were applied : multilayer feedforward neural network ( mfnn ) , naive bayes , and logistic regression . 
 a wrapper - based feature selection method was also used to identify a subset of major snps . 
 experimental results showed that the mfnn model with the wrapper - based approach was the best predictive model for inferring disease susceptibility based on the complex relationship between osteoporosis and snps in taiwanese women . 
 the findings suggest that patients and doctors can use the proposed tool to enhance decision making based on clinical factors such as snp genotyping data . 
1. Introduction
2. Materials and Methods
3. Results
4. Discussion
5. Conclusion