Integrating Statistical Predictions and Experimental Verifications for Enhancing Protein-Chemical Interaction Predictions in Virtual Screening
In the early stages of the drug discovery process , prediction of the binding of a chemical compound to a specific protein can be of great benefit in the identification of lead compounds ( candidates for a new drug ) ., Moreover , the effective screening of potential drug candidates at an early stage generates large cost savings at a later stage of the overall drug discovery process ., In the field of virtual screening for the drug discovery , docking analyses and molecular dynamics simulations have been the principal methods used for elucidating the interactions between proteins and small molecules 1–4 ., Fast and accurate statistical prediction methods for binding affinities of any pair of a protein and a ligand have also been proposed for the case where information regarding 3D structures , binding pockets and binding affinities ( e . g . pKi ) for a sufficient number of pairs of proteins and chemical compounds is available 5 ., However , the requirement of these programs for 3D structural information is a severe disadvantage , as the availability of these data is extremely limited ., Although a number of structures in PDB 6 is increasing ( from 23 , 642 structures in 2003 to 48 , 091 structures in 2007 ) , not all proteins which have been derived from many genome-sequencing projects are suitable for experimental structure determination ., Hence , the genome-wide application of these methods is in fact not feasible ., For example , among the GPCRs ( G-protein coupled receptors ) , whose modulation underlies the actions of 30% of the best-known commercial drugs 7 , the full structure of only a few mammalian members , including bovine rhodopsin 8 and human beta 2 adrenoreceptor 9 , is known ., To achieve more comprehensive and faster protein-chemical interaction predictions in the post-genome era producing a vast number of protein sequences whose structural information is not available , it is essential to be able to utilize more readily available biological data and more generally applicable methods which do not require 3D structural data 10–12 ., In our previous study , we developed a comprehensively applicable statistical method for predicting the interactions between proteins and chemical compounds by exploiting very general biological data , including amino acid sequences , 2-dimensional chemical structures , and mass-spectrometry ( MS ) data 11 ., These statistical approaches provided a novel framework where the input space consists of pairs of proteins and chemical compounds ., These pairs are classified into binding and non-binding pairs , while most chemoinformatics approaches assess only chemical compounds and classify them according to their pharmacological effects ., Our previous study 11 demonstrated that screening target proteins for a chemical compound could be performed on a genome-wide scale ., This is due to the fact that our method can be applied to all proteins whose amino acid sequences have been determined even though the 3D structural data is not yet available ., Genome-wide target protein predictions were conducted for MDMA , or ecstasy , which is one of the best known psychoactive drugs , from a pool of 13 , 487 human proteins , and known bindings of MDMA were correctly predicted 11 ., Although the method yielded a relatively high prediction performance ( more than 80% accuracy ) in cross-validation and usefulness in the comprehensive prediction of target proteins for a given chemical compound with tens of thousands of prediction targets 11 , it suffered from the problem of predicting many false positives when comprehensive predictions were conducted ., Although these false positives might include some unknown true positives , they were mainly due to the low quality of the negative data , which is one of the common problems in utilizing statistical classification methods such as Support Vector Machines ( SVMs ) and Artificial Neural Networks ( ANNs ) ., In this paper , we describe two strategies , namely two-layer SVM and reasonable negative data design , which are used for the purpose of reducing the number of false positives and improving the applicability of our method for comprehensive prediction ., In two-layer SVM , in which outputs produced by the first-layer SVM model are utilized as inputs to the second-layer SVM , in order to design negative data which produce fewer false positives , we iteratively constructed SVM models or classification boundaries and selected negative sample candidates according to pre-determined rules ., By using these two strategies , the number of predicted candidates was reduced to around 100 ( Table, 1 ) in experiments in which the potential ligands for some druggable proteins ( UniProt ID P10275 ( androgen receptor ) , P11229 ( muscarinic acetylcholine receptor M1 ) and P35367 ( histamine H1 receptor ) ) are predicted on the basis of more than 100 , 000 compounds in the PubChem Compound database ( http://pubchem . ncbi . nlm . nih . gov/ ) ., With the aim of validating the usefulness of our method , our proposed prediction model with fewer false positives was applied to the PubChem Compound database in order to predict the potential ligands for the “androgen receptor” , which is one of the genes responsible for prostate cancer ., We verified some of these predictions by measuring the IC50 values in an in vitro assay ., Biological experiments , conducted to verify the computational predictions based on statistical methods , docking methods or molecular dynamics methods , typically involve success as well as failure ., In addition to fast calculation and wide applicability , one of the merits of using statistical methods that involve training with known data is that results obtained by verification experiments can be efficiently utilized as feedback to produce new and more reliable predictions ., Most previous work on virtual screening has focused on the computational prediction and listing of dozens or hundreds of candidates , followed by their experimental verification ., However , only on rare occasions have these experimental results been utilized for the further improvement of computational predictions and experiments ., Moreover , even without verification experiments , additional data acquired from , for example , relevant literature can be used for enhancing the prediction reliability ., Therefore , we propose a strategy based on the effective combination of computational prediction and experimental verification ., Our second computational prediction utilizing feedback from the first experimental verification successfully discovered novel ligands ( Figure 1 and, 2 ) for the androgen receptor ., Our approach suggests the significance of utilizing statistical learning methods and feedback from experimental results in drug lead discovery ., In the following section , we first describe the real application of our method involving the computational prediction , the experimental verification and the feedback , and then explain the computational experiments conducted to verify the usefulness of our computational prediction method in comprehensive prediction ., In bioinformatics , statistical approaches extract rules from numerical data corresponding to biological properties ., Here , it is not guaranteed that the extracted rules are biologically valid , and furthermore it is possible to utilize statistical methods to obtain general rules from any kind of numerical data which are meaningless and irrelevant to biological properties ., The biological relevance of our approach can be verified as follows on the basis of supporting evidence which indicates that our method can extract significant rules only if biologically valid and relevant data is given ., First , high prediction performances on diverse datasets might support the validity of our approach ., In several datasets consisting of known pairs of proteins , including nuclear receptors , GPCRs , ion channels and enzymes , and drugs and random protein-drug pairs , our statistical approach with SVM showed high prediction performances ( details are provided in Text S1 , Table S1 and Figure S2 ) ., The fact that more than 0 . 85 AUC and an accuracy of 80% were obtained for diverse datasets suggests that it is possible to extract some properties accountable for interactions between proteins and drugs by statistical approaches ., This possibility can be further supported by the fact that integrating several datasets whose target proteins were not relevant to each other improved the prediction performances with respect to pairs of proteins and chemical compounds which had a specific binding mode ( details are provided in Text S1 and Table S2 ) ., Second , we showed the biological relevance of these high prediction performances by calculating the prediction performances using biologically meaningless artificial datasets as positives ., Several datasets which contained fractions of valid samples found in the DrugBank dataset , and which comprised artificial pseudo-positive samples of protein-chemical pairs produced by shuffling with the same frequency of chemical compounds and proteins as that in the DrugBank dataset , were generated ., Our method was applied to these shuffled artificial datasets ( Figure 3 ) ., Here , if our approach did not depend on the biological properties of the given dataset but only succeeded in classifying given pairs comprising a protein and a chemical compound and random pairs derived from them , the prediction accuracy for each shuffled dataset was assumed not to fluctuate ., As shown in Figure 3 , the prediction accuracy was proportional to the content rate of the biologically valid samples ., Therefore , the classification of our approach was shown to function only when a certain amount of biologically valid pairs comprising a protein and a chemical compound are given ., This result suggests that our statistical approach succeeds in extracting the rules which are only relevant for the biological binding properties ., It is often observed that although statistical learning approaches achieve very high prediction performances in given datasets , statistical prediction models suffer from the problem of generating vast prediction sets including many false positives when applied to a huge dataset , such as the PubChem database ., In our approach , SVM models based on feature vectors directly representing amino acid sequences , chemical structures , and random protein-compound pairs as negatives also produced many predictions and inevitably yielded many false positives ( Table 1A random ) ., Upon the introduction of the two-layer SVM and the negatives designed to overcome this drawback , the prediction precision , or the confidence of positive prediction , was significantly improved in computational experiments based on the DrugBank dataset ( Table 2 ) ., In Table 2 , the external dataset consisted of 170 positives and 2 , 450 negatives that were randomly chosen from 1 , 731 positives and 24 , 500 designed negatives with the mlt rule ( details are provided in Materials and Methods ) and that were excluded in constructing first-layer and second-layer SVM models ., The external dataset contained much more negatives than positives as it simulated the real application of virtual screening with vast databases where only a fraction of chemical compounds in the databases have the effect of interest ., Tables 2A and 2B showed improvement of precision by introducing the designed negatives and the two-layer SVM respectively ., Table 2B also indicated that the application of SVM to outputs of the first-layer SVM models was superior to other statistical learning methods 15 and naive combination of the first-layer SVM models , and that rational selection of the first-layer SVM models achieved significant higher precision ( P-value\u200a=\u200a0 . 0081 by t test ) than randomly selected models ( other comparisons are provided in Text S1 , Table S3 and Table S4 ) ., Particularly , the second-layer SVM utilizing the allpos first-layer SVM models achieved higher precision than use of higher thresholds in the other SVM models ( Table 2C ) ., The high precision contributes to the selection of more reliable predictions and thus to the reduction of the number of false positives ., Following these results on given datasets , our approaches were evaluated with respect to comprehensive binding ligand prediction ., For three proteins ( UniProt ID P10275 ( androgen receptor ) , P11299 ( muscarinic acetylcholine receptor M1 ) and P35367 ( histamine H1 receptor ) ) , their binding ligands were predicted from PubChem Compound 0000001–00125000 which contains 109 , 841 compounds ( Table 1 ) ., Here , P35367 and P11299 are the two most frequently targeted proteins in the DrugBank dataset , and P10275 is a protein of average occurrence in the DrugBank dataset ., Among the 109 , 841 compounds , 47 , 45 , and 5 known ligands were included for P35367 , P11299 , and P10275 , respectively ., As shown in Tables 1A , 1B and 1C , the use of carefully selected negatives , the introduction of the two-layer SVM , and the integration of these two approaches efficiently reduced the number of predictions and thus the number of false positives ., For example , in comparison to Tables 1A and 1C , the number of candidates discovered by using the max dataset in the allpos two-layer SVM approach was about one fiftieth of the number of chemical compounds predicted by using the random negative dataset in the one-layer SVM ., Furthermore , in comparison to other approaches based solely on the use of chemical compounds ( Tables 1D and 1E ) , our approaches gave a reasonable number of predictions ( other comparisons are described in Text S1 and Tables S5 , S6 , S7 ) ., These results suggest that our prediction models select a reasonable number of ligand candidates from all chemical compounds in large databases and encourage the comprehensive binding ligand prediction for the target protein ., The experimental verification of the computational predictions produces feedback data or samples which are not included in the given training datasets ., The efficient utilization of these data can contribute to the fast identification of compounds with the desired properties and can be of advantage to statistical learning approaches ., We compared several strategies for utilizing feedback data as follows ., For three proteins ( UniProt ID P10275 ( androgen receptor ) , P11299 ( muscarinic acetylcholine receptor M1 ) and P353367 ( histamine H1 receptor ) ) , ligand data which were not included in the DrugBank dataset were collected from relevant literature 16–18 and public databases , PDSP Ki database 19 and GLIDA 20 , in February 2008 ., Overall , 35 androgen receptor-ligand pairs , 49 muscarinic acetylcholine receptor M1-ligand pairs , and 1 , 060 histamine H1 receptor-ligand pairs were supplemented ., Additional models were constructed by using these supplemental pairs as positives ( details are provided in Text S1 ) ., As shown in Figure 4 , the use of the additional model with a sufficient weighting factor controlled the increase of the predictions with a slight decrease of the recall rate ., The use of large weighting factors results in the relative decrease of the influence of other first-layer SVM models derived from the DrugBank dataset in classification ., However , the low performance of “only additional model:st2” , shown in Figure 4A , where only one first-layer SVM model derived from additional data was used to construct the second-layer SVM model , indicates the need for first-layer SVM models derived from the DrugBank dataset as well as combinations of these first-layer SVM models with an additional first-layer SVM model ., With this efficient strategy for utilizing feedback data , computational prediction and experimental verification improve each other to enable faster search toward the identification of useful small molecules ., We proposed a comprehensively applicable computational method for predicting the interactions between proteins and chemical compounds , in which the number of false positives was reduced in comparison to other methods ., Furthermore , we proposed the strategy for the efficient utilization of experimental feedback and the integration of computational prediction and experimental verification ., The application of our method to the androgen receptor resulted in 67% ( 4/6 ) prediction precision according to in vitro experimental verification in the first computational prediction and 60% ( 3/5 ) in the second prediction , which included the feedback of the first experimental verification ., However , these relatively low precision values do not represent the true statistical significance of the method ., This 60–70% precision can also be evaluated by using the following P-value . Here , N is the number of prediction targets , M the number of ligands potentially binding to the target proteins , t is the number of tested compounds , and p is the number of true positives ., With N\u200a=\u200a19171127 , which is the number of chemical compounds in the PubChem Compound database , and M\u200a=\u200a19171127× ( 456/3000 ) × ( 7/964 ) ≒21160 , which is based on the optimistic assumption that all compounds can be regarded as potential drugs for some target protein , it is estimated that 3 , 000 druggable proteins exist 21 ., Moreover , the distribution of target proteins and drugs in the DrugBank dataset , consisting of 456 target proteins and 964 drugs , including 7 known ligands for the human androgen receptor , and P-values of 2∶21×10−11 and 1∶34×10−8 are obtained for the prediction precision of the first and the second computational prediction , respectively ., These extremely small P-values prove the significance of the virtual screening and its precision in the drug discovery process ., These prediction performances are as good as or better than several previous virtual screening studies based mainly on docking analyses 22–24 ., For example , at a threshold of 100 µM , 7% precision ( 3/39 ) for Mycobacterium tuberculosis adenosine 5′-phosphosulfate reductase 22 , 71% precision ( 22/31 ) for Staphylococcus aureus methyonyl-tRNA synthetase 23 and 8% precision ( 16/192 ) for human DNA ligase I 24 were obtained , respectively ., In addition , 0 . 566 AUC was achieved in the docking analysis using AutoDock 3 ( Figure 5 ) for the 17 chemical compounds ( 12 chemical compounds verified in the first experimental verification , with the exception of 6 known drugs , and 5 chemical compounds verified in the second experimental verification ) ., In contrast , 0 . 681 AUC was obtained with our method ., Here , in the calculation of AUC , the threshold level of IC50\u200a=\u200a100 µM for experimental verification was used to define a label ( binding or non-binding ) for each chemical compound , and or the predicted probability was regarded as a value for each molecule ., Note that the docking analysis with AutoDock was not applied to the 19 , 171 , 127 compounds in the PubChem Compound database for the screening purpose , but was applied only to 17 compounds , which were the results of virtual screening by our method ., In terms of computational time , for binding prediction of one pair of a protein and a chemical compound , using one Opteron 275 2 . 2 GHz CPU , AutoDock took approximately 100 minutes on average with 100 genetic algorithm ( GA ) runs , while our method required less than 0 . 3 seconds ., These computational time comparisons indicate that our method can perform a virtual screening of more than 19 million chemical compounds from the PubChem Compound database for any proteins in genome-wide scale and this immense screening task would be infeasible to accomplish with any of the existing docking methods ., Therefore , our statistical approach can contribute as the first fast and rather accurate virtual screening tool for the drug discovery process ., It can be followed by the application of more time-consuming but more informative approaches , such as docking analysis and molecular dynamics analysis , which can provide information regarding the binding affinities and the molecular binding mechanisms to outputs of the first screening ., In another perspective , the re-evaluation of statistical prediction approaches by using 23 chemical compounds experimentally verified in this study showed that our proposed methods , which utilized information of both protein sequence and chemical structures , were superior to a conventional LBVS ( Ligand Based Virtual Screening ) method where only structures of specific chemical compounds were considered ( Figure 6 ) ., As shown in Figure 6A , our proposed methods ( “one-layer SVM” , “two-layer SVM-subpos” and “two-layer SVM-allpos” ) achieved a higher recall rate at ranks higher than 500 compared to a conventional Ligand Based Virtual Screening method ( “only compound SVM” in Figure 6A ) ., The fact that experimentally verified chemical compounds were identified at higher ranks in the pool by our proposed prediction models suggests that our proposed models were highly efficient with respect to the screening method ., Figure 6B also shows that our proposed methods were more successful at discriminating between 15 experimentally verified binding and 8 non-binding ligands better than the LBVS method ., These comparisons suggest that our proposed method utilizing information of protein sequences as well as chemical structures can be regarded as a more useful substitute for usual ligand-based virtual screening methods utilizing only chemical structures ., Furthermore , the fact that the second computational prediction , or the use of feedback data , contributed to the discovery of novel ligands ( Figure 2B–D ) supports the utilization of statistical learning methods in virtual screening ., Regarding the computational prediction method used in this paper , we made the method available to the public as a web-based service named COPICAT ( COmprehensive Predictor of Interactions between Chemical compounds And Target proteins; http://copicat . dna . bio . keio . ac . jp/ ) ., The DrugBank dataset was constructed from Approved DrugCards data , which were downloaded in February , 2007 from the DrugBank database 25 ., These data consist of 964 approved drugs and their 456 associated target proteins , constituting 1 , 731 interacting pairs or positives ., Given Np positive and Nn negative samples in known data and Mp positives and Mn negatives in additional or feedback data , a straightforward strategy for the integration of additional data into statistical training , such as SVM , is to train a statistical model based on a dataset consisting of Np+Mp positives and Nn+Mn negatives ., When the two-layer SVM strategy is applied , another strategy of feedback and supplement involves the utilization of an additional model based on additional data ., In this strategy , the second-layer SVM is trained on the basis of Np+Mp positives and Nn+Mn negatives , and a sample si in the second layer is represented as follows , Here , is an output of the additional model trained on the basis of Mp positives and Mn negatives ., is an output of the first-layer SVM model j , and is a weighting factor ., AutoDock 4 3 was applied to the human androgen receptor ligand-binding domain ( PDB code; 2AM9 31 ) and tested compounds whose 3D structure was generated by Obgen in the Open Babel package ver . 2 . 2 . 0 32 or CORINA 33 ., The conditions of AutoDock followed Jenwitheesuk and Samudrala , 2005 34 ., ARG752 of 2AM9 , which was considered important for the binding of androgens by the human androgen receptor 31 , was set to a flexible residue in AutoDock .
Predictions of interactions between target proteins and potential leads are of great benefit in the drug discovery process ., We present a comprehensively applicable statistical prediction method for interactions between any proteins and chemical compounds , which requires only protein sequence data and chemical structure data and utilizes the statistical learning method of support vector machines ., In order to realize reasonable comprehensive predictions which can involve many false positives , we propose two approaches for reduction of false positives:, ( i ) efficient use of multiple statistical prediction models in the framework of two-layer SVM and, ( ii ) reasonable design of the negative data to construct statistical prediction models ., In two-layer SVM , outputs produced by the first-layer SVM models , which are constructed with different negative samples and reflect different aspects of classifications , are utilized as inputs to the second-layer SVM ., In order to design negative data which produce fewer false positive predictions , we iteratively construct SVM models or classification boundaries from positive and tentative negative samples and select additional negative sample candidates according to pre-determined rules ., Moreover , in order to fully utilize the advantages of statistical learning methods , we propose a strategy to effectively feedback experimental results to computational predictions with consideration of biological effects of interest ., We show the usefulness of our approach in predicting potential ligands binding to human androgen receptors from more than 19 million chemical compounds and verifying these predictions by in vitro binding ., Moreover , we utilize this experimental validation as feedback to enhance subsequent computational predictions , and experimentally validate these predictions again ., This efficient procedure of the iteration of the in silico prediction and in vitro or in vivo experimental verifications with the sufficient feedback enabled us to identify novel ligand candidates which were distant from known ligands in the chemical space .
This work describes a statistical method that identifies chemical compounds binding to a target protein given the sequence of the target or distinguishes proteins to which a small molecule binds given the chemical structure of the molecule ., As our method can be utilized for virtual screening that seeks for lead compounds in drug discovery , we showed the usefulness of our method in its application to the comprehensive prediction of ligands binding to human androgen receptors and in vitro experimental verification of its predictions ., In contrast to most previous virtual screening studies which predict chemical compounds of interest mainly with 3D structure-based methods and experimentally verify them , we proposed a strategy to effectively feedback experimental results for subsequent predictions and applied the strategy to the second predictions followed by the second experimental verification ., This feedback strategy makes full use of statistical learning methods and , in practical terms , gave a ligand candidate of interest that structurally differs from known drugs ., We hope that this paper will encourage reevaluation of statistical learning methods in virtual screening and that the utilization of statistical methods with efficient feedback strategies will contribute to the acceleration of drug discovery .
chemical biology, mathematics/statistics, pharmacology/drug development, computational biology