Proc . Natl. Acad . Sci . USA Vol . 95 , pp. 14863 14868 , December 1998 Genetics Cluster analysis and display of genome-wide expression patterns MICHAEL B. EISEN * , PAUL T. SPELLMAN * , PATRICK O. BROWN , AND DAVID BOTSTEIN * * Department of Genetics and Department of Biochemistry and Howard Hughes Medical Institute , Stanford University School of Medicine , 300 Pasteur Avenue , Stanford , CA 94305 Contributed by David Botstein , October 13 , 1998 ABSTRACT A system of cluster analysis for genome-wide expression data from DNA microarray hybridization is described that uses standard statistical algorithms to arrange genes according to similarity in pattern of gene expression . The output is displayed graphically , conveying the clustering and the underlying expression data simultaneously in a form intuitive for biologists . We have found in the budding yeast Saccharomyces cerevisiae that clustering gene expression data groups together efficiently genes of known similar function , and we find a similar tendency in human data . Thus patterns seen in genome-wide expression experiments can be interpreted as indications of the status of cellular processes . Also , coexpression of genes of known function with poorly characterized or novel genes may provide a simple means of gaining leads to the functions of many genes for which information is not available currently . The rapid advance of genome-scale sequencing has driven the development of methods to exploit this information by characterizing biological processes in new ways . The knowledge of the coding sequences of virtually every gene in an organism , for instance , invites development of technology to study the expression of all of them at once , because the study of gene expression of genes one by one has already provided a wealth of biological insight . To this end , a variety of techniques has evolved to monitor , rapidly and efficiently , transcript abundance for all of an organism 's genes -LRB- 1 3 -RRB- . Within the mass of numbers produced by these techniques , which amount to hundreds of data points for thousands or tens of thousands of genes , is an immense amount of biological information . In this paper we address the problem of analyzing and presenting information on this genomic scale . A natural first step in extracting this information is to examine the extremes , e.g. , genes with significant differential expression in two individual samples or in a time series after a given treatment . This simple technique can be extremely efficient , for example , in screens for potential tumor markers or drug targets . However , such analyses do not address the full potential of genome-scale experiments to alter our understanding of cellular biology by providing , through an inclusive analysis of the entire repertoire of transcripts , a continuing comprehensive window into the state of a cell as it goes through a biological process . What is needed instead is a holistic approach to analysis of genomic data that focuses on illuminating order in the entire set of observations , allowing biologists to develop an integrated understanding of the process being studied . A natural basis for organizing gene expression data is to group together genes with similar patterns of expression . The first step to this end is to adopt a mathematical description of similarity . For any series of measurements , a number of sensible measures of similarity in the behavior of two genes can The publication costs of this article were defrayed in part by page charge payment . This article must therefore be hereby marked `` advertisement '' in accordance with 18 U.S.C. 1734 solely to indicate this fact . 1998 by The National Academy of Sciences 0027-8424 98 9514863 - 6 $ 2.00 0 PNAS is available online at www.pnas.org . be used , such as the Euclidean distance , angle , or dot products of the two n-dimensional vectors representing a series of n measurements . We have found that the standard correlation coefficient -LRB- i.e. , the dot product of two normalized vectors -RRB- conforms well to the intuitive biological notion of what it means for two genes to be `` coexpressed ; '' this may be because this statistic captures similarity in `` shape '' but places no emphasis on the magnitude of the two series of measurements . It is not the purpose of this paper to survey the various methods available to cluster genes on the basis of their expression patterns , but rather to illustrate how such methods can be useful to biologists in the analysis of gene expression data . We aim to use these methods to organize , but not to alter , tables containing primary data ; we have thus used methods that can be reduced , in the end , to a reordering of lists of genes . Clustering methods can be divided into two general classes , designated supervised and unsupervised clustering -LRB- 4 -RRB- . In supervised clustering , vectors are classified with respect to known reference vectors . In unsupervised clustering , no predefined reference vectors are used . As we have little a priori knowledge of the complete repertoire of expected gene expression patterns for any condition , we have favored unsupervised methods or hybrid -LRB- unsupervised followed by supervised -RRB- approaches . Although various clustering methods can usefully organize tables of gene expression measurements , the resulting ordered but still massive collection of numbers remains difficult to assimilate . Therefore , we always combine clustering methods with a graphical representation of the primary data by representing each data point with a color that quantitatively and qualitatively reflects the original experimental observations . The end product is a representation of complex gene expression data that , through statistical organization and graphical display , allows biologists to assimilate and explore the data in a natural intuitive manner . To illustrate this approach , we have applied pairwise average-linkage cluster analysis -LRB- 5 -RRB- to gene expression data collected in our laboratories . This method is a form of hierarchical clustering , familiar to most biologists through its application in sequence and phylogenetic analysis . Relationships among objects -LRB- genes -RRB- are represented by a tree whose branch lengths reflect the degree of similarity between the objects , as assessed by a pairwise similarity function such as that described above . In sequence comparison , these methods are used to infer the evolutionary history of sequences being compared . Whereas no such underlying tree exists for expression patterns of genes , such methods are useful in their ability to represent varying degrees of similarity and more distant relationships among groups of closely related genes , as well as in requiring few assumptions about the nature of the data . The computed trees can be used to order genes in the original data table , so that genes or groups of genes with similar expression patterns are adjacent . The ordered table can then be displayed graphically , as above , with a representation of the tree to indicate the relationships among genes . To whom reprint requests should be addressed . e-mail : botstein @ genome.stanford.edu . 14863 14864 Genetics : Eisen et al. . Proc . Natl. Acad . Sci . USA 95 -LRB- 1998 -RRB- pairs of genes . The matrix is scanned to identify the highest value -LRB- representing the most similar pair of genes -RRB- . A node is created joining these two genes , and a gene expression profile is computed for the node by averaging observation for the joined elements -LRB- missing values are omitted and the two joined elements are weighted by the number of genes they contain -RRB- . The similarity matrix is updated with this new node replacing the two joined elements , and the process is repeated n-1 times until only a single element remains . Software implementation of this algorithm can be obtained from the authors at http : rana.stanford.edu clustering . Ordering of Data Tables . For any dendrogram of n elements , there are 2n 1 linear orderings consistent with the structure of the tree -LRB- at each node , either of the two elements joined by the node can be ordered ahead of the other -RRB- . An optimal linear ordering , one that maximizes the similarity of adjacent elements in the ordering , is impractical to compute . However , to consistently arrange nodes between analyses , we use simple methods of weighting genes , such as average expression level , time of maximal induction , or chromosomal position , and we place the element with the lower average weight earlier in the final ordering . Display . Using any ordering , the primary data table is represented graphically by coloring each cell on the basis of the measured fluorescence ratio . Cells with log ratios of 0 -LRB- ratios of 1.0 genes unchanged -RRB- are colored black , increasingly positive log ratios with reds of increasing intensity , and increasingly negative log ratios with greens of increasing intensity . A representation of the dendrogram is appended to the colored table to indicate the nature of the computed relationship among genes in the table . MATERIALS AND METHODS Sources of Experimental Data . Data analyzed here were collected on spotted DNA microarrays -LRB- 6 , 7 -RRB- . Gene expression in the budding yeast Saccharomyces cerevisiae was studied during the diauxic shift -LRB- 8 -RRB- , the mitotic cell division cycle -LRB- 9 -RRB- , sporulation -LRB- 10 -RRB- , and temperature and reducing shocks -LRB- P.T.S. , P.O.B. , and D.B. , unpublished results -RRB- by using microarrays containing essentially every ORF from this fully sequenced organism -LRB- 8 -RRB- . Gene expression of primary human fibroblasts stimulated with serum following serum starvation was studied by using a microarray with 9,800 cDNAs representing approximately 8,600 distinct human transcripts -LRB- 11 -RRB- . In all experiments , RNA from experimental samples -LRB- taken at selected times during the process -RRB- was labeled during reverse transcription with the red-fluorescent dye Cy5 -LRB- Amersham -RRB- and was mixed with a reference sample labeled in parallel with the green-fluorescent dye Cy3 -LRB- Amersham -RRB- -LRB- the reference sample was time 0 for all experiments , except for the yeast cell cycle where asynchronous cells were used -RRB- . After hybridization and appropriate washing steps , separate images were acquired for each fluor , and fluorescence intensity ratios were obtained for all target elements . We maintain and update master data tables for all experiments conducted in our labs ; in these tables , rows represent all genes for which data has been collected , columns represent individual array experiments -LRB- e.g. , single timepoints or conditions -RRB- , and each cell represents the measured Cy5 Cy3 fluorescence ratio at the corresponding target element on the appropriate array . All ratio values are log transformed -LRB- base 2 for simplicity -RRB- to treat inductions or repressions of identical magnitude as numerically equal but with opposite sign . Individual observations are rejected in some cases , on the basis of measurement quality parameters produced by the image analysis software . For the analyses presented here , smaller tables containing selected genes and experiments were produced ; full descriptions are given in the appropriate figure legends . These tables are available on the PNAS web site at -LRB- www.pnas.org -RRB- or at http : rana.stanford.edu clustering . Metrics . The gene similarity metric we use is a form of correlation coefficient . Let Gi equal the -LRB- log-transformed -RRB- primary data for gene G in condition i. For any two genes X and Y observed over a series of N conditions , a similarity score can be computed as follows : S X , Y where G RESULTS We applied this method to two sets of data , a single time course -LRB- Fig. 1 -RRB- of a canonical model of the growth response in human cells -LRB- 11 -RRB- and an aggregation of data from experiments on the budding yeast S. cerevisiae -LRB- Fig. 2 -RRB- , including time courses of the mitotic cell division cycle -LRB- 9 -RRB- , sporulation -LRB- 10 -RRB- , the diauxic shift -LRB- 8 -RRB- , and shock responses -LRB- P.T.S. , P.O.B. , and D.B. , unpublished results -RRB- . A striking property of the clustered images in Figs. 1 and 2 is the presence of large contiguous patches of color representing groups of genes that share similar expression patterns over multiple conditions . To verify that this structure is of biological origin and is not an artifact of the clustering procedure , the initial data from the human growth response experiment were randomized in three different ways and were clustered by using the same procedure -LRB- Fig. 3 -RRB- . No similar structure resulted from any of these randomized data sets , indicating that the patterns seen in Figs. 1 and 2 depict biological order in the gene expression response of the organism during the studied processes . A central feature of Figs. 1 and 2 is that one can look at such images , identify patterns of interest , and readily zoom in on the detailed expression patterns and identities of the genes contributing to these patterns . An important test of the value of this approach comes when we examine the identity of the clustered genes at varying levels of identity . Redundant Representations of Genes Cluster Together . At the finest level , we have found repeatedly that genes represented by more than one array element or genes with high degrees of sequence identity are clustered next to , or in the immediate vicinity of , each other . Thus the exact representation of a gene on the array -LRB- alternate cDNA clones of differing length in the case of human arrays or highly homologous genes in S. cerevisiae -RRB- makes little difference in the observed pattern of gene expression . Moreover , even though groups of genes may show very similar patterns of expression , as is seen in Figs. 1 and 2 , in general individual genes can be distinguished from all other genes on the basis of subtle differences in their 1 N Xi i 1 , N X offset X Yi Y offset Y Gi i 1 , N G offset 2 . N When Goffset is set to the mean of observations on G , then G becomes the standard deviation of G , and S -LRB- X , Y -RRB- is exactly equal to the Pearson correlation coefficient of the observations of X and Y. Values of Goffset which are not the average over observations on G are used when there is an assumed unchanged or reference state represented by the value of Goffset , against which changes are to be analyzed ; in all of the examples presented here , Goffset is set to 0 , corresponding to a fluorescence ratio of 1.0 . Hierarchical Clustering . The hierarchical clustering algorithm used is based closely on the average-linkage method of Sokal and Michener -LRB- 5 -RRB- , which was developed for clustering correlation matrixes such as those used here . The object of this algorithm is to compute a dendrogram that assembles all elements into a single tree . For any set of n genes , an upper-diagonal similarity matrix is computed by using the metric described above , which contains similarity scores for all Genetics : Eisen et al. . Proc . Natl. Acad . Sci . USA 95 -LRB- 1998 -RRB- 14865 regulation . Finally , this result also indicates that noise present in single observations does not contribute significantly when genes are compared across even a relatively small number of nonidentical conditions . Therefore , when designing experiments , it may be more valuable to sample a wide variety of conditions than to make repeat observations on identical conditions . Genes of Similar Function Cluster Together . A far more striking result is found when larger groups of clustered genes are examined , where we observe a strong tendency for these genes to share common roles in cellular processes . This relationship is clearest in data from experiments on the budding yeast S. cerevisiae , where arrays representing essentially all of the genes from this organism are available -LRB- 8 -RRB- and for which a large fraction of the identified genes -LRB- more than 35 % -RRB- have been studied in some detail . Fig. 2 A represents a clustering analysis of 2,467 genes , all the genes that currently have a functional annotation in the Saccharomyces Genome Database -LRB- 12 -RRB- . As can be seen in Fig. 2 B K , numerous groups of coexpressed genes representing diverse expression patterns across the sampled conditions are involved in common cellular processes . Although one might be concerned about the possibility of crosshybridization , it is clear in the examples below that genes of unrelated sequence but similar function cluster tightly together . A particularly dramatic example is the extensive cluster -LRB- shown in Fig. 2I -RRB- of 126 genes strongly down-regulated in response to stress -LRB- after each of the shocks , at the latter stages of the diauxic shift where glucose levels are diminished , and after transfer to nutrient-limited sporulation media -RRB- , and which covary throughout the cell cycle . This cluster is dominated by genes encoding ribosomal proteins -LRB- 112 genes -RRB- and other proteins involved in translation -LRB- initiation and elongation factors and tRNA synthetases -RRB- . It has been reported that yeast responds to favorable growth conditions by increasing the production of ribosomes -LRB- 13 -RRB- through transcriptional regulation of genes encoding ribosomal proteins -LRB- 14 -RRB- . Mitochondrial protein synthesis genes were also expressed concordantly along with a number of genes involved in respiration -LRB- Fig. 2F -RRB- in a pattern roughly similar to a large cluster of coexpressed genes involved in ATP synthesis -LRB- predominantly members of the F1F0 ATPase complex -RRB- and oxidative phosphorylation -LRB- Fig. 2G -RRB- . Oxygen-related transcriptional regulation of genes involved in oxygen utilization has been characterized extensively -LRB- 15 -RRB- . The genes encoding the bulk of the components of the proteasome -LRB- Fig. 2C -RRB- and the mini-chromosome maintenance DNA replication complex -LRB- Fig. 2 J -RRB- are also coexpressed . In addition , there are many examples of coexpressed genes that share a common or related function but are not members of large protein complexes , such as genes encoding numerous glycolytic enzymes -LRB- Fig. 2E -RRB- , genes involved in the tricarboxhuman genes . All measurements are relative to time 0 . Genes were selected for this analysis if their expression level deviated from time 0 by at least a factor of 3.0 in at least 2 time points . The dendrogram and colored image were produced as described in the text ; the color scale ranges from saturated green for log ratios 3.0 and below to saturated red for log ratios 3.0 and above . Each gene is represented by a single row of colored boxes ; each time point is represented by a single column . Five separate clusters are indicated by colored bars and by identical coloring of the corresponding region of the dendrogram . As described in detail in ref . 11 , the sequence-verified named genes in these clusters contain multiple genes involved in -LRB- A -RRB- cholesterol biosynthesis , -LRB- B -RRB- the cell cycle , -LRB- C -RRB- the immediate early response , -LRB- D -RRB- signaling and angiogenesis , and -LRB- E -RRB- wound healing and tissue remodeling . These clusters also contain named genes not involved in these processes and numerous uncharacterized genes . A larger version of this image , with gene names , is available at http : rana.stanford.edu clustering serum.html . FIG. 1 . Clustered display of data from time course of serum stimulation of primary human fibroblasts . Experimental details are described elsewhere -LRB- 11 -RRB- . Briefly , foreskin fibroblasts were grown in culture and were deprived of serum for 48 hr . Serum was added back and samples taken at time 0 , 15 min , 30 min , 1 hr , 2 hr , 3 hr , 4 hr , 8 hr , 12 hr , 16 hr , 20 hr , 24 hr . The final datapoint was from a separate unsynchronized sample . Data were measured by using a cDNA microarray with elements representing approximately 8,600 distinct 14866 Genetics : Eisen et al. . Proc . Natl. Acad . Sci . USA 95 -LRB- 1998 -RRB- FIG. 2 . -LRB- Legend appears at the bottom of the opposite page . -RRB- Genetics : Eisen et al. ylic acid cycle and oxidative phosphorylation -LRB- Fig. 2K -RRB- , and genes involved in mating -LRB- not shown -RRB- . These examples emphasize that the observed coregulation occurs primarily at the level of cellular function and not only with the exact protein function -LRB- e.g. , enzymic reaction catalyzed -RRB- of the gene product . Finally , there is an extremely tight cluster of eight histone genes -LRB- duplicates of each of histones H2A , H2B , H3 , and H4 -RRB- . It is well known that these genes are coregulated and are transcribed at a particular point in the cell cycle -LRB- 16 -RRB- . In human data sets , relationships among the functions of genes in clusters are obscured somewhat by the less complete functional annotation of human gene sequences . Nonetheless , when the composition of the clusters is examined , they are often found to contain genes known to share a common role in the cell . This observation is well illustrated in the data from the response of human tissue culture cells to serum after serum starvation -LRB- Fig. 1 -RRB- . When available functional information on the genes studied in this experiment was examined , keeping in mind the often poor state of annotation of the human genome , the clusters of genes indicated by colored branches were found to generally contain genes involved in cholesterol biosynthesis -LRB- cluster A -RRB- , the cell cycle -LRB- cluster B -RRB- , the immediate early response -LRB- cluster C -RRB- , signaling and angiogenesis -LRB- cluster D -RRB- , and tissue remodeling and wound healing -LRB- cluster E -RRB- ; a detailed description of these observations is contained in ref . 11 . Proc . Natl. Acad . Sci . USA 95 -LRB- 1998 -RRB- 14867 FIG. 3 . To demonstrate the biological origins of patterns seen in Figs. 1 and 2 , data from Fig. 1 were clustered by using methods described here before and after random permutation within rows -LRB- random 1 -RRB- , within columns -LRB- random 2 -RRB- , and both -LRB- random 3 -RRB- . DISCUSSION Microarray-based genomic surveys and other high-throughput approaches -LRB- ranging from genomics to combinatorial chemistry -RRB- are becoming increasingly important in biology and chemistry . As a result , we need to develop our ability to `` see '' the information in the massive tables of quantitative measurements that these approaches produce . Our approach to this problem can be generalized as follows . First , we use a commonsense approach to organize the data , based on order inherent in the data . Next , recognizing that the rate-limiting step in exploring and searching large tables of numerical data is a trivial one : reading the numbers -LRB- human brains are not well adapted to assimilating quantitative data by reading digits -RRB- , we represent the quantitative values in the table by using a naturalistic color scale rather than numbers . This alternative encoding preserves all the quantitative information , but transmits it to our brains by way of a much higher-bandwidth channel than the `` number-reading '' channel . A natural way of viewing complex data sets is first to scan and survey the large-scale features and then to focus in on the interesting details . What we have found to be the most valuable feature of the approach described here is that it allows this natural and intuitive process to be applied to genomic data sets . The approach is a general one , with no inherent specificity to the particular method used to acquire data or even to gene-expression data . It is therefore likely that very similar approaches may be applied to many other kinds of very large data sets . In each case , it may be necessary to find alternative algorithms and computation methods to bring out inherent structures in the data , and , equally important , to find dense naturalistic visual representations that convey the quantitative information effectively . We recognize that the particular clustering algorithm we used is not the only , or even the best , method available . We have used and are actively exploring alternatives such as parametric ordering of genes -LRB- 9 -RRB- and supervised clustering methods based on representative handpicked or computer-generated expression profiles -LRB- 10 -RRB- . The success of these very simple approaches has given us confidence to face the coming flood of functional genomic data . The examples presented here demonstrate a feature of gene expression that makes these methods particularly useful , namely the tendency of expression data to organize genes into functional categories . It is , of course , not very surprising that genes that are expressed together share common functions . Nonetheless , the extent to which gene expression patterns suffice to separate genes into functional categories across a relatively small and redundant collection of conditions is surprising . It seems likely that the addition of more and diverse conditions can only enhance these observations . When the clustering analysis described here is FIG. 2 . -LRB- On the opposite page . -RRB- Cluster analysis of combined yeast data sets . Data from separate time courses of gene expression in the yeast S. cerevisiae were combined and clustered . Data were drawn from time courses during the following processes : the cell division cycle -LRB- 9 -RRB- after synchronization by alpha factor arrest -LRB- ALPH ; 18 time points -RRB- ; centrifugal elutriation -LRB- ELU ; 14 time points -RRB- , and with a temperature-sensitive cdc15 mutant -LRB- CDC15 ; 15 time points -RRB- ; sporulation -LRB- 10 -RRB- -LRB- SPO , 7 time points plus four additional samples -RRB- ; shock by high temperature -LRB- HT , 6 time points -RRB- ; reducing agents -LRB- D , 4 time points -RRB- and low temperature -LRB- C ; 4 time points -RRB- -LRB- P. T. S. , J. Cuoczo , C. Kaiser , P.O. B. , and D. B. , unpublished work -RRB- ; and the diauxic shift -LRB- 8 -RRB- -LRB- DX , 7 time points -RRB- . All data were collected by using DNA microarrays with elements representing nearly all of the ORFs from the fully sequenced S. cerevisiae genome -LRB- 8 -RRB- ; all measurements were made against a time 0 reference sample except for the cell-cycle experiments , where an unsynchronized sample was used . All genes -LRB- 2,467 -RRB- for which functional annotation was available in the Saccharomyces Genome Database were included -LRB- 12 -RRB- . The contribution to the gene similarity score of each sample from a given process was weighted by the inverse of the square root of the number of samples analyzed from that process . The entire clustered image is shown in A ; a larger version of this image , along with dendrogram and gene names , is available at http : rana.stanford.edu clustering yeastall.html . Full gene names are shown for representative clusters containing functionally related genes involved in -LRB- B -RRB- spindle pole body assembly and function , -LRB- C -RRB- the proteasome , -LRB- D -RRB- mRNA splicing , -LRB- E -RRB- glycolysis , -LRB- F -RRB- the mitochondrial ribosome , -LRB- G -RRB- ATP synthesis , -LRB- H -RRB- chromatin structure , -LRB- I -RRB- the ribosome and translation , -LRB- J -RRB- DNA replication , and -LRB- K -RRB- the tricarboxylic acid cycle and respiration . The full-color range represents log ratios of 1.2 to 1.2 for the cell-cycle experiments , 1.5 to 1.5 for the shock experiments , 2.0 to 2.0 for the diauxic shift , and 3.0 to 3.0 for sporulation . Gene name , functional category , and specific function are from the Saccharomyces Genome Database -LRB- 13 -RRB- . Cluster I contains 112 ribosomal protein genes , seven translation initiation or elongation factors , three tRNA synthetases , and three genes of apparently unrelated function . 14868 Genetics : Eisen et al. 2 . 3 . 4 . 5 . 6 . 7 . 8 . 9 . 10 . 11 . 12 . 13 . 14 . 15 . 16 . Proc . Natl. Acad . Sci . USA 95 -LRB- 1998 -RRB- Velculescu , V. E. , Zhang , L. , Vogelstein , B. & Kinzler , K. W. -LRB- 1995 -RRB- Science 270 , 484 487 . Lockhart , D. J. , Dong , H. , Byrne , M. C. , Follettie , M. T. , Gallo , M. V. , Chee , M. S. , Mittmann , M. , Wang , C. , Kobayashi , M. , Horton , H. , et al. -LRB- 1996 -RRB- Nat . Biotechnol . 14 , 1675 1680 . Kohonen , T. -LRB- 1997 -RRB- Self-Organizing Maps -LRB- Springer , New York -RRB- . Sokal , R. R. & Michener , C. D. -LRB- 1958 -RRB- Univ. . Kans. . Sci . Bull . 38 , 1409 1438 . Schena , M. , Shalon , D. , Heller , R. , Chai , A. , Brown , P. O. & Davis , R. W. -LRB- 1996 -RRB- Proc . Natl. Acad . Sci . USA 93 , 10614 10619 . Shalon , D. , Smith , S. J. & Brown , P. O. -LRB- 1996 -RRB- Genome Res . 6 , 639 645 . DeRisi , J. L. , Iyer , V. R. & Brown , P. O. -LRB- 1997 -RRB- Science 278 , 680 686 . Spellman , P. T. , Sherlock , G. , Iyer , V. R. , Zhang , M. , Anders , K. , Eisen , M. B. , Brown , P. O. , Botstein , D. & Futcher , B. -LRB- 1998 -RRB- . Mol . Biol . Cell , in press . Chu , S. , DeRisi , J. , Eisen , M. , Mulholland , J. , Botstein , D. , Brown , P. O. & Herskowitz , I. -LRB- 1998 -RRB- Science 282 , 699 705 . Iyer , V. R. , Eisen , M. B. , Ross , D. R. , Schuler , G. , Moore , T. , Lee , J. C. F. , Trent , J. M. , Hudson , J. , Boguski , M. , Lashkari , D. , et al. -LRB- 1998 -RRB- Science , in press . Cherry , J. M. , Ball , C. , Weng , S. , Juvik , G. , Schmidt , R. , Adler , C. , Dunn , B. , Dwight , S. , Riles , L. , Mortimer , R. K. , et al. -LRB- 1997 -RRB- Nature -LRB- London -RRB- 387 , 67 73 . Kief , D. R. & Warner , J. R. -LRB- 1981 -RRB- Mol . Cell . Biol . 1 , 1007 1015 . Kraakman , L. S. , Griffioen , G. , Zerp , S. , Groeneveld , P. , Thevelein , J. M. , Mager , W. H. & Planta , R. J. -LRB- 1993 -RRB- Mol . Gen. Genet . 239 , 196 204 . Kwast , K. E. , Burke , P. V. & Poyton , R. O. -LRB- 1998 -RRB- J. Exp . Biol . 201 , 1177 1195 . Hereford , L. M. , Osley , M. A. , Ludwig , T. R. 2nd . & McLaughlin , C. S. -LRB- 1981 -RRB- Cell 24 , 367 375 . applied to all of the approximately 6,200 genes of S. cerevisiae , the clusters of functionally related genes are maintained , but are usually expanded with the addition of uncharacterized genes -LRB- the results of this analysis will be the subject of a subsequent report -RRB- . On the basis of our observations here , it is probable that many of these genes will also share common functions . While not based on biological necessity , similarity of pattern of expression may be the easiest available means of making at least provisional attribution of function on a genomic scale . Finally , the functional concordance of coexpressed genes imparts biological significance to the broad patterns seen in images like those of Figs. 1 and 2 . For example , the representation of the transcriptional response of human fibroblasts to serum shown in Fig. 1 is not simply a list of genes and their associated expression patterns , nor is it an arbitrary structure that is being seen , but rather it is a comprehensive representation of the state of the cell throughout its response to serum . Likewise , for yeast experiments , information on the state of many cellular processes can be inferred quickly by combining and comparing new experiments with the data presented here . We thank J. De Risi for excellent technical assistance and many useful suggestions and the staff of the Saccharomyces Genome Database . We also thank J. Cuoczo and C. Kaiser for the use of unpublished results . This work was supported by a grants from the National Institutes of Health -LRB- GM 46406 , HG 00983 , and CA77097 -RRB- . P.O.B. is an associate investigator with the Howard Hughes Medical Institute . P.T.S. was supported by a training grant from the National Eye Institute -LRB- Bethesda , MD -RRB- . M.B.E. was supported by a postdoctoral fellowship from the Alfred E. Sloan Foundation -LRB- New York , NY -RRB- . 1 . Schena , M. , Shalon , D. , Davis , R. W. & Brown , P. O. -LRB- 1995 -RRB- Science 270 , 467 470 .

O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O


Adaptive

Coding

of

Reward

Value

by

Dopamine

Neurons

Philippe

N.

Tobler

et

al.

Science

-LRB-

-RRB-

DOI

science

The

following

resources

related

to

this

article

are

available

online

at

-LRB-

this

information

is

current

as

of

May

-RRB-

Updated

information

and

services

including

high-resolution

figures

can

be

found

in

the

online

version

of

this

article

at

Supporting

Online

Material

can

be

found

at

This

article

cites

articles

of

which

can

be

accessed

for

free

This

article

has

been

cited

by

article

-LRB-

s

-RRB-

on

the

ISI

Web

of

Science

This

article

has

been

cited

by

articles

hosted

by

HighWire

Press

see

Information

about

obtaining

reprints

of

this

article

or

about

obtaining

permission

to

reproduce

this

article

in

whole

or

in

part

can

be

found

at

Science

-LRB-

print

ISSN

online

ISSN

-RRB-

is

published

weekly

except

the

last

week

in

December

by

the

American

Association

for

the

Advancement

of

Science

New

York

Avenue

NW

Washington

DC

Copyright

by

the

American

Association

for

the

Advancement

of

Science

all

rights

reserved

The

title

Science

is

a

registered

trademark

of

AAAS

Downloaded

from

on

May

REPORTS

T.

Nakagawa

K.

Touhara

unpublished

data

R.

A.

Steinbrecht

M.

Laue

G.

Ziegelberger

Cell

Tissue

Res

-LRB-

-RRB-

B.

Pophof

Chem

Senses

-LRB-

-RRB-

B.

S.

Hansson

H.

Ljungberg

E.

Hallberg

C.

Lofstedt

Science

-LRB-

-RRB-

R.

Kanzaki

K.

Soo

Y.

Seki

S.

Wada

Chem

Senses

-LRB-

-RRB-

J.

W.

Wang

A.

M.

Wong

J.

Flores

L.

B.

Vosshall

R.

Axel

Cell

-LRB-

-RRB-

J.

Krieger

et

al.

Proc

Natl.

Acad

Sci

U.S.A.

-LRB-

-RRB-

We

thank

Y.

Kubo

T.

Shimizu

H.

Watanabe

and

M.

Tominaga

for

valuable

advice

H.

Mitsuno

and

C.

Kitamura

for

technical

assistance

and

H.

Kataoka

J.

Takabayashi

and

members

of

the

Touhara

lab

for

helpful

discussion

This

work

was

supported

in

part

by

grants

from

Japan

Society

for

the

Promotion

of

Science

-LRB-

JSPS

-RRB-

and

Program

for

Promotion

of

Basic

Research

Activities

for

Innovative

Biosciences

-LRB-

PROBRAIN

-RRB-

The

sequences

reported

in

this

Report

have

been

deposited

in

the

GenBank

database

-LRB-

accession

codes

and

for

cDNA

sequences

of

and

respectively

-RRB-

Supporting

Online

Material

Materials

and

Methods

Figs

and

References

October

accepted

January

Published

online

February

science

Include

this

information

when

citing

this

paper

Adaptive

Coding

of

Reward

Value

by

Dopamine

Neurons

Philippe

N.

Tobler

Christopher

D.

Fiorillo

Wolfram

Schultz

It

is

important

for

animals

to

estimate

the

value

of

rewards

as

accurately

as

possible

Because

the

number

of

potential

reward

values

is

very

large

it

is

necessary

that

the

brain

's

limited

resources

be

allocated

so

as

to

discriminate

better

among

more

likely

reward

outcomes

at

the

expense

of

less

likely

outcomes

We

found

that

midbrain

dopamine

neurons

rapidly

adapted

to

the

information

provided

by

reward-predicting

stimuli

Responses

shifted

relative

to

the

expected

reward

value

and

the

gain

adjusted

to

the

variance

of

reward

value

In

this

way

dopamine

neurons

maintained

their

reward

sensitivity

over

a

large

range

of

reward

values

In

order

to

select

the

action

associated

with

the

largest

reward

it

is

critical

that

the

neural

representation

of

reward

has

minimal

uncertainty

A

fundamental

difficulty

in

representing

the

value

of

rewards

-LRB-

and

many

other

stimuli

-RRB-

is

that

the

number

of

possible

values

has

no

absolute

limits

By

contrast

the

representational

capacity

of

the

brain

is

limited

as

exemplified

by

its

finite

number

of

neurons

and

the

limited

number

of

possible

spike

outputs

of

each

neuron

If

a

limited

outputs

were

allocated

evenly

to

represent

the

large

potentially

infinite

number

of

possible

reward

values

then

that

activity

would

allow

for

little

if

any

discrimination

between

rewards

However

a

discriminative

capacity

can

be

improved

if

the

neuron

has

access

to

information

indicating

that

some

reward

values

are

more

likely

to

occur

than

others

and

if

it

can

allocate

most

of

its

spike

outputs

to

representing

the

most

probable

values

Conditioned

reward-predicting

stimuli

could

provide

such

information

for

neurons

as

they

do

in

a

more

general

way

for

behavior

-LRB-

-RRB-

Here

we

investigate

how

dopamine

neurons

adapt

to

the

information

about

reward

value

contained

in

predictive

stimuli

These

neurons

play

a

major

role

in

Department

of

Anatomy

University

of

Cambridge

Downing

Street

Cambridge

UK

and

Institute

of

Physiology

University

of

Fribourg

Fribourg

Switzerland

Present

address

Department

of

Biology

Stanford

University

Stanford

CA

USA

To

whom

correspondence

should

be

addressed

E-mail

reward

processing

-LRB-

-RRB-

and

respond

to

rewards

and

reward-predicting

stimuli

-LRB-

-RRB-

We

presented

distinct

visual

stimuli

that

specified

both

the

probability

and

magnitude

of

otherwise

identical

juice

rewards

to

monkeys

well

trained

in

a

Pavlovian

procedure

-LRB-

-RRB-

Standard

procedures

were

employed

to

extracellularly

record

the

activity

of

single

dopamine

neurons

of

midbrain

groups

and

in

two

awake

Macaque

monkeys

-LRB-

-RRB-

We

report

data

for

all

recorded

neurons

that

displayed

electrophysiological

characteristics

typical

of

dopamine

neurons

-LRB-

wide

impulses

at

low

rates

-RRB-

-LRB-

-RRB-

In

an

attempt

to

accurately

portray

the

whole

population

of

dopamine

neurons

we

did

not

select

neurons

on

the

basis

of

their

modulation

by

a

reward

event

The

expected

value

of

future

rewards

-LRB-

the

sum

of

possible

reward

magnitudes

each

weighted

by

its

probability

-RRB-

is

thought

to

be

an

important

variable

determining

choice

behavior

-LRB-

-RRB-

To

test

this

we

trained

an

animal

with

a

set

of

five

distinct

visual

stimuli

presented

in

pseudorandom

alternation

Each

stimulus

indicated

the

probability

that

a

specific

liquid

volume

would

be

delivered

s

after

stimulus

onset

Anticipatory

licking

before

liquid

delivery

was

elicited

by

the

smallest

positive

expected

liquid

volume

tested

-LRB-

ml

at

probability

p

-RRB-

and

increased

with

expected

liquid

volume

suggesting

that

the

animals

had

learned

to

use

the

stimuli

to

predict

liquid

delivery

and

that

the

larger

liquid

volumes

corresponded

to

larger

reward

values

-LRB-

Fig.

-RRB-

The

transient

VOL

activation

of

dopamine

neurons

increased

monotonically

with

the

expected

liquid

volume

associated

with

each

stimulus

-LRB-

Fig.

B

and

C

-RRB-

For

example

the

stimulus

predicting

ml

at

p

elicited

significantly

greater

neural

activation

than

the

stimulus

predicting

the

same

magnitude

reward

at

p

but

less

activation

than

the

stimulus

predicting

ml

at

p

The

activation

of

dopamine

neurons

also

increased

with

the

combination

of

magnitude

and

probability

when

the

stimuli

predicted

that

either

of

two

nonzero

magnitudes

would

occur

with

equal

probability

-LRB-

Fig.

animal

B

-RRB-

To

investigate

whether

individual

neurons

might

be

preferentially

sensitive

to

probability

or

magnitude

we

took

independent

measures

of

sensitivity

to

magnitude

and

probability

in

each

neuron

-LRB-

n

neurons

-RRB-

There

was

a

positive

correlation

-LRB-

P

G

-RRB-

indicating

that

those

neurons

that

were

most

sensitive

to

reward

magnitude

were

also

most

sensitive

to

probability

-LRB-

Fig.

-RRB-

Thus

it

appears

that

dopamine

neurons

encode

a

combination

of

magnitude

and

probability

as

expressed

for

example

by

the

expected

reward

value

rather

than

distinguishing

between

the

two

Having

examined

responses

to

rewardpredicting

stimuli

of

differing

values

we

investigated

the

extent

to

which

dopamine

neurons

discriminated

between

different

volumes

of

unpredicted

liquid

We

delivered

three

distinct

liquid

volumes

-LRB-

and

ml

-RRB-

in

pseudorandom

alternation

with

a

variable

intertrial

interval

-LRB-

-RRB-

and

in

the

absence

of

any

explicit

predictive

stimuli

Both

individual

dopamine

neurons

-LRB-

of

neurons

tested

P

G

Wilcoxon

test

-RRB-

and

the

population

as

a

whole

-LRB-

neurons

-RRB-

showed

greater

activation

for

the

large

than

for

the

small

liquid

volume

-LRB-

Fig.

-RRB-

Thus

the

activation

of

dopamine

neurons

increased

with

the

reward

value

of

unpredicted

liquids

similar

to

the

responses

to

reward-predicting

visual

stimuli

Although

these

results

suggest

that

dopamine

neurons

encode

the

reward

value

in

a

monotonically

increasing

fashion

past

work

indicates

that

they

do

not

represent

absolute

value

Rather

they

appear

to

encode

value

as

a

prediction

error

by

representing

at

each

moment

in

time

the

difference

between

the

MARCH

SCIENCE

Downloaded

from

on

May

REPORTS

Fig.

Behavioral

and

neuronal

responses

to

conditioned

stimuli

increase

with

expected

reward

value

-LRB-

A

-RRB-

Anticipatory

licking

responses

during

the

delay

between

the

conditioned

stimuli

and

liquid

delivery

Each

point

shows

the

mean

-LRB-

T

SEM

-RRB-

of

at

least

trials

-LRB-

animal

A

-RRB-

and

is

significantly

different

from

all

other

points

-LRB-

t

tests

-RRB-

Similar

results

were

obtained

from

animal

B

although

the

mean

licking

durations

varied

over

a

smaller

range

-LRB-

B

-RRB-

Single-neuron

-LRB-

top

-RRB-

and

population

responses

-LRB-

bottom

-RRB-

-LRB-

n

neurons

-RRB-

from

the

experiment

in

-LRB-

A

-RRB-

Visual

conditioned

stimuli

with

their

expected

magnitude

of

reward

are

shown

above

the

rasters

Expected

values

-LRB-

probability

magnitude

-RRB-

were

from

left

to

right

ml

-LRB-

probability

ml

magnitude

-RRB-

ml

-LRB-

ml

-RRB-

ml

-LRB-

ml

-RRB-

ml

-LRB-

ml

-RRB-

and

ml

-LRB-

-RRB-

Bin

width

is

ms

in

histograms

of

all

figures

-LRB-

C

-RRB-

-LRB-

Left

-RRB-

Population

responses

as

a

function

of

expected

liquid

volume

Measurements

were

taken

to

ms

-LRB-

animal

A

-RRB-

and

to

ms

-LRB-

animal

B

-RRB-

after

the

onset

of

visual

stimuli

The

median

-LRB-

confidence

intervals

-RRB-

percent

change

in

firing

rates

within

the

population

was

calculated

after

normalization

of

responses

within

each

neuron

to

the

response

evoked

after

onset

of

the

stimulus

associated

with

the

largest

expected

value

This

stimulus

elicited

a

median

activation

of

in

animal

A

-LRB-

n

neurons

-RRB-

and

in

animal

B

-LRB-

n

neurons

-RRB-

For

animal

A

-LRB-

squares

-RRB-

stimuli

indicated

probability

and

magnitude

as

in

-LRB-

B

-RRB-

For

animal

B

-LRB-

circles

-RRB-

one

stimulus

was

never

followed

by

liquid

whereas

each

of

the

other

three

stimuli

was

associated

with

two

volumes

of

equal

probability

-LRB-

or

ml

or

ml

and

or

ml

-RRB-

In

each

animal

the

population

of

neurons

discriminated

among

each

expected

value

tested

except

for

versus

ml

in

animal

A.

-LRB-

Right

-RRB-

An

alternative

analysis

illustrating

the

sensitivity

-LRB-

spikes/s/ml

-RRB-

of

a

typical

dopamine

response

to

expected

liquid

volume

For

each

individual

neuron

Fig.

Neural

discrimination

of

liquid

volume

-LRB-

A

-RRB-

-LRB-

Top

-RRB-

Rasters

and

histograms

of

activity

from

a

single

dopamine

neuron

-LRB-

Bottom

-RRB-

Population

histograms

of

activity

from

all

neurons

tested

-LRB-

n

neurons

-RRB-

Three

volumes

of

liquid

were

delivered

in

pseudorandom

alternation

in

the

absence

of

any

explicit

predictive

stimuli

The

intertrial

interval

ensured

that

the

expected

volume

at

any

given

moment

was

low

-LRB-

-RRB-

Thick

horizontal

bars

above

the

rasters

indicate

the

time

of

reward

delivery

and

thin

horizontal

bars

indicate

the

single

standard

time

window

that

was

used

for

measuring

the

magnitude

of

all

responses

in

all

neurons

as

summarized

in

-LRB-

B

-RRB-

Similar

windows

were

used

for

all

analyses

and

plots

-LRB-

supporting

text

-RRB-

-LRB-

B

-RRB-

Neural

response

as

a

function

of

liquid

volume

Median

-LRB-

confidence

intervals

-RRB-

percentage

the

number

of

impulses

after

stimulus

onset

was

plotted

as

a

function

of

expected

magnitude

and

a

line

was

fit

The

lines

shown

are

the

median

lines

of

each

population

of

neurons

-LRB-

animal

A

solid

line

spikes/s

magnitude

animal

B

spikes/s

magnitude

-RRB-

-LRB-

D

-RRB-

Positive

correlation

between

the

sensitivity

of

individual

neurons

to

reward

probability

and

magnitude

-LRB-

P

G

-RRB-

For

the

data

from

animal

A

in

-LRB-

C

-RRB-

responses

in

each

neuron

-LRB-

n

neurons

-RRB-

are

plotted

both

as

a

function

of

expected

value

as

determined

both

by

reward

probability

-LRB-

ml

at

p

and

-RRB-

and

by

liquid

volume

-LRB-

and

ml

at

p

-RRB-

A

line

was

fit

in

each

case

and

the

slopes

provided

independent

estimates

of

the

sensitivity

of

that

neuron

to

reward

probability

and

magnitude

For

each

neuron

the

slopes

are

plotted

against

each

other

change

in

activity

for

the

population

of

neurons

-LRB-

n

neurons

-RRB-

was

calculated

for

responses

to

each

volume

after

normalization

in

each

neuron

to

the

response

after

delivery

of

ml

which

itself

elicited

a

median

activation

of

above

baseline

activity

reward

value

-LRB-

the

sum

of

current

and

future

rewards

-RRB-

and

its

expected

value

-LRB-

before

observation

of

current

sensory

input

-RRB-

Recent

work

demonstrates

that

when

signaling

prediction

errors

dopamine

neurons

are

able

to

use

contextual

information

in

addition

to

information

from

explicitly

conditioned

stimuli

-LRB-

-RRB-

In

the

experiments

shown

in

Figs.

and

all

visual

stimuli

and

liquid

volumes

were

delivered

in

a

context

in

which

the

expected

reward

value

at

each

moment

in

time

was

low

and

invariant

across

trial

types

because

of

the

intertrial

interval

-LRB-

-RRB-

In

our

next

set

of

experiments

we

delivered

differ

ent

volumes

of

liquid

in

the

presence

of

explicit

predictions

indicated

by

conditioned

stimuli

allowing

us

to

systematically

vary

the

expected

value

and

range

of

reward

Consistent

with

past

work

a

reward

occurring

exactly

at

the

expected

value

-LRB-

ml

-RRB-

elicited

no

response

However

when

liquid

volume

was

unpredictably

smaller

-LRB-

ml

-RRB-

or

larger

-LRB-

ml

-RRB-

in

a

minority

of

trials

dopamine

neurons

were

suppressed

or

activated

respectively

compared

to

both

the

prestimulus

baseline

and

the

response

to

the

expected

volume

delivered

in

the

majority

of

trials

-LRB-

P

G

Mann-Whitney

test

-RRB-

-LRB-

Fig.

SCIENCE

VOL

A

and

B

-RRB-

In

an

additional

experiment

one

stimulus

predicted

that

either

the

small

or

medium

volume

would

be

delivered

with

equal

probability

whereas

another

stimulus

predicted

either

the

medium

or

large

volume

with

equal

probability

In

both

cases

delivery

of

the

larger

of

the

two

potential

volumes

elicited

an

increase

in

activity

whereas

the

smaller

volume

elicited

a

decrease

-LRB-

Fig.

-RRB-

Thus

the

identical

medium

volume

had

opposite

effects

on

activity

depending

on

the

prediction

-LRB-

P

G

in

of

neurons

MannWhitney

P

G

for

the

population

of

neurons

Wilcoxon

test

-RRB-

-LRB-

Fig.

-RRB-

These

MARCH

Downloaded

from

on

May

REPORTS

results

show

how

dopamine

neurons

process

reward

magnitude

relative

to

a

predicted

magnitude

and

that

a

reward

outcome

that

is

positive

on

an

absolute

scale

can

nonetheless

suppress

the

activity

of

dopamine

neurons

Although

these

results

suggest

that

dopamine

responses

shift

relative

to

the

predicted

reward

magnitude

it

is

not

known

how

their

activity

scales

with

the

difference

between

actual

and

expected

reward

To

this

end

we

analyzed

the

dopamine

responses

at

the

time

of

the

reward

in

the

experiment

shown

in

Fig.

Each

of

three

distinct

visual

stimuli

presented

on

pseudorandomly

alternating

trials

predicted

that

one

of

two

potential

liquid

volumes

would

be

delivered

with

equal

probability

Animals

discriminated

behaviorally

between

the

three

reward-predicting

stimuli

-LRB-

Fig.

-RRB-

Confirming

the

data

described

above

the

larger

of

the

two

volumes

always

elicited

an

increase

in

activity

at

the

time

of

the

reward

and

the

smaller

a

decrease

However

the

magnitude

of

activation

or

suppression

appeared

to

be

identical

in

each

case

despite

the

fact

that

the

absolute

difference

between

actual

and

expected

volume

varied

over

a

range

-LRB-

Fig.

A

and

B

-RRB-

Thus

the

responses

of

dopamine

neurons

did

not

appear

to

scale

according

to

the

absolute

difference

between

actual

and

expected

reward

Rather

the

sensitivity

or

gain

of

the

neural

responses

appeared

to

adapt

according

to

the

discrepancy

in

volume

between

the

two

potential

outcomes

To

document

this

result

further

we

plotted

the

median

neural

responses

as

a

function

of

liquid

volume

and

drew

a

straight

line

to

connect

the

data

points

representing

the

larger

and

smaller

outcomes

after

each

visual

stimulus

-LRB-

Fig.

-RRB-

The

slope

of

these

lines

provided

an

estimate

of

the

neurons

gain

or

sensitivity

with

respect

to

liquid

volume

When

the

discrepancy

was

large

the

sensitivity

of

dopamine

neurons

was

low

and

when

the

discrepancy

was

small

sensitivity

was

high

As

a

result

of

this

adaptation

the

neural

responses

discriminated

between

the

two

likely

outcomes

equally

well

regardless

of

their

absolute

difference

in

magnitude

The

present

data

are

not

sufficient

to

determine

precisely

to

which

aspect

of

the

reward

prediction

the

sensitivity

adapted

but

further

analysis

provided

limited

evidence

that

sensitivity

adapted

to

the

discrepancy

between

potential

liquid

volumes

-LRB-

such

as

the

difference

or

variance

-RRB-

rather

than

to

their

expected

value

-LRB-

-RRB-

-LRB-

fig

-RRB-

Our

results

suggest

that

the

activity

of

dopamine

neurons

carries

information

on

the

magnitude

of

reward

In

representing

reward

magnitude

neural

activity

displayed

two

forms

of

adaptation

that

depended

on

the

prediction

that

was

in

place

at

the

time

of

the

reward

First

the

activity

increased

or

de

Fig.

Bidirectional

dopamine

responses

to

reward

outcomes

reflect

deviations

from

predictions

-LRB-

A

-RRB-

A

single

conditioned

stimulus

was

usually

followed

by

an

intermediate

volume

of

liquid

-LRB-

ml

-RRB-

that

elicited

no

change

in

the

neuron

's

activity

-LRB-

center

-RRB-

However

on

a

small

minority

of

trials

smaller

-LRB-

ml

-RRB-

or

larger

-LRB-

ml

-RRB-

volumes

were

unpredictably

substituted

and

neural

activity

decreased

-LRB-

left

-RRB-

or

increased

-LRB-

right

-RRB-

respectively

Neural

responses

to

the

large

liquid

volume

were

relatively

long-lasting

-LRB-

supporting

online

text

-RRB-

-LRB-

B

-RRB-

Median

responses

-LRB-

confidence

intervals

-RRB-

from

the

population

as

a

function

of

liquid

volume

for

the

experiment

in

-LRB-

A

-RRB-

-LRB-

neurons

from

animal

A

neurons

from

animal

B

-RRB-

Responses

in

each

neuron

were

normalized

to

the

response

after

the

unpredicted

delivery

of

liquid

-LRB-

ml

-RRB-

in

a

separate

block

of

trials

and

in

the

absence

of

any

explicit

reward-predicting

stimulus

-LRB-

C

-RRB-

Responses

of

a

single

neuron

to

three

liquid

volumes

delivered

in

the

context

of

two

different

predictions

One

stimulus

predicted

small

or

medium

volume

with

equal

probability

whereas

another

stimulus

predicted

medium

or

large

volume

The

medium

volume

activated

the

neuron

in

one

context

but

suppressed

activity

in

the

other

-LRB-

D

-RRB-

Population

responses

-LRB-

n

neurons

animal

B

-RRB-

to

medium

reward

in

the

experiment

in

-LRB-

C

-RRB-

The

plot

shows

the

median

the

confidence

intervals

-LRB-

notches

corresponding

to

obtuse

angles

-RRB-

the

and

percentiles

-LRB-

boundaries

corresponding

to

right

angles

-RRB-

and

the

and

percentiles

-LRB-

bars

-RRB-

In

each

neuron

percentage

change

in

activity

was

normalized

to

the

response

to

unpredicted

liquid

-LRB-

ml

which

elicited

a

median

increase

in

activity

of

-RRB-

Fig.

Neural

sensitivity

to

liquid

volume

adapts

in

response

to

predictive

stimuli

-LRB-

A

-RRB-

Activity

of

a

single

neuron

showing

nearly

identical

responses

to

three

liquid

volumes

spanning

a

range

Each

of

three

pseudorandomly

alternating

visual

stimuli

-LRB-

shown

at

left

-RRB-

was

followed

by

one

of

two

liquid

volumes

at

p

-LRB-

top

or

ml

middle

or

ml

bottom

or

ml

-RRB-

Responses

after

onset

of

visual

stimuli

increased

with

their

associated

expected

reward

values

Only

rewarded

trials

are

shown

-LRB-

B

-RRB-

Population

histograms

for

different

liquid

volumes

from

the

experiment

in

-LRB-

A

-RRB-

-LRB-

neurons

animal

A

-RRB-

-LRB-

C

-RRB-

Each

line

connects

responses

occurring

in

the

context

of

a

specific

conditioned

stimulus

and

its

slope

provides

a

measure

of

gain

or

sensitivity

Each

point

represents

the

median

-LRB-

confidence

intervals

-RRB-

response

of

the

population

taken

after

normalizing

the

percentage

change

in

activity

in

each

neuron

to

the

response

after

unpredicted

liquid

-LRB-

ml

-RRB-

delivered

in

a

separate

block

of

trials

-LRB-

which

elicited

an

activation

of

above

baseline

in

animal

A

n

neurons

and

in

animal

B

n

neurons

-RRB-

-LRB-

Left

-RRB-

The

experiment

in

-LRB-

A

-RRB-

and

-LRB-

B

-RRB-

-LRB-

Right

-RRB-

The

same

experiment

but

performed

in

animal

B

with

two

nonzero

liquid

volumes

per

conditioned

stimulus

at

equal

probability

-LRB-

p

-RRB-

-LRB-

stimulus

versus

ml

stimulus

versus

ml

stimulus

versus

ml

-RRB-

MARCH

VOL

SCIENCE

Downloaded

from

on

May

REPORTS

creased

depending

on

whether

the

reward

outcome

was

larger

or

smaller

respectively

than

an

intermediate

reference

point

such

as

expected

value

A

second

unanticipated

form

of

adaptation

was

the

change

in

sensitivity

or

gain

of

neural

activity

that

appeared

to

depend

on

the

range

of

likely

reward

magnitudes

-LRB-

Fig.

-RRB-

Thus

the

larger

of

two

potential

rewards

always

elicited

the

same

increase

in

activity

and

the

smaller

of

the

two

elicited

the

same

decrease

in

activity

regardless

of

absolute

magnitude

The

identical

responses

to

liquid

volumes

spanning

a

range

were

not

due

to

an

insensitivity

of

the

dopamine

neurons

which

were

capable

of

greater

activations

-LRB-

Fig.

note

normalization

of

data

points

-RRB-

and

discriminated

well

among

these

same

liquid

volumes

when

delivered

in

the

absence

of

explicit

predictive

stimuli

-LRB-

Fig.

-RRB-

Rather

the

gain

of

neural

activity

with

respect

to

liquid

volume

appeared

to

adapt

in

proportion

to

the

range

or

standard

deviation

of

the

predicted

reward

outcomes

so

that

neural

discrimination

between

the

two

reward

outcomes

that

were

most

probable

from

the

perspective

was

robust

regardless

of

their

absolute

difference

in

magnitude

The

efficiency

and

accuracy

with

which

neural

activity

can

code

the

value

of

a

stimulus

-LRB-

such

as

liquid

volume

-RRB-

can

be

greatly

increased

if

neurons

make

use

of

information

about

the

probabilities

of

potential

reward

values

Neural

activity

can

then

be

devoted

to

representing

probable

values

at

the

expense

of

improbable

values

Our

evidence

suggests

that

the

transient

dopamine

response

to

conditioned

stimuli

may

carry

information

on

expected

reward

value

and

previous

work

shows

that

the

more

sustained

activity

of

dopamine

neurons

reflects

a

measure

of

reward

uncertainty

such

as

variance

-LRB-

-RRB-

If

the

system

possesses

prior

information

consisting

of

the

expected

value

and

variance

of

reward

then

this

information

need

not

be

represented

redundantly

at

the

time

of

reward

Discarding

this

old

information

may

be

achieved

by

subtracting

the

expected

value

from

the

absolute

reward

value

and

then

dividing

by

the

variance

Analogous

normalization

processes

appear

to

occur

in

early

visual

neurons

-LRB-

-RRB-

It

is

not

known

to

what

extent

the

normalization

processes

observed

in

dopamine

neurons

are

actually

performed

in

dopamine

neurons

as

opposed

to

their

afferent

input

structures

-LRB-

-RRB-

Because

the

new

information

is

by

definition

precisely

the

information

that

the

system

needs

to

learn

the

activity

of

dopamine

neurons

would

be

an

appropriate

teaching

signal

-LRB-

-RRB-

Adaptation

appears

to

be

a

nearly

universal

feature

of

neural

activity

There

is

substantial

evidence

particularly

from

the

early

visual

system

that

adaptation

contributes

to

the

efficient

representation

of

stimuli

-LRB-

-RRB-

We

have

extended

the

principles

of

efficient

representation

to

the

study

of

reward

Reward

is

central

to

processes

underlying

behavior

such

as

reinforcement

learning

and

decisionmaking

and

consideration

of

limitations

and

efficiency

in

the

neural

representation

of

reward

may

yield

insights

into

these

processes

References

and

Notes

I.

P.

Pavlov

Conditional

Reflexes

-LRB-

Oxford

Univ.

Press

Oxford

-RRB-

E.

L.

Thorndike

Animal

Intelligence

Experimental

Studies

-LRB-

Macmillan

New

York

-RRB-

R.

A.

Rescorla

A.

R.

Wagner

in

Classical

Conditioning

II

Current

Research

and

Theory

A.

Black

W.

F.

Prokasy

Eds

-LRB-

Appleton-Century-Crofts

New

York

-RRB-

pp.

H.

C.

Fibiger

A.

G.

Phillips

in

The

Nervous

System

vol

of

Handbook

of

Physiology

F.

E.

Bloom

Ed

-LRB-

Williams

Wilkins

Baltimore

MD

-RRB-

pp.

R.

A.

Wise

D.

C.

Hoffman

Synapse

-LRB-

-RRB-

T.

E.

Robinson

K.

C.

Berridge

Brain

Res

Rev.

-LRB-

-RRB-

T.

W.

Robbins

B.

J.

Everitt

Curr

Opin

Neurobiol

-LRB-

-RRB-

P.

Waelti

A.

Dickinson

W.

Schultz

Nature

-LRB-

-RRB-

T.

Satoh

S.

Nakai

T.

Sato

M.

Kimura

J.

Neurosci

-LRB-

-RRB-

C.

D.

Fiorillo

P.

N.

Tobler

W.

Schultz

Science

-LRB-

-RRB-

R.

Kawagoe

Y.

Takikawa

O.

Hikosaka

J.

Neurophysiol

-LRB-

-RRB-

Materials

and

methods

are

available

as

supporting

material

on

Science

Online

No

significant

correlations

were

found

between

neuronal

position

in

areas

and

and

the

different

types

of

responses

in

all

the

experiments

reported

thus

the

data

were

pooled

A.

Arnauld

P.

Nicole

La

logique

ou

L'art

de

penser

-LRB-

Librairie

philosophique

J.

Vrin

Paris

-RRB-

P.

W.

Glimcher

Decisions

Uncertainty

and

the

Brain

-LRB-

MIT

Press

Cambridge

MA

-RRB-

P.

Shizgal

Curr

Opin

Neurobiol

-LRB-

-RRB-

M.

I.

Leon

C.

R.

Gallistel

J.

Exp

Psychol

Anim

Behav

Process

-LRB-

-RRB-

All

trials

were

separated

by

an

average

intertrial

interval

of

s

consisting

of

a

fixed

s

plus

an

interval

drawn

from

a

truncated

Poisson-like

distribution

with

a

mean

of

s.

Thus

the

probability

that

a

trial

would

begin

at

any

given

moment

was

low

H.

Nakahara

H.

Itoh

R.

Kawagoe

Y.

Takikawa

O.

Hikosaka

Neuron

-LRB-

-RRB-

M.

V.

Srinivasan

S.

B.

Laughlin

A.

Dubs

Proc

R.

Soc

London

Ser

B.

-LRB-

-RRB-

N.

Brenner

W.

Bialek

R.

R.

de

Ruyter

van

Steveninck

Neuron

-LRB-

-RRB-

A.

L.

Fairhall

G.

D.

Lewen

W.

Bialek

R.

R.

de

Ruyter

van

Steveninck

Nature

-LRB-

-RRB-

W.

Schultz

J.

Neurophysiol

-LRB-

-RRB-

W.

Schultz

P.

Dayan

R.

R.

Montague

Science

-LRB-

-RRB-

H.

B.

Barlow

in

Sensory

Communication

W.

A.

Rosenblith

Ed

-LRB-

MIT

Press

Cambridge

MA

-RRB-

pp.

S.

B.

Laughlin

Z.

Naturforsch

Teil

C

Biochem

Biophys

Biol

Virol

-LRB-

-RRB-

I.

Ohzawa

G.

Sclar

R.

D.

Freeman

Nature

-LRB-

-RRB-

S.

M.

Smirnakis

M.

J.

Berry

D.

K.

Warland

W.

Bialek

M.

Meister

Nature

-LRB-

-RRB-

We

thank

K.

I.

Tsutsui

I.

Hernadi

A.

Dickinson

and

S.

B.

Laughlin

for

helpful

comments

Supported

by

the

Wellcome

Trust

Swiss

National

Science

Foundation

-LRB-

W.S.

and

P.N.T.

-RRB-

Roche

Research

Foundation

-LRB-

P.N.T.

-RRB-

Janggen-Poehn

Foundation

-LRB-

P.N.T.

-RRB-

and

Human

Frontiers

Science

Program

-LRB-

C.D.F.

-RRB-

Supporting

Online

Material

Materials

and

Methods

SOM

Text

Figs

and

References

and

Notes

September

accepted

January

science

SCIENCE

VOL

MARCH

Downloaded

from

on

May

O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O B O O O O O O B O O O B O O O O O O B O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O B O O O O O O O O O O O O O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O B O O O O O O O O O B O O O O O O O O O O O O O O B O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O B O O O O O O O O B O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O B O O O B O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O B O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O B O B O O O O O O O O O B I O O O O O O O O O O O O O O B O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O O

neuroeconomics
decision making
learning
monkey