Embedding Responses in Spontaneous Neural Activity Shaped through Sequential Learning
The way in which neural processing of sensory inputs leads to cognitive functions is one of the most important issues in neuroscience ., Neural activity in the presence of sensory stimuli 1–4 and during the execution of cognitive tasks in response to sensory inputs have been measured experimentally 5 , 6 , and neural network models that exhibit the requested responses to the inputs have been investigated theoretically 7–12 ., Learning algorithms have also been proposed to memorize several input/output ( I/O ) mappings 12–16 ., The response activity has been the main focus both in modeling studies and experiments , while pre-stimulus , i . e . , spontaneous , activity has been dismissed simply as background noise ., However , spontaneous activity has recently been garnering more attention since experimental measurements have revealed that the spontaneous activity is not random noise and that it shows characteristic spatiotemporal patterns 17–19 ., Furthermore , many observations have revealed that the response activities to external stimuli 20 , 21 or cognitive tasks depend on the spontaneous activity 22 , 23 ., Evoked responses are generated not only by external inputs but also through the interplay of the spontaneous activity and external stimuli ., Thus , to establish a neural basis for the cognition and computation in a neural system , it is important to understand the nature of this interplay ., Spontaneous activity has been analyzed theoretically over the last few decades by using neural network models of rate-coding or spiking neurons with random , designed , or biologically realistic connections 24–28 ., However , apart from a few publications 29 , 30 , the relationship between the spontaneous activity and response to external input has rarely been investigated ., Furthermore , how the learning shapes the spontaneous activity and its response to an input is still an open question , but recent experimental studies suggest that learning and developmental processes modify and shape the spontaneous activity 31 , 32 ., In the present paper , we analyze how the spontaneous activity is formed when I/O mappings are memorized ., We do this by introducing a simple learning rule to the neural dynamics in order to study the interplay between the spontaneous activity and input-evoked response ., To analyze the formation of the spontaneous activity and its response to the memorized input through the learning of I/O mappings , we previously proposed a novel view on memory in 33 , 34 , which we called “memories as bifurcations” in contrast to the traditional theoretical viewpoint of “memories as attractors . ”, According to the memories-as-attractors viewpoint , each memory is embedded in one of the attractors in a unique neural dynamical system 11 ., An input specifies an initial condition of the dynamical system , and from that initial state , the neural activity reaches an attractor that matches the target corresponding to the given input ., Thus , the initial states are determined by the given inputs , but the neural activity in the absence of inputs is not examined ., In contrast , according to the memories-as-bifurcations viewpoint , an input modifies the neural dynamics as a parameter , and the flow structure of the neural activity is also changed from that without an input ., In the absence of input , the neural activity evolves and corresponds to spontaneous activity ., In the presence of a learned input , the flow structure in the neural dynamics changes and an attractor that matches the requested target corresponding to the applied input emerges ., With an increase in the input strength , the flow structure changes via a sequence of bifurcations in terms of dynamical systems theory ., Here , the flow structure can be changed substantially by applying different memorized inputs ., Thus , for this viewpoint , memories are embedded in the flow structure of the neural dynamics such that they enable appropriate bifurcations to appear upon input application ., Previously , we designed a neural-network connection matrix through correlations among memorized inputs and targets so that an output that matches a target is generated , as a result of bifurcations from the spontaneous activity , by applying the corresponding input 34 ., In the model , similarity between the spontaneous and evoked activities was demonstrated and is consistent with recent observations in experimental studies 32 , 35–37 ., Although the simplicity of the model is an advantage for analyzing the relationship between spontaneous and evoked neural activities , it remains unclear whether the simplistic structure in the designed network in 34 is the only way to store associative memories or if there exists a variety of networks that show similar behavior and generate a sufficient memory capacity ., Also , how such network structures for memorizing I/O mappings are formed by learning through a widely-accepted synaptic plasticity rule , such as the Hebbian rule , is still open for debate ., In the present study , we introduce a sequential learning model with a simple Hebbian-type learning rule that changes the synaptic strength according to the activities of the pre- and postsynaptic neurons ., From extensive numerical simulations , we have confirmed that through this learning the networks memorize mappings ( where is the number of elements ) satisfying the memories-as-bifurcations viewpoint ., Here , spontaneous activity shows chaotic behavior with approaches to memorized output patterns ., By applying each memorized input , this activity is transformed ( after a sequence of bifurcations ) into different attractors that generate the target pattern corresponding to the applied input ., In spite of the sequential learning scheme , the neural network does not lose the memory it learned earlier; it has a capacity of up to ., This capacity is not so small , and interestingly it is not possible in conventional sequential learning models in which the learning of a new I/O mapping easily pushes out previous memories ., As long as the memorized targets are attractors in the same dynamical system , the formation of a new attraction to a novel attractor will easily destroy the attraction to earlier target patterns ., Our model differs in that the different targets are attractors in the presence of the corresponding input , i . e . , they are embedded in different neural dynamical systems , so that attractors for earlier targets are not destroyed ., Here , the spontaneous activity is flexible; it is possible to apply an input so that a new target is embedded in the network structure without destroying the information of the previous targets ., Remarkably , the network generated through the learning process to obtain a high memory capacity is found to have a similar structure to the network designed in 18 ., Although the learning process can generate a huge variety of networks , which are not similar to the designed network , a common structure is generated by the learning ., A simple learning rule for synaptic change is sufficient for generating such a network ., We first select two random binary patterns , and , as the input and target patterns , respectively ., The neural activity evolves in the presence of whose strength is constant during the learning process for ., The synaptic connection also evolves according to ( 2 ) where is a learning parameter that is the inverse of the time scale ratio of the synaptic to neural dynamics ., The above synaptic dynamics are determined by correlations between the activities of the pre- and postsynaptic neurons ., This learning rule takes a similar form as the perceptron learning rule where the synaptic connection is changed by correlations between activities of elements in the input and output layers 16 ., Here , although the validity of this learning rule is not mathematically proven in contrast to the perceptron , it is expected by the following argument ., According to Eq ., 1 , the change in the neural activity during with the connection modified by the learning , , is given by ( 3 ) ( 4 ) Following the synaptic dynamics in Eq ., 2 , the change in the neural activity due to is given by ( 5 ) where is a positive value determined by and differential coefficient ., Thus , when is larger ( smaller ) than , increases ( decreases ) , respectively ., Hence , the change in the synapses will drive the successive activity toward the target ., Note , however , that the distance between the neural activity and the target is not necessarily guaranteed to decrease monotonically through the learning , because the total change in the neural activity depends also on ., The learning process stops automatically when the neural activity matches the target since in this case , otherwise , the learning process continues ., Here we impose several I/O mappings to be successively learned , and after learning the preceding mapping , another input pattern with the same strength as the previous learning is applied while giving a new target pattern ., The learning process for each single I/O mapping is called a learning step in what follows ., In this learning algorithm , which belongs to a class of palimpsest learning models 38–40 , each mapping is learned sequentially and previously learned mappings are overwritten by the latest mapping ., Thus , it is possible that older mappings are forgotten through the learning process ., During the learning process , double ( neural and synaptic ) dynamics run concurrently , and the neural and synaptic states have to be set as initial states: the neural and synaptic states are randomly selected from with a uniform probability and from a binary ensemble of with equal probability , respectively ., In this model , fully-connected networks without self-connections are used ., Through different learning processes , different sets of mappings are learned so that the generated networks are also different ., For a statistical analysis , we take an average over many networks shaped through different learning processes ., As our purpose in this study is to analyze the relationship between the spontaneous and evoked dynamics , we analyze the neural dynamical system in the absence and presence of input after learning ., After the learning is completed , the synaptic connections are fixed and only the neural activities evolve ., Note that there is no need for the input strengths for learning and memory recall to be identical: we can set the input strength used during the recall process after the learning and independently of the input strength used during the learning process ., For example , after learning with , we can analyze the evoked dynamics by applying the input with ., To distinguish the two clearly , the input strength used in the learning process is denoted by and that used in the analysis of the neural activities after learning is denoted by ., The spontaneous and evoked dynamics are given by and , respectively ., As recall and memory for the memories-as-bifurcations viewpoint are defined differently from those for the memories-as-attractors viewpoint , we outline the definitions of recall and then memory here ., A network succeeds in recalling a target for an input of , if , on application of input for = , the overlap of the evoked activity with the target is higher than the overlap with any other pattern ., Here , is a transposed vector of and the inner product is given by ., By considering a case in which the evoked attractor is not a fixed-point attractor , the temporal average overlap is taken as this criterion ., By denoting the temporal average overlap with the target as , the criterion for the successful recall of corresponding to the applied input is given by ( 6 ) where we measure the avaraged overlaps in the presence of the input and is the pattern that has the largest overlap with the activity among other targets and inputs , as well as other random patterns ., Memory is defined as the ability of a network to recall a target for most initial states ., The condition for whether a network memorizes an I/O ( / ) mapping is ( 7 ) where represents the average over the initial states of this network ., By extending this criterion , we adopt a condition for determining whether networks memorize the I/O mapping for a certain parameter as ( 8 ) where denotes the average over different networks ., Fig . 1 exhibits a learning process shown as a raster plot and the time series of the overlap with the target for ., After wandering over many neural activity patterns , the neural activity reaches the target pattern and the learning process is completed ., The learning process does not stop by becoming trapped in a local minimum , nor does it continue to wander over the neural patterns ., We confirmed that in all trials with parameters of , the learning was completed ., During a learning process , the flow structures of the spontaneous and evoked activities change ., Hence , the recall process also changes through the learning process ., Fig . 2 shows a recall process before and after learning for and ., Before learning , an attractor matching the applied input pattern is generated when that input is applied ( in Fig . 2A ) , but the overlap with the required target is not high and the network thus fails to recall the target ., After learning , two types of neural dynamics are generated depending on the parameter values ( ) ( see also Table 1 ) : We now analyze the spontaneous and evoked neural dynamics in these two regimes ., First , to reveal the dependence of the evoked dynamics on the parameters , as a function of and , is shown in Fig . 3A ., In the R regime , for larger and smaller values , only the target attractor exists and the average overlap is equal to one , while in the NR regime , both the target and reverse-target attractors exist and the average overlap is lower than that in the R regime ., As decreases or increases , the volume of the reverse-target attractor basin increases and that of the target attractor decreases so that the average overlap with the target also decreases ., The dotted line in Fig . 3A represents the boundary between the R and NR regimes computed using the spontaneous activity , as discussed below ., To analyze the spontaneous dynamics , we note that due to the symmetry , the mean overlap for each target over time is generally zero because the orbit can approach both the target and reverse-target with equal probability ., Thus , we measure the standard deviation ( SD ) of the overlap to quantify the approach to each target ., The SD ( ) of an overlap with the -th target over time is computed as ., If this SD is much larger than that for the overlap with a random pattern , then the spontaneous activity selectively approaches the target ( and its reverse ) ., A numerical computation of the SD as a function of and is plotted in Fig . 3B ., In the R regime , chaotic behavior appears and the SD takes a finite positive value , while in the NR regime , fixed-point attractors exist and so the SD is zero ., Interestingly , a band that has a higher SD , which stretches from ( 2 . 6 , 0 . 001 ) to ( 16 , 1 ) , and whose ridge divides the R and NR regimes appears in the figure ., In Fig . 3B , the ridge is shown as the dotted line , which is also plotted as a reference in Fig . 3A ., Around the ridge , the SD of the spontaneous activity is much higher than that in other areas , and the chaotic spontaneous activity shows switching behavior between the target and reverse target ., While the target and reverse-target attractors are unstable , their ruins still exist and the neural dynamics intermittently visit them ., In Figs ., 3A and B , the boundary defined by the SD might be slightly ambiguous because of the finite-size effect ., However , by extrapolating the result for larger system sizes ( to be discussed later ) , it is expected that , in the absence of inputs , all the networks in the NR regime show fixed-point behavior and those in the R regime show chaotic behavior , in the thermodynamic limit ., By increasing or decreasing , the minimum distance between the activity and the target ( or the reverse-target ) increases in the R regime ., Thus , in this limit , the SD in the NR regime is zero ., It suddenly increases to nearly one at the transition point , and then gradually decreases in the R regime ., The ridge of the SD thus indicates the transition between the NR and R regimes well ., The area with the average overlap taking nearly one above the dotted line in Fig . 3A is expected to remain even in the thermodynamic limit ., However , this area is included in the NR regime , since according to the analysis of the neural dynamics after multiple learning steps , to be discussed later , no more than a single pattern is recalled , as in the rest of the NR regime ., We also show how spontaneous activity changes into evoked activity with an increase in in each regime , as shown in Fig . 3C ., In the R regime , by increasing from zero , the neural activity shows successive bifurcations such that the overlap with the target increases to approach unity at ., The fixed-point attractor matching the target appears at ., In the NR regime , the target and reverse target attractors do not change on application of the input , but the basin volumes of the attractors increase ., We analyze the connection matrix that is shaped through the learning process , in the R and NR regimes , by measuring the element of the matrix C which is projected onto and , as defined by ( 9 ) where = ., Note that for a given binary pattern , if the system has a large matrix element , then pattern is more stable in the absence of inputs for the neural dynamics in Eq ., ( 1 ) ., Similarly , when is larger , is less stable ., Fig . 4 shows a time series of the elements , and for the NR , and R , regimes ., In the NR regime , only the element is much larger than the others after learning , while in the R regime , both and take salient positive values and and take salient negative values ., The result that dominates in the NR regime means that the generated connection matrix takes a similar form to that of the Mattis model in a spin system 41 , which corresponds to the Hopfield network with only one memorized pattern ., In the network where is larger and the other elements are much smaller , the target and reverse-target patterns remain highly stable ., This is consistent with the above analysis in the NR regime ., In the R regime , in contrast , the connection matrix shows a form distinct from those of the matrices in Mattis and Hopfield-type networks ., Remarkably , the matrix takes a similar form to that of the model in 34 , where was adopted ., Indeed , the behaviors of the spontaneous and evoked activities in this regime agree with that observed in that model 34 ., In general , the behaviors are strongly dependent on the matrix elements ., In Fig . 5 , the elements as a function of are plotted ., For , all of the elements deviate saliently from zero , and as decreases , the elements , and decrease rapidly , while does not change ., The regime changes from the R to NR regime as this occurs ., We now analyze why such connection matrices are formed through the learning process ., The evolution of the matrix element is also determined by Eq ., ( 2 ) as follows: ( 10 ) ( 11 ) Although also evolves temporally , we set as a constant value , because relative scale of the elements is relevant for understanding the behavior ., In the same way , the evolutions of the other elements are determined by ( 12 ) ( 13 ) ( 14 ) In both the regimes , the activity approaches a target and thus is greater than zero ( and smaller than ) for most of the learning process ., Thus , is positive for most of the learning process and then , takes a large positive value ., In contrast , the change in the other elements is distinct between both regimes , which is explained by the initial behavior of the learning process ., In the R regime , the overlap with the input increases in the early stage of the learning process as is directed toward by the input , as shown in Fig . 4A, ( ii ) ., It is estimated that is and positive , which is much larger than ., Thus , and are negative in the R regime , while is positive ., These estimates of the sign of the elements are consistent with the matrix elements in Fig . 4B ., For the NR regime , in which is smaller and is larger , the increase in the overlap with the input in the early stage is much smaller than that in the R regime; if is small , the neural activity does not respond strongly to the input , whereas if is large , the learning is completed before the overlap with the input increases ., Thus , the temporal changes in , and are much smaller ., Hence , only takes a large value , and thus the Mattis-type network is generated ., Neural activities that are shaped through multiple learning steps are analyzed for I/O mappings that are learned sequentially , as shown in Fig . 6 ., In the presence of each input ( as indicated by the colored bars above the plot ) , the neural activity converges to the target to be memorized in the same way as in the learning process of a single mapping ( shown in Fig . 1 ) ., Note that although the learning process changes the synaptic connections and flow structure of the neural activity , some of the structure generated in earlier learning steps is preserved because the change in the flow structure in each learning step occurs in the presence of a different input pattern ., We mainly present the results after the learning of 40 mappings and analyze the behaviors of spontaneous and evoked activities for later 30 mappings in the following analysis ., ( We choose 30 mapping because memory capacity is almost 20 as shown later . The number 30 and 40 can be arbitrary , as long as they are chosen to be larger than the many capacity . ), Corresponding to each phase in the one-step learning , we also found two distinct behaviors in the multiple learning:, ( i ) Neural activity responds to multiple inputs so that an attractor that matches each learned pattern is generated respectively upon each input ., Thus , multiple mappings are successfully memorized ., ( ii ) The neural activity does not respond to any input ., The two attractors that match the latest learned target and its reverse pattern exist in the absence and presence of the input ., Recall in response to an input is not observed either ., We call these the R and NR regimes , respectively , in the same manner as the analysis for one-learning step ., In Fig . 7 , we plot the neural dynamics in the presence and absence of inputs after 40 learning steps for in the R regime ., The recall processes of 1st , 5th , and 30th targets are shown by the overlaps with for and 30 in the absence and presence of the 1st , 5th , and 30th input , respectively ., From here on , the index ( 5 , and in this case ) denotes the order of the I/O mapping beginning with the most recent , i . e . , the 1st mapping is the latest learned one , while the 5th is that learned 5 steps earlier , and so forth ., In the R regime , by applying an input , the overlap with the required target increases and takes on the highest value of all overlaps ., In particular , in the presence of the latest input , the overlap with the latest target takes a much higher value , of nearly one , and an attractor that matches the latest target is generated ., Thus , the latest target is successfully recalled by applying the corresponding input ., In the presence of earlier inputs , the overlaps with the requested targets take smaller values than that with the latest target , but they are still larger than the overlaps with other patterns ( see Fig . S1 ) , as long as the retrieved mapping is not one that was learned much earlier ( as shown below ) ., ( The overlaps with the applied inputs also take higher values than the overlaps with other patterns , as well as the overlaps with the required targets . Thus , we compare the overlaps with the targets with those with the inputs in the following part . ), For example , the overlap with the 5th target is highest among the overlaps with others , in particular higher than that with the 5th input ( Fig . 7B ) ., Thus , the 5th target is also recalled according to Eq ., 4 ., From almost all initial values , the neural activity evolves to an attractor that gives the corresponding target pattern upon application of the appropriate input ., Thus , the 1st and 5th targets are always recalled ., According to the definition of memory in Eq ., 6 , the 1st and 5th mappings are memorized in this network ., In contrast , the overlap with the 30th target , which is learned much earlier , takes a much smaller value and is lower than the overlap with the 30th input ., Thus the network cannot recall the 30th target , i . e . , the target has not been memorized ., Hence the memory capacity of the present network lies between 5 and 30 ., To examine the memory capacity , we compute the average overlaps with the targets in the presence of each earlier input , as well as the average overlap with the input itself , as shown in Fig . 7B ., The overlap with an earlier target upon application of the corresponding input gradually decreases with an increase in , while the overlap with the applied input increases ., The difference between the average overlaps with the -th target and input under the -th input = decreases with an increase in ., Here , eventually crosses 0 at around 20 ., According to definition of memory in Eq ., 8 , the system in this regime succeeds in recalling the target by applying the corresponding input to 20 I/O mappings ., To reduce the artifact from the fluctuations of the overlap on memory capacity due to the finite size effect , we modify the definition of the memory capacity slightly as ( 15 ) Here , we set , however , as long as the value is small , there is no essential change in the memory capacity ., According to this modified definition , is computed to be 19 ., We also analyze the spontaneous neural dynamics that underlie the responses to the learned inputs analyzed above in the R regime ., The spontaneous neural activity shows noisy behavior , and no fixed pattern is stable , as shown in Fig . 7A ., Irrespectively of the noisy behavior , the overlaps with the memorized targets often show high values from time to time ., We compute the distributions over time of these overlaps and present them in Fig . 7C ., The overlap distribution with the latest target is much broader than that with a random pattern , and thus , the neural activity gets selectively closer to the latest target from time to time , even in the absence of input ., The distributions of the overlaps with earlier targets are also broader than that with a random pattern , even though the magnitude is smaller than that of the overlap with the latest target ., Following the analysis introduced in the single-step learning , we measure the SDs of the distributions of the overlaps with all the targets , as represented by dots in Fig . 7D ., We also compute the SD by averaging over the networks , as shown in Fig . 7D as the light blue line ., As shown , the SDs of the later targets decrease as increases ., The major source of decrease in the SD comes from a decrease in the amplitude of the overlap ., Therefore , the spontaneous activity approaches the learned targets from time to time and the closeness to the target during the spontaneous dynamics decreases with ., The SD decreases approximately as a power law as , with ., This decay rate roughly agrees with that of the evoked activity , which is approximated by with ., Both of the exponents are computed from a fit of the overlap and averaged SD to and , respectively , by using the least-squares method ., We will analyze the dependence of the decay rates on the parameters and below ., In the NR regime , in contrast , the latest target and its reverse pattern exist as attractors in the absence and presence of inputs for ( see Fig . S2 ) ., This is identical to the NR-regime behavior after one learning step , for which was nearly zero ., Due to the stability of the latest target attractor , the neural activity does not respond to the earlier input ( ) either , so that is also nearly zero ., According to the definition of memory , Eq ., 15 , ., By decreasing or increasing , the reverse target attractor is less stable in the presence of the latest input , and loses stability at some parameter values , while this attractor is still stable in the absence of the input ., In this region , is equal to one , while there is still no response to an earlier input , and thus in this region , =\u200a1 ., So far , we have analyzed the spontaneous neural activity with 0 and the evoked activity with ., We now examine how the spontaneous activity is transformed into the evoked activity with , as is increased ., This change with changing is regarded as a bifurcation or a sequence of bifurcations in terms of the dynamical system theory ., The bifurcations of the neural activity , revealed by increasing for the 1st , 5th , and 30th input strengths for the network given in Fig . 7 , are shown in Fig . 8 ., In the R regime , the overlap with the 1st ( i . e . , latest ) target increases monotonically and continuously by increasing the strength of the 1st input ., Finally , the fixed point that matches the 1st target is generated for not only the network used in the figure , but also most of the networks in the R regime ., The change to a fixed point is understood as a low-dimensional bifurcation , while the whole sequence of neural activity changes involves higher-dimensional dynamics ., For the 5th and 30th inputs , the overlap with the corresponding input is increased continuously with an increase in the input strength , in a similar manner as the bifurcation diagram for the 1st input ., In contrast to the latest input , however , the attractor is not a fixed-point attractor even for , where the evoked activity still shows chaotic behavior ., Apart from the change to a fixed-point attractor , the bifurcation sequences involve a large degree of freedom in a high-dimensional ( ) space ., Hence , plotting a few macroscopic variables , i . e . , the overlaps of the neural activity with a few targets , is not sufficient to capture the entire bifurcation sequence ., Therefore , to consider the chaotic dynamics , we measured the Lyapunov spectrum for the neural activity dynamics ., With an increase in the input strength , the number of positive Lyapunov exponents decreases , implying the existence of successive bifurcations from a high-dimensional attractor to a lower-dimensional attractor ( see Fig . 8 ) ., Accordingly , the dimension of the neural-activity attractor also decreases ., No positive Lyapunov exponents exist once the fixed-point attractor is reached for the input that was just learned , while even for the application of an earlier input , a decrease in the number of positive exponents is observed but the number does not reach zero ., In the NR regime , the latest target and reverse-target fixed-point attractors exist with ., Even by increasing the input strength , these attractors remain stable and no bifurcation occurs ., The dependence of the spontaneous and evoked activities on the two parameters , and , are analyzed through the capacity and SD ., The dependence of the evoked activity is explored by measuring the capacity according to Eq ., 15 , with the results shown in Fig . 9A ., In the R regime with a larger and smaller , a high capacity is observed , while in the NR regime with a smaller and larger , the capacity is zero or one ., Over the entire parameter space , the overlap with the requested target in the presence of an earlier input decreases , i . e . , decreases as increases , while that with the corresponding input increases ., However , the decay rate of the overlap with the target as a function of and the growth rate of the overlap with the input are dependent on and ., For a large and small , e . g . , as shown in Fig . 7B 
Recent experimental measurements have demonstrated that spontaneous neural activity in the absence of explicit external stimuli has remarkable spatiotemporal structure ., This spontaneous activity has also been shown to play a key role in the response to external stimuli ., To better understand this role , we proposed a viewpoint , “memories-as-bifurcations , ” that differs from the traditional “memories-as-attractors” viewpoint ., Memory recall from the memories-as-bifurcations viewpoint occurs when the spontaneous neural activity is changed to an appropriate output activity upon application of an input , known as a bifurcation in dynamical systems theory , wherein the input modifies the flow structure of the neural dynamics ., Learning , then , is a process that helps create neural dynamical systems such that a target output pattern is generated as an attractor upon a given input ., Based on this novel viewpoint , we introduce in this paper an associative memory model with a sequential learning process ., Using a simple Hebbian-type learning , the model is able to memorize a large number of input/output mappings ., The neural dynamics shaped through the learning exhibit different bifurcations to make the requested targets stable upon an increase in the input , and the neural activity in the absence of input shows chaotic dynamics with occasional approaches to the memorized target patterns ., These results suggest that these dynamics facilitate the bifurcations to each target attractor upon application of the corresponding input , which thus increases the capacity for learning ., This theoretical finding about the behavior of the spontaneous neural activity is consistent with recent experimental observations in which the neural activity without stimuli wanders among patterns evoked by previously applied signals ., In addition , the neural networks shaped by learning properly reflect the correlations of input and target-output patterns in a similar manner to those designed in our previous study .
The neural activity without explicit stimuli shows highly structured patterns in space and time , known as spontaneous activity ., This spontaneous activity plays a key role in the behavior of the response to external stimuli generated by the interplay between the spontaneous activity and external input ., Studying this interplay and how it is shaped by learning is an essential step toward understanding the principles of neural processing ., To address this , we proposed a novel viewpoint , memories-as-bifurcations , in which the appropriate changes in the activity upon the input are embedded through learning ., Based on this viewpoint , we introduce here an associative memory model with sequential learning by a simple Hebbian-type rule ., In spite of its simplicity , the model memorizes the input/output mappings successively , as long as the input is sufficiently large and the synaptic change is slow ., The spontaneous neural activity shaped after learning is shown to itinerate over the memorized targets in remarkable agreement with the experimental reports ., These dynamics may prepare and facilitate to generate the learned response to the input ., Our results suggest that this is the possible functional role of the spontaneous neural activity , while the uncovered network structure inspires a design principle for the memories-as-bifurcations .
physics, mathematics, biology, computational biology, nonlinear dynamics, biophysics, neuroscience