Controlling the Quality of Distillation in Response-Based Network Compression

The performance of a distillation-based compressed network is governed by the quality of distillation. The reason for the suboptimal distillation of a large network (teacher) to a smaller network (student) is largely attributed to the gap in the learning capacities of given teacher-student pair. While it is hard to distill all the knowledge of a teacher, the quality of distillation can be controlled to a large extent to achieve better performance. Our experiments show that the quality of distillation is largely governed by the quality of teacher's response, which in turn is heavily affected by the presence of similarity information in its response. A well-trained large capacity teacher loses similarity information between classes in the process of learning fine-grained discriminative properties for classification. The absence of similarity information causes the distillation process to be reduced from one example-many class learning to one example-one class learning, thereby throttling the flow of diverse knowledge from the teacher. With the implicit assumption that only the instilled knowledge can be distilled, instead of focusing only on the knowledge distilling process, we scrutinize the knowledge inculcation process. We argue that for a given teacher-student pair, the quality of distillation can be improved by finding the sweet spot between batch size and number of epochs while training the teacher. We discuss the steps to find this sweet spot for better distillation. We also propose the distillation hypothesis to differentiate the behavior of the distillation process between knowledge distillation and regularization effect. We conduct all our experiments on three different datasets.

Introduction
Wider and deeper deep learning (DL) models have helped us build more and more sophisticated systems across a broad range of areas LeCun et al. [2015]. But the huge computation and memory requirements of these models create hurdles in its deployment on edge devices Simonyan and Zisserman [2015], Howard et al. [2017]. A number of network compression techniques have been developed to compress large DL models into computationally efficient models with comparable performance, like model quantization Wu et al. [2016], model binarization Courbariaux et al. [2016], parameter sharing Han et al. [2015], Wang and Yoon [2021], low-rank factorization Denton et al. [2014], Yu et al. [2017], convolution filter compression Zhai et al. [2016] and knowledge distillation (KD) Buciluǎ et al. [2006], . KD has two basic steps, knowledge extraction and distillation. Knowledge extraction can be response-based, feature-based, or relation-based and its distillation can be offline, online, or self-distillation Gou et al. [2021]. In this paper, we explore the KD process that uses a teacher's response to distill the knowledge in an offline manner to achieve network compression. Figure 1: (a) sweet spot between batch size and number of epochs for a given teacher-student pair for effective KD (b) and (c) difference in teacher's response in presence and absence of similarity information, respectively.

Figure 2: The distillation hypothesis
Our experiments suggest that the quality of distillation is largely controlled by the amount of "similarity information" about classes that a teacher provides in its responses. In other words, teachers provide a distribution over the possible class labels for any given example. A highly-confident teacher produces a "hard" result that has a peak response for exactly one class, whereas a less-confident teacher provides a more spread, higher-entropy distribution. We find, somewhat counter-intuitively, that the less-confident teacher provides the student more information about the relationships between classes through the non-zero responses to the other classes. We find that careful training of the teacher can largely offset the negative effect of the gap between models. Fig. 1 (a) presents the idea. For a given teacher-student pair of any capacity gap, we can find a balance between batch size and number of epochs of training a teacher to retain similarity information in its response. Fig. 1 (b) and (c) show the fundamental difference in the quality of a teacher's response in the presence and absence of similarity information, respectively. A good response has relevant information about similarity between two classes and a poor response lacks this information. The RBKD process is fast and effective with the similarity-rich response as it learns about more than one class from a single input -one example-many class learning. It requires fewer examples per class to learn about the whole distribution of examples. In contrast, the process is slow, less effective, and requires more examples per class when similarity information is absent.
We also observe that for a fixed student network, as the teacher model becomes more powerful, wider and deeper in size, it loses the rich similarity information present in the soft labels and behaves like a label smoothing (LS) Müller et al. [2019], Szegedy et al. [2016 process. We use this observation to differentiate the nature of the distillation process between KD and regularization effect through the distillation hypothesis shown in Fig. 2. We argue that any distillation process with little or no similarity information in the teacher's response brings more of a regularization effect than the distillation effect and when it completely lacks similarity information, the KD process is equivalent to the LS.
Our contributions are as follows: (i) We show that rich similarity information in teacher's response can facilitate one example-many classes learning and accelerates KD. (ii) We argue that the gap in teacher-student pair is not the root cause or the dominating factor of poor distillation, and provide a method to carefully train any teacher to retain more similarity information in its response to achieve better distillation. (iii) We also propose the distillation hypothesis to understand the underlying nature of the distillation process. Buciluǎ et al. [2006] devise a method of network compression by distilling the knowledge of a teacher network, input to the softmax layer, to a student network.  popularized this idea by introducing a temperature term T ,

Related Work
class probabilities (q i ) and logits (z i ), in softmax that controls the softness of response. Increasing  Table 1: The LS labels for different input images of same class at α LS = 0.6. It shows that the distribution of soft-labels in the absence of similarity information. This distribution promotes one example-one class learning.
T decreases q i for correct class, indicating decrease in the confidence of network's response, while increasing q i for incorrect classes, indicating increase in similarity information in network's response.
The response-based network compression is a very effective tool to instill the knowledge of a cumbersome model into an efficient model to achieve uncompromised network compression. A common understanding is that a more powerful teacher should be able to provide more knowledge to its student. But it is observed that wider and deeper networks do poor distillation Mirzadeh et al. [2020], Yuan et al. [2020], Gao et al. [2020]. Kim and Kim [2017] and Müller et al. [2019] ascribe the effectiveness of RBKD being similar to the effectiveness of LS. Ding et al. [2019] explain RBKD as the regularization effect brought about by the response of the teacher model. However, the RBKD process relies on the teacher's response and can not properly explain the hidden-layer supervision Gou et al. [2021]. We argue that whether RBKD behaves as regularization, distillation, or LS, depends on the presence of similarity information in the teacher's response. This, in tern, affects the extent of network compression that can be achieved by this method.
Mirzadeh et al. [2020] attribute the gap in learning capacities of teacher-student pair as the only reason for poor distillation, and propose a teacher assistant (TA) model, with a learning capacity in between teacher and student, to address this problem. The knowledge from the teacher is routed through one or more TA(s) to the student. While TAs improve the distillation performance but it also makes the whole process of network compression computationally expensive.
With a similar hypothesis, Yuan et al. [2020] propose a teacher-free KD (TFKD) process to reduce the gap between teacher-student pairs. The TFKD process distills its knowledge to itself during training. Theoretically, TFKD reduces the gap to zero, but its knowledge is limited by its learning capacity and extent of training. It also deviates from the network compression task. Gao et al. [2020] propose residual KD to distill the knowledge by introducing an assistant model to learn the residual error between teacher and assistant models. This method also tries to reduce the gap between teacher-student pairs. This method is also effective, but it does bring additional computational costs with added networks to achieve network compression.
All the above methods try to find a solution to reduce the gap between the teacher-student pair. They focus specifically on the distillation process and not on the knowledge inculcation process. In this paper, we show that the quality of knowledge distillation can be controlled by controlling the quality of knowledge inculcation in a teacher. We also explain how the nature of KD changes with a change in similarity information in the teacher's response with the distillation hypothesis.

Label Smoothing
The LS process uses a less confident form of one-hot label to train a model. It decreases the model confidence about the correct class and treats all other classes as equal, providing no similarity information Müller et al. [2019], Szegedy et al. [2016]. The LS labels are computed as, where, y c is the one-hot vector, c is the number of classes, and 1/c is a uniform distribution. Examples are shown in Table 1. To calculate the cross-entropy over these labels, we denote the true soft-label distribution q(c|x) (x is input) as q for LS process, q for RBKD process and use p to denote distribution p(c|x) generated by model Yuan et al. [2020]. The loss function is then,  Table 2: Soft-labels generated by small capacity teacher model with 3 hidden layers on MNIST data.

Input class
Soft-labels by large learning capacity teacher  Table 3: Soft-labels generated by a large capacity teacher model with 6 hidden layers on MNIST data where H(u) is a fixed entropy value of uniform distribution and D KL is the Kullback-Leibler divergence (KL), The soft-labels on the LS process is shown in Table 1. The LS process use a fixed distribution to produce "hard" result that has a peak response for exactly one class, and assigns all other classes equal value without considering the similarity between classes. This paradigm promotes one example-one class learning ( Fig. 1 (c)) i.e. each example provides useful information only about its class and treat all other classes equally.

Response-Based Knowledge Distillation
The RBKD process has the teacher produce probabilistic distribution for each class at a temperature T . It contains rich similarity information between classes to support one examples-many classes learning ( Fig. 1 (b)) . These information-rich labels are used to distill the knowledge to the student by minimizing the weighted sum of KL divergence and cross-entropy losses as, where H(p, q), p, p T , and p t T denote the cross-entropy loss for student with true labels (q), output of student model, output of student model softened at T and output of teacher model softened at T , respectively. α KD is the contribution factor for loss function.
The loss functions of RBKD and LS processes have similar formulations as shown in Equation (2) and (4), but they use different methods for generating soft-labels. The only difference is the p t T in D KL (p t T , p T ), which is generated by a teacher model, and u in D KL (u, p), which is a uniform distribution Yuan et al. [2020]. This difference decides the extent of similarity information in the response, and that governs the nature of the distillation process. We discuss this later in Section 4. It is safe to conclude that LS is a special case of the RBKD process in which soft labels are generated by a constant distribution as prior knowledge instead of learned-knowledge of a pre-trained teacher.

MNIST
F-MNIST CIFAR-10   Table 4: Student accuracy on missing class (MNIST) when distilled with small and large learning capacity teacher

Quality of Teacher's Response
The student mimics the teacher's response by minimizing the KL divergence between their response at T . The RBKD loss, Equation (4), is weighed higher α KD , 0.99, for KD. Each example generates two types of soft-labels, for itselfconfidence label, and for all other classes -similarity labels. The confidence label shows the confidence of a teacher for the correct class and the similarity labels provide a probabilistic value of similarity of all other classes with input class. Tables 1, 2, and 3 show the confidence label in bold text and similarity labels in plain text. The hyper-parameter α LS is 0.6 for LS process in Table 1, while temperature T is 9 for RBKD process in Tables 2 and 3.
The quality of distillation is controlled by the similarity information in the teacher's response, which can be controlled through T . But the increase and decrease in similarity information are not linear with T , but instead dependent on the similarity relation learned by the teacher during knowledge inculcation. The teacher perceives each example differently, even within the same class, and assigns a different confidence and similarity labels in its response. This introduces a variation in confidence and similarity labels -a variance in response, as shown row-wise and column-wise in Tables 2 and 3. More variance in similarity is desirable for KD as it provides knowledge about which classes are most similar to others. A highly confident teacher produces less variance as compared to a less confident teacher in its response, degrading the quality of distillation.
The behavior of a highly confident teacher model is analogous to the LS process. The similarity labels are treated equally with a little or no variance in response, providing neither similarity information nor variance in response. We describe this observation through the distillation hypothesis.

The Distillation Hypothesis
The distillation hypothesis, Figure 2, defines the nature of distillation by observing the quality of similarity labels and variance in response of a teacher. It states that for a given student network, as the learning capacity of the teacher

Missing
KD by small capacity teacher KD by large capacity teacher class Student accuracy(%) on removed class Student accuracy (%) on missing class  T=3  T=6  T=9  T=12 T=15 T=20 T=3 T=6 T=9 T=12 T=15 T=20  T- Table 6: Student accuracy on missing class (CIFAR 10) when distilled with small and large learning capacity teacher network increases, the nature of the distillation process starts to move away from a similarity label-based RBKD process to a non-similarity-based LS process. The shift towards the LS process is caused by the loss of similarity information and variance in response of a teacher and leads to poor distillation. In other words, the nature of the knowledge distillation process shifts away from one example-many classes learning to one example-one class learning.
Since the quality of distillation is directly dependent on the knowledge inculcation process of a teacher. We argue that any teacher model can be trained to retain more similarity information by finding a sweet spot between the batch size and the number of epochs for a given teacher-student pair (see Figure 1 (a)). The blue region symbolizes the right balance between the batch size and the number of epochs for better knowledge distillation.
Every DL model has an optimal batch size for optimum learning. The optimal batch size is the number of examples that the model is effectively able to process. If the batch size is larger than its optimal value, then the model is not able to process all the information at once and takes more epochs to achieve high confidence. These two factors can be iteratively balanced to reach the optimal batch size and number of epochs for knowledge inculcation of a teacher.

Experiments and Results
First, we present empirical results to support the distillation hypothesis, and then show the results of improved distillation achieved using our proposed method on MNIST, Fashion-MNIST, and CIFAR-10 datasets.

Similarity Information in Teacher's Response
The entropy of the soft labels is directly proportional to the presence of similarity information in the teacher's response. It is calculated as E Sof t−labels = − C i=1 p i log(p i ) where C is the number of classes in the dataset, p i is the probabilistic value from teacher. We use two different capacities of teachers, one with large learning capacity and the second with small learning capacity, to study this behavior. We argue that small capacity teachers having fewer parameters are not able to learn the fine-grained discriminative properties between classes and retain more similarity information in  their response as compared to the large capacity teacher which can learn fine-grained properties and lose similarity information in the process.
We show two experiments to support our argument. First, we compare the average entropy in teachers' responses for both small and large capacity teachers at different T , see Figure 3. At all T the average entropy of small capacity teachers remains higher than large capacity teachers. In other words, it is safe to say that a moderately confused teacher is better for knowledge distillation as compared to a well-trained large capacity teacher.
Second, we scrutinize the impact of similarity information for KD. We remove one out of N classes from distillation set during the KD process, and then check the performance of the student on the removed class, see Tables 4, 5, and 6. For the student network, the missing class is something it has never seen during training. It learns about the removed class only through the probabilistic value of the similarity information in the teacher's response.
The impact of similarity information on the quality of distillation is visible in Table 4, 5 and 6. Each table is divided into two columns, KD by the small and large capacity teachers. For each teacher, we perform KD on the same student and check its accuracy on the removed class on all three datasets. KD by the small teacher is very effective throughout but KD by the large teacher significantly under-performs on the removed class test. This shows that the quality of KD  is controlled by the presence of similarity information in the teacher's response. In presence of similarity-rich soft responses to classes, the student can learn about more than one class from a single input example (one example-many class learning), while it can learn about only one class from one input example if the response is "hard". We further argue that this critical information in soft labels defines the nature of distillation to be KD, regularization, or LS. We use entropy as an indicator of similarity information in the rest of the paper.

KD is one example-many class learning
We established that similarity labels are the most critical information for KD. Now, we show that similarity information accelerates the KD process and only requires a handful of examples to distill knowledge from the teacher. With two different capacity teacher models and using entropy as an indicator at T = 9, we select different number of examples varying from 5 to 2000 per class for KD. We compare the highest validation accuracy achieved by the student network at each step as shown in Figure 4. The validation accuracy of the student network distilled by a large capacity teacher always remains smaller than by a small capacity teacher on each dataset. For MNIST, distillation by a large capacity teacher requires 10 times more examples per class to achieve similar accuracy, and this gap increases for more complex datasets (15 times for Fashion-MNIST and 18 times for CIFAR-10). This shows that a large capacity teacher with less similarity information in its response fails to distill its knowledge using one example-many class learning and requires a much larger transfer set to achieve the same performance. This also establishes that one example-many class learning is a very important component of a good KD process.  Müller et al. [2019] argue that clusters that are relatively spread carry more similarity information in their response as compared to compact clusters.

Penultimate Layer Representation
The distillation hypothesis says that with the increase in learning capacity of a teacher model it loses similarity information in its response. This loss of similarity indicates that the nature of distillation shifts towards the nature of the LS process. We use this experiment to corroborate the distillation hypothesis by showing the change in clustering of penultimate layer projection which directly relates to similarity information in soft labels.
We randomly select three classes and then choose 300 examples from each class randomly. We extract its activation from the penultimate layer of both small and large capacity models and plot its projection, see Figure 5. We observe that the penultimate layer representation of the stronger teacher model forms tighter clusters as compared to small capacity teachers for each dataset. The spread-out clusters indicate that the presence of similarity information between classes that transcends to soft-labels generated by respective teachers, whereas the tighter cluster indicates the absence of similarity information between classes. This provides strong evidence for the distillation hypothesis.

Similarity-Rich Knowledge Inculcation
We already established the importance of similarity information in a teacher's response. But instead of trying to improve the KD process, we focus on the knowledge inculcation process -training the teacher. We carefully calibrate the optimum value of batch size and number of epochs to train the teacher described in section 3.4. We show improvement for each experiment shown before.
Based on our hypothesis, we show the change in average entropy of teachers' responses for different batch sizes and number of epochs in Figure 6. It shows that to increase the entropy of soft labels, we can either increase in batch size at a fixed epoch or we can decrease the number of epochs for given batch size. By carefully and iteratively balancing these two factors, we can find one of the many possible sweet spots suited for KD. This controls the extent of similarity information in the teacher's response, which is responsible for the quality of KD and network compression in this process.
Next, we show the improvement in penultimate layer representation of the large capacity teacher by adapting to our method of knowledge inculcation in Figure 7. Figure 7(a) shows tight clusters, indicating a lack of similarity information. These clusters spread out for many combinations of batch sizes and number of epochs indicating more similarity information in the teacher's response. A similar clustering is observed for batch size 4096 -epochs 50 and for batch size 8192 -epochs 40 in Fig. 7(b) and (c), respectively. We can conclude that there can be many combinations of batch size and number of epochs for a given teacher-student pair for a more efficient KD.
With all these improvements in the quality of teachers' responses, the minimum number of examples needed to achieve optimal performance should also reduce for large teachers. As shown in Fig. 8, we improve around 5 times in the minimum number of examples required per class to achieve similar performance, decreased from 500 to 100 examples Figure 9: Improved performance on training with suitable batch size and epochs, per class, for efficient KD. This provides further support to our idea of focusing on the knowledge inculcation process for achieving better and faster distillation, leading to better network compression.
Even with the above empirical results supporting our hypothesis, we still wanted to test our hypothesis for extreme teacher-student pair situations by maximizing the learning capacity gap between them. To achieve this, we use a single-layer student (baseline student) model with only a classification layer (we provide detail of models used in this paper in Table 7). We retrain the same large capacity teacher models for each dataset but train it differently -with different batch sizes and number of epochs, to accommodate for the larger gap in model capacities and use it for KD to the baseline student. We also use the small capacity teacher model to show the relative improvements in Fig. 9. The black, blue, and red lines in Fig. 9 shows the performance of baseline students on RBKD by small, and large capacity teachers trained with different batch size and epochs at different temperatures. The gap between the red and the blue line shows the improvement in KD performance by a single-layer student.

Summary and Conclusion
For building intuition about the nature of distillation, we present the entropy-version of the distillation hypothesis (previously shown in Fig. 2) to approximate the nature of the distillation process in Figure 10. For all the processes lying on the extreme ends in Fig. 2, we can imaging multiple lines with different entropy spread across, as shown in Fig.  10. We can draw an imaginary line (the dashed line in the figure) that separates the RBKD process and the LS process.  Table 7: Details of teacher and student models for different datasets. Conv, Dense, Dropout, and BN denote the number of convolution layers, number of Dense layers, dropout layers, and Batch Normalization layers, respectively. For efficient distillation from any teacher to any student, the corresponding entropy line should lie in the RBKD region. This can be achieved using our proposed method of knowledge inculcation during teacher's training. We emphasize that this representation is to develop an intuitive understanding of the KD process.
We discuss that the quality of distillation can be controlled at various stages of the KD process. While most works focuses on perfecting the distillation step of the KD process, we focus on controlling the quality of distillation by improving the knowledge inculcation process of the teacher model. Our experiments suggest that similarity information in a teacher's response plays a dictating role in determining the quality of distillation. We show the role of similarity information in accelerating the distillation process through one example-many class learning, making the KD process more effective and efficient. The distillation hypothesis can help in approximating the nature of the KD process to be RBKD, regularization, or LS process and to help make the process more effective. While we argue that the distillation performance of any teacher-student pair can be improved by our method of knowledge inculcation, we do not explore the extent to which the KD process can be improved. We believe that the factor of improvement should be different for each teacher-student pair. We leave this for future exploration.


Action recognition based on element-level fine-grained multi-modal fusion

Traditional action recognition algorithms often only pay attention to video RGB features or optical flow features. These methods do not make good use of the audio information in the video. Based on RGB and optical flow characteristics, this paper introduces the processing of audio information, and classifies videos based on element-level fine-grained multi-modal fusion. Through experimental comparison, the accuracy of the multi-modal fusion algorithm proposed in this paper is improved by 7.38% on the HMDB51 dataset and 3.18% on the UCF101 dataset compared to the simple modal splicing. At the same time, it is proved that the introduction of audio modes can effectively improve the performance of the model.

Introduction
Action recognition is to recognize human actions from a video. However, the video gathers rich content information such as visual information and auditory information. Research shows that different modalities can affect the accuracy of action video recognition, so how to be effective Extracting content features and integrating them becomes the research focus. In recent years, a large number of scholars have paid attention to the research based on multi-modal action recognition and have produced many excellent results, but there are still some shortcomings. Existing action recognition often uses the visual information of the video when analyzing the features of the video content, and treats the extraction of the visual features of the video as image feature extraction for processing, and lacks the use of the rich information contained in the video frame. In addition, the optical flow information of the video can reflect the motion information in the video, and the audio branch can contain the emotional representation of the video. The characteristics of the two branches can bring positive benefits to the video understanding, so they cannot be ignored. In the feature fusion stage, the existing fusion methods are difficult to analyze the relationship between modal branches.
In response to the above problems, this article proposes some improvement methods.
(1) Aiming at the problem of insufficient utilization of video frame-level information, this paper first uses the inter-frame difference method to extract the key frames corresponding to the video, uses the FlowNet network to extract the optical flow information corresponding to the video, and processes the RGB spatial information and optical information of the video key frames in parallel. Stream information.
(2) For each key frame, this paper introduces the AdaScan algorithm [1] to analyze its contribution to the understanding of video semantics, so that the model tends to strengthen the frames that have a greater impact on the understanding of video semantics while ignoring redundant frames. (3) In view of the existing fusion methods that are difficult to analyze the related information between modal branches, this paper proposes an element-level fine-grained feature fusion method, calculates the element-level influence of each modal branch, and fuses them with weight.

Feature extraction and fusion
2.1. Visual feature extraction 2.1.1 Key frame selection Video in real life contains a very large number of frame sequences, and there are very serious redundancy problems between consecutive video frames. In order to reduce the computational complexity, it is necessary to filter key frames. Because the video frame sequence has continuous characteristics and can express the characteristics of the video in the time sequence dimension, the key frame selection should be able to reflect the timing dependence of the relevant frame on the video on the side, and the sampling should save as much as possible the summary of the video content of the key frame sequence ability to avoid the loss of main information. The existing key frame selection algorithms include uniform sampling, continuous sampling, frame difference method and clusteringbased methods. Continuous sampling usually selects the starting position of sampling in a random manner. From this position, several consecutive video frames are taken. It can be found that continuous sampling cannot fully express the video content, and serious information loss may occur. Uniform sampling is to extract frames from the video at a certain interval. Extracting frames without filtering is prone to individual frame redundancy and some key information loss. The clustering method treats all the frames in the video as a set. By calculating the similarity between the frames, the clustering algorithm is used, and finally the center of the cluster is selected as the key frame. It can be found that a group represents a person lifting up in the video The hand movements are all regarded as similar frames, and the clustering algorithm often chooses one frame to extract, which cannot solve the timing problem in the video. Considering comprehensively, this paper chooses the inter-frame difference method as the method of extracting video key frames. First, read the video file, make the difference between the pixels at the corresponding positions of two adjacent frames, as the difference strength of the two frames, and then select the video frame with the inter-frame difference greater than the threshold as the key frame. However, it should be noted that this article does not address short videos such as gesture recognition and action classification. In long videos, there are often scene changes. At this time, the differential strength may be too large. In order to make the sampling method suitable for longer videos, it is understood that this article also adds the upper threshold of the threshold to filter the sudden changes between frames (such as scene changes, etc.).

Optical flow information
In this paper, the FlowNet network is selected as the optical flow extraction algorithm. The network is divided into the encoding module shown in Figure 1 and the decoding module shown in Figure 2. The coding module part includes 9 convolutional layers and a ReLU function activation layer, and the decoding part includes 4 deconvolutions and a ReLU function activation layer. The network directly stacks two frames of images as the input of the encoding module; and in the decoding module, for each deconvolution layer, the output of the previous layer is input at the same time, the optical flow predicted by the network for the previous layer, and the encoding module The feature layer, in this way, obtains deeper abstract information and shallow specific information of the image to make up for the information lost due to the reduction of the feature space scale.

Feature extraction
The spatial features of video mainly include scene, texture and edge features, which can generally be well understood by processing RGB images. Therefore, this paper uses the key frames extracted by the above steps to extract video spatial branching features. Literature [2] pointed out that usually only a part of the frames in a video contain useful information for video understanding. At the same time, the AdaScan network is proposed to extract the spatial characteristics of the video by traversing the video frames and overlaying their relative importance with weights. The idea of the AdaScan algorithm is to determine the importance of each frame of video after receiving the characteristics of each frame of the video. For video frames with minimal recognition effect, the author assigns them a very small weight to achieve the goal of "discarding" useless frames effect. The key module of the algorithm is adaptive pooling, which recursively predicts the importance of the recognition and understanding of the current frame based on the characteristics of the current frame and the frame set vector up to the current frame. In this way, the video frames that have a greater influence on the recognition video are gathered, reducing the overall impact of redundant frames and reducing noise. The AdaScan network structure is shown in Figure 3. This article divides it into two branches, which are used to process RGB frames and optical flow information. The following uses RGB frames as an example to illustrate the calculation process of the AdaScan algorithm. The optical flow information is similar.

Audio feature extraction
Audio data cannot be directly input to the network for calculation, so the audio data needs to be preprocessed before use. This article first resamples the audio file, then uses the Fourier transform to obtain the corresponding spectrogram, and finally uses the VGGish network to extract the characteristics of the spectrogram as the audio characteristics. After the spectrogram corresponding to the audio file is obtained, it is input into the VGGish network.

Feature fusion
The existing multi-modal fusion methods are mainly divided into early fusion and late fusion. Early fusion [3][4] extracted the feature vector expressions of each modal branch, and spliced them as the input of the final classifier, but the early fusion ignored the connection between the modal branches, and had some expression ability. Lack; late fusion [5][6][7] trains the model separately on each modal branch and obtains the corresponding results, and finally takes the weighted average or takes the maximum value as the output result of the model. However, neither early fusion nor late fusion can analyze the correlation between modal branches at a fine-grained level. The relationship between the various modal branches of the video is complicated, and the understanding of the semantics of the video content is not the same. This article proposes an element-level feature fusion method as shown in Figure 4 to analyze the modal branches at a fine-grained level. Analyze the influence of element level between different modal branches in parallel, strengthen the influence of main elements on the model, and suppress noise data at the same time.

dataset
This article uses a total of two data sets, namely HMDB51 [8], UCF101 [9]. The video data in UCF101 is collected from YouTube. The data set has 13,320 videos in total, which are divided into 101 categories.
UCF101 is currently one of the video classification data sets with the largest number of action categories and samples. The 101 action category videos in this data set are divided into 25 groups, and each group roughly contains 4-7 action categories.
Most of HMDB51's data are clips from movies, and some are from public databases, such as Prelinger archives, YouTube videos, and Google videos. The data set contains a total of 6849 video clips. These video clips are divided into 51 action categories, and each action category contains at least 101 video clips.

Experiment results
This paper compares the two data sets of HMDB51 and UCF101, and combines the feature fusion algorithm proposed in this paper with RGB, RGB and Flow information, as well as simple mosaic of RGB, Flow and Audio respectively. By comparing the experimental results of RGB, RGB+Flow and RGB+Flow+Audio, it is found that the increase in the number of modes can effectively improve the accuracy of the results, and the increased modes make up for the information not contained in other modes to a certain extent. Comparing the experimental results of the proposed algorithm and RGB+Flow+Audio, it is found that the element-level feature fusion algorithm proposed in this paper can effectively improve the accuracy of the model.

Conclusion and prospect
Traditional action recognition algorithms have problems such as insufficient use of video frame-level features or excessive redundant information, and insufficient fusion of visual information and auditory information. Therefore, this chapter first uses video key frames to extract visual features, and then uses AdaScan to perform. The secondary screening is mainly divided into two aspects: RGB spatial information and optical flow information, and then fully integrated with the auditory information extracted through the pre-training network.
In the feature fusion stage, this paper proposes an element-level feature fusion method to analyze the relationship between modal branches, analyze the influence of different modal branches at the element level in parallel, and add entropy regularization to help the model tend to choose Elements with greater impact on video recognition are discarded, and elements with less impact are discarded to achieve the purpose of element screening.


extfieldsofstudy:
Physics
Computer Science
Biology

text:
Ensemble perspective for understanding temporal credit assignment

Recurrent neural networks are widely used for modeling spatio-temporal sequences in both nature language processing and neural population dynamics. However, understanding the temporal credit assignment is hard. Here, we propose that each individual connection in the recurrent computation is modeled by a spike and slab distribution, rather than a precise weight value. We then derive the mean-field algorithm to train the network at the ensemble level. The method is then applied to classify handwritten digits when pixels are read in sequence, and to the multisensory integration task that is a fundamental cognitive function of animals. Our model reveals important connections that determine the overall performance of the network. The model also shows how spatio-temporal information is processed through the hyperparameters of the distribution, and moreover reveals distinct types of emergent neural selectivity. It is thus promising to study the temporal credit assignment in recurrent neural networks from the ensemble perspective.

FIG. 1: Illustration of a RNN learning the temporal credit assignment. Each connection is described by the spike and slab distribution; the spike indicates the corresponding synaptic weight absent for a task, while the slab represents a Gaussian distribution responsible for the weight uncertainty. The Gaussian slab is displayed in the plot, and the arrow in the reservoir shows different directions of information flow. Different sizes of connection show how probable these connections should be present during recurrent computation. moreover the selectivity of each recurrent unit, we are able to provide mechanistic understanding about the recurrent dynamics in both engineering applications and computational neuroscience modeling. Our method learns the statistics of the RNN ensemble, producing a dynamic architecture that adapts to temporally varying inputs, which thereby goes beyond traditional training protocols focusing on a stationary network topology. The hyperparameters governing the weight distribution reveal which credit assignments are critical to the network behavior, which further explains distinct functions of each computational layer and the emergent neural selectivity. Therefore, our ensemble theory can be used as a promising tool to explore internal dynamics of widely used RNNs.

II. MODEL AND ENSEMBLE TRAINING
In this study, we consider a recurrent neural network processing a time-dependent input x(t) of time length T . The input signal is sent to the recurrent reservoir via the input weight matrix W in . The number of input units are determined by design details of the task, and w in ij denotes the weight value for the connection from input unit j to recurrent node i. The neural responses of recurrent units are represented by an activity vector r(t) at time step t. We define w ij as the connection from node j to node i in the reservoir. In general, w ij = w ji , indicating different weights for different directions of information flow. In addition, we do not preclude the self-interaction w ii , which can be used to maintain the representation encoded in the history of the internal dynamics [19]. The specific statistics of this self-interaction can be determined by the learning shown below. The internal activity r(t) is read out through the output matrix W out in the form of the time-dependent output y(t).
We first define h i (t) as the time-dependent synaptic current of neuron i. The dynamics of the recurrent network can thus be summarized as follows, where u(·) is the pre-activation function, φ(·) denotes the nonlinear transfer function, for which we use the rectified linear unit (ReLU) function for all tasks. α = ∆t τ , where ∆t is a small time interval (e.g., emerging from discretization of a continuous dynamics [20]), and τ denotes the time constant of the dynamics. n i ∼ N (0, 1) indicates a normally distributed random number with zero mean and unit variance, sampled independently at every time step, and σ controls the strength of the recurrent noise intrinsic to the network. This intrinsic noise is present in modeling cognitive tasks, but absent for engineering applications. Considering a Gaussian white noise and a baseline (x 0 ), we write the input vector x(t) as where the input has been written in a discrete time form, x task t denotes the sequence of the task, and ξ i ∼ N (0, 1), which is independently sampled at each time step. The noise term is also called the external sensory noise of strength σ in , commonly observed in brain circuits [21,22], but is absent in engineering applications. For a MNIST classification task, f (·) is chosen to be the softmax function, specifying the probability over all the classes at the last time step T , i.e., y k (T ) = e z k (T ) j e z j (T ) . We defineŷ k as the target label (one-hot form), and use the cross entropy L = − iŷ i ln y i (T ) as the loss function to be minimized. For a multisensory integration task, f (·) is an identity function, and the mean squared-error is chosen to be the objective function.
To search for the optimal random network ensemble for time-dependent tasks in recurrent neural networks, we model the statistics of the weight matrices by the SaS probability distribution [13] as follows, The spike mass at δ(·) is related to the network compression, indicating the necessary weight resources required for a specific task. The continuous slab, N (w ij |m ij , Ξ ij ), denotes the Gaussian distribution with mean m ij and variance Ξ ij , characterizing the weight uncertainty when the corresponding connection can not be absent (see Fig. 1). Next, we derive the learning equations about how the SaS parameters are updated, based on mean-field approximation. More precisely, we consider the average over the statistics of the network ensemble during training. Notice that, the first and second moments of the weight w ij for three sets of weights can be written in a common form as . Given a large fan-in, the pre-activation u i (t) and the output z i (t) can be re-parametrized by using standard Gaussian random variables , i.e., it is reasonable to assume that they are subject to N (u i (t)|G in , respectively, according to the central-limit-theorem. Note that when the number of fan-in to a recurrent unit is small, the centrallimit-theorem may break. In this situation, we take the deterministic limit. Therefore, the mean-field dynamics of the model becomes where Note that { u (t)} and { out (t)} are both independent random variables sampled from the standard Gaussian distribution with zero mean and unit variance, which are quenched for every single training mini-epoch and also time-step dependent, maintaining the same sequence of values in both feedforward and backward computations. Updating the network parameters (θ θ θ in ik , θ θ θ ik , θ θ θ out ki ) can be achieved by the gradient descent on the objective function L. First of all, we update θ θ θ out ki ≡ (m out ki , π out ki , Ξ out ki ).
where ∂L ∂z k (t) is related to the form of the loss function. For categorization tasks, f (·) is chosen to be the softmax function, and we use the cross entropy as our objective function, for which ∂L ∂z k (t) = (y k (T ) −ŷ k )δ t,T . It is worth noticing that for multi-sensory integration tasks, this derivative does not vanish at intermediate time steps and become thus time-dependent. The other term ∂z k (t) ∂θ out ki can be directly computed, showing how sensitive the network activity is read out under the change of the SaS parameters in the output layer: (7c) Next, we derive the learning equation for the hyper-parameters in the reservoir and input layer, i.e., θ θ θ ij ≡ (m ij , π ij , Ξ ij ) and θ θ θ in ij ≡ (m in ij , π in ij , Ξ in ij ). To get a general form, we setm ij ≡ (m ij , m in ij ),π ij ≡ (π ij , π in ij ), and Ξ ij ≡ (Ξ ij , Ξ in ij ). Based on the chain rule, we then arrive at the following equations, where we have defined δ i (t) ≡ ∂L ∂hi(t) . The auxiliary variable δ i (t) can be computed by the chain rule once again, resulting in a BPTT equation of the error signal starting from the last time-step T : where t = 0, 1, 2, ..., T − 1, and φ (·) denotes the derivative of the transfer function. The first summation in δ i (t) can be directly expanded as j ∂L ∂h j (t + 1) .
The second summation in δ i (t) is given by It is worth noting that for the last time-step T , the error signal δ i (T ) is written as To compute Eq. (8), we need to work out the following derivatives, which characterize the sensitivity of the preactivation under the change of hyper-parameters θ θ θ ij and θ θ θ in ij . We summarize the results as follows, In this learning process, our model learns a RNN ensemble to realize the time-dependent computation, in contrast to the standard BPTT algorithm which gives only a point-estimate of RNN weights. In particular, if we set π = 0 and Ξ = 0, our learning equation reduces to the standard BPTT. Hence, our model can be thought of as a generalized version of BPTT (i.e., gBPTT), as shown in Fig. 2, where each weight parameter should be understood as the SaS hyper-parameters, corresponding to our ensemble setting.
The size of the candidate network space can be captured by the network entropy S = − R D P (w) ln P (w)dw, where D is the number of weight parameters in the network. If we assume the joint distribution of weights to be factorized across individual connections, the overall energy S can be obtained by summing up the entropy of individual weights as S = S . The entropy of each directed connection is derived as follows [13]: where Γ( s ) = ln [π δ(m + √ Ξ s ) + 1−π √ Ξ N ( s |0, 1)] and B denotes the number of standard Gaussian variables s . This model entropy is just an approximate estimate of the true value whose exact computation is impossible.
If π = 0, the probability distribution of w can be written as P (w ) = N (w |m , Ξ ), and the entropy S can be analytically computed as 1 2 ln (2πeΞ ). If Ξ = 0, the Gaussian distribution reduces to a Dirac delta function, and the entropy becomes an entropy of discrete random variables, S = π ln π + (1 − π ) ln (1 − π ). However, there may exist a mixture of discrete and continuous contributions to the entropy. Therefore, the entropy value can take negative values. In practice, we use δ( 2a to approximate the delta-peak with a small value of a. The stochasticity of the SaS distribution can be also decomposed into two levels: one at the choice of connection that is absent and the other at the Gaussian slab itself. The former one is a discrete entropy characterized by the spike probability, and the latter is a continuous entropy just characterized by the variance of the Gaussian slab.

III. RESULTS
In this section, we show applications of our ensemble theory of temporal credit assignment to both engineering and computational cognitive tasks, i.e., pixel-by-pixel MNIST digit classification, and the multisensory integration demonstrating the benefit of multiple sources of information for decision making. We will explore in detail rich properties of trained RNN model accomplishing the above computational tasks of different nature.
Illustration of the generalized back-propagation through time. Xt is a time-dependent input into the network, resulting in a sequence of hidden representations r(t) (i.e., φ(ht)) and a time-dependent loss Lt. The error signal ∂L t ∂h t propagates back from the last time-step T to the first time step, yielding the temporally accumulated gradients for hyper-parameters to be updated. The contribution of ∂L t ∂z t is not shown in this plot. Xt, zt and ht indicate x(t), z(t) and h(t) in the main text, respectively.
FIG. 3: Properties of trained RNN models for the pixel-by-pixel MNIST classification. (a) Test accuracy versus training epoch. The network of N = 100 neurons is trained on the full MNIST dataset (60 000 images), and tested on the unseen data of 10 000 images. Five independent runs of the algorithm are considered. There are no baseline inputs. Other parameters for training are as follows: α = 0.1, 2 regularization strength is 10 −4 , and the initial learning rate lr0 = 0.001. (b) Evolution of the network sparsity per connection. Recurrent I denotes the case of wij (i < j) along which the recurrent feedback passes from neuron j to neuron i; while recurrent II denotes the case of wji (j > i) along which the recurrent feedback passes from neuron i to neuron j. The fluctuation is estimated from five independent runs [the same training conditions as in (a)].

A. Pixel-by-Pixel MNIST digit classification
Training RNNs is hard because of long-term dependency in the sequence of inputs. A challenging task of long-term dependency is to train a RNN to classify the MNIST images when the 784 pixels are fed into the network one by one (note that x 0 = 0), where the network is required to predict the category of the image after seeing all the 784 pixels. This task displays a long range of dependency, because the network reads one pixel at a single time-step in a scan-line order from the top left pixel to the bottom right pixel (Fig. 5), and the information of as long as 784 time steps must be maintained before the final decision. We apply the vanilla RNN with N = 100 recurrent units to achieve this challenging goal. Because the input size is one in this task, and thus the set of hyper-parameters (π in , Ξ in ) are set to zero (i.e., the deterministic limit). The entire MNIST dataset is divided into mini-batches for stochastic gradient descent, and we apply cross entropy as the objective function to be minimized. In this task, the  Fig. 3. Five independent runs are considered. The input weight is deterministic without any stochasticity. In addition, to avoid divergence, we relax all values of Ξ smaller than a to be a in the computation of the entropy. Our results are not sensitive to the choice of a, once the value of a is small. (a,b) We decompose the variability of the SaS distribution into two levels: the variability of selecting zero synapses (π-entropy) and the variability inside the Gaussian slab (Ξ-entropy). (c,d) Evolution of entropy per connection (B = 100). We use the Gaussian distribution of small variance (indicated by a) to approximate the delta-peak. The value of a does not affect the qualitative behavior of the entropy profile.
error signal appears only after the network reads all the 784 pixels. In the current setting, we ignore the noise terms in the dynamics equations [Eq. (1) and Eq. (2)]. Surprisingly, although working at the ensemble level, our model can achieve a comparable or even better performance than the traditional BPTT method, as shown in Fig. 3 (a).
Our simulation reveals that the sparsity per connection, l π l M (a total of M directed connections in the network), achieves a larger value in the recurrent layer compared with the output layer; this behavior does not depend on the specific directions of the coupling [ Fig. 3 (b)]. During training, the sparsity level grows until saturation, suggesting that the training may remove or minimize the impacts of irrelevant information across time steps. Interestingly, the entropy per connection in the recurrent layer also increases at the early stage of the training, but decreases at the late stage [ Fig. 4], showing that the training is able to reorganize the information landscape shaped by the recurrent feedbacks. The network at the end of training becomes more deterministic (e.g., Ξ gets close to zero, or the spike probability goes away from one half). The discrete π-entropy always increases until saturation at a certain level. The entropy profile in the output layer shows a similar behavior yet with a lower entropy value. This is consistent with distinct roles of recurrent and readout layers. The recurrent layer (or computing reservoir) searches for an optimal way of processing temporally complex sensory inputs, while the output layer implements the decoding of the signals hidden in the reservoir dynamics. Note that two types of deterministic weights yield vanishing individual entropy values, i.e., (i) π= 1 (unimportant (UIP) weight); (ii) π = 0, Ξ = 0 and m = 0 (very important (VIP) weight).
To study the network behavior from the perspective of hyper-parameter distributions, we plot the distribution of three sets of parameters (m, Ξ, π) for the output layer, two different directions of connections in the recurrent layer, and m of the input layer ((π in , Ξ in ) are set to zero as explained before), as shown in Fig. 5. The distribution of spike probability π has the shape of L for all layers. The extreme at π = 0 indicates that the corresponding synaptic weight carries important feature-selective information, and are thus critical to the network performance. The shape of π-distribution profile reflects the task difficulty for the considered network (given the same initialization condition), since we observe that in a simpler task for the network to read 28 pixels at each time step (rather than one pixel by one pixel), the profile of π-distribution develops a U shape, with the other peak at π = 1, implying the emergence of a significant fraction of UIP weights. In addition, Ξ has an L-shaped distribution which peaks at zero, suggesting that the corresponding weight distribution takes a deterministic value of m, which becomes the weight value of that connection.
As for the readout layer, the mean of the continuous slab becomes much more dispersed, compared with that of the recurrent layer, while the statistics profile for π and Ξ becomes more converged, which is in an excellent agreement with the entropy profile shown in Fig. 4. We thus conclude that the recurrent layer has a greater variability than the output layer. This great variability allows the recurrent layer to transform the complex sensory inputs with hierarchical spatiotemporal structures in a flexible way. Nevertheless, the minimal variability makes the readout behavior more robust. It is therefore interesting in future works to address the precise mechanism underlying how the statistics of the connections in a RNN facilitates the formation of latent dynamics for computation.
We next look at the specific profile of individual hyper-parameters in a matrix form (Fig. 6). We find that the diagonal (self-interaction) of m emerges from gBPTT (note that the diagonals of π and Ξ nearly vanish), demonstrating the significant role of self-interactions in maintaining long-term memory and thereby the learning performance, in accord with the heuristic strategy used in a recent study [19]. There also appears heterogeneity in the hyper-parameter matrices, i.e., some directed connections play a more important role than the others. In particular, some connections can be eliminated for saving computation resources.
Our method can identify the exact nature of each directed connection in the network. Targeted weight perturbation can thus be performed in the recurrent layer (Fig. 7) . In the course of training, the fraction of VIP connections slowly decreases, although the fraction of VIP connections is not significant (around 1%) [ Fig. 7(b)]. Surprisingly, pruning these VIP weights could strongly deteriorate the test performance of the network (up to the chance level), whereas pruning the same number of randomly selected connections from the entire weight population yields a negligible drop of the test accuracy. We thus conclude that in a RNN, there exists a minority of VIP weights carrying key spatiotemporal information essential for decision making of the network.
We then explore whether the selectivity of neurons for the sensory inputs could emerge from our training. The degree of selectivity for an individual unit, say neuron i, can be characterized by an index, namely SLI [23] as follows where r i,s indicates the response of the neuron to the input stimulus s with the total number of stimulus-class being N s (N s = 10 in the MNIST experiment). Note that r i,s for the RNN depends on the time step as well. The value of SLI ranges from 0 (when the neuron responds identically to all stimuli) to 1 (when the neuron responds only to a single stimulus), and a higher SLI indicates a higher degree of selectivity, and vice versa. Interestingly, we find that most neurons have mixed selectivity shown in Fig. 8 (the right panel), whose SLI lies between 0 and 1, and such neurons respond strongly to a few types of images. The mixed selectivity was also discovered in complex cognitive tasks [24]. We also observe that some neurons are particularly selective to only one type of images with a high value of SLI [ Fig. 8 (middle)], while the remaining part of neurons keeps silent with an SLI near to zero [ Fig. 8 (left)]. As expected, the selectivity property emerges slightly before the decision making of the network. We finally remark that gBPTT produces a network ensemble, characterized by the hyper-parameter set of the SaS distribution. A concrete network can thus be sampled from this ensemble. During training, we use independently time-dependent Gaussian noise u (t) to approximate the statistics of the ensemble. Accordingly, we find that a timedependent concrete network yields the identical test performance with the mean-field propagation (i.e., using the parametrized pre-activation, see Eq. (4)]. The overall statistics of sampled networks does not vary significantly across dynamic steps (Fig. 9). This observation is in stark contrast to the traditional RNN training, where a deterministic weight matrix is used in all time steps. In fact, in a biological neural circuit, changes in synaptic connections are essential for the development and function of the nervous system [25,26]. In other words, the specific details of the connection pattern for a circuit may not be critical to the behavior, but rather, the parameters underlying the statistics of weight distributions become dominant factors for the behavioral output of the network (see also a recent paper demonstrating that innate face-selectivity could emerge from statistical variation of the feedforward projections in hierarchical neural networks [27]). Therefore, our current ensemble perspective of training RNNs offers a promising framework to achieve neural computation with a dynamic network, which is likely to be consistent with biologically plausible computations with fluctuating dendritic spines, e.g., dendritic spines can undergo morphological remodeling in adaptation to sensory stimuli or in learning [26].

B. Multisensory integration task
Multisensory integration is a fundamental ability of the brain to combine cues from multiple senses to form robust perception of signals in the noisy world [16,17]. In typical cognitive experiments, two separate sources of information  FIG. 10: Schematic illustration of the RNN performing a multi-sensory integration task. In one trial, the task consists of three consecutive stages (of lengths 300, 900, and 400 ms, respectively) including a fixation period (t0 ∼ 0), a stimulus period (0 ∼ t1), and a decision period (t1 ∼ t2). Note that in a test trial, the lengths of three stages are 500, 1000, and 300 ms, respectively. During the fixation period, the network must maintain a low output value, ensuring that the network will be sensitive to the stimulus signals. During the stimulus period, either or both (or multisensory) sources of signals with the same frequency f , including visual signal and auditory signal, act as the input to the network. The frequency f ranges from 9 to 16. Each type of sensory input contains positively tuned and negatively tuned signals (only the positively tuned one is shown in the plot). During the decision period, there are no modality input signal, and the network must learn to discriminate the even rate of the stimulus period, i.e., the output must hold a high or low value correctly. The output at the last time step of the decision period is assigned the final decision of one trial. Other parameters for training are as follows: x0 = 0.2 for all neurons, τ = 100 ms, α = 0.2, σ = 0.15, σin = 0.01, 2 regularization strength is 10 −3 , and the initial learning rate lr0 = 0.001. ∆t = 0.5 ms for testing.
are provided to animals, e.g., rats, before the animals make a decision. The source of information can be either auditory clicks or visual flashes, or both [28]. When the task becomes difficult, the multisensory training is more effective than the unisensory one. Akin to animals trained to perform a behavior task, RNNs can also learn the input-output mapping through an optimization procedure, which is able to provide a quantitative way to understand the computational principles underlying the multisensory integration. In particular, by increasing biological plausible levels of network architectures and even learning dynamics [8,10], one may generate quantitative hypotheses about the dynamical mechanisms that may be implemented in real neural circuits to solve the same behavior task. In this section, we restrict the network setting to the simplest case, i.e., a group of neurons are reciprocally connected with continuous firing rate dynamics. To consider more biological details is straightforward, e.g., taking Dale's principle [21]. The goal in this section is to show that our ensemble perspective also works in modeling cognitive computational tasks.
In the multisensory integration experiment (MSI), either or both of auditory and visual signals are fed into the recurrent network via deterministic weights (the same reason as in pixel-by-pixel MNIST task), which is required, after a stimulus period, to report whether the frequency of the presented stimulus is above a decision boundary (12.5 events per second)(see Fig. 10). One third of network units receive only visual input, while another third receive only auditory input, and the remaining third do not receive any input. The continuous variable r(t) in our model is an activity vector indicating the firing rates of neurons, obtained through a non-linear transfer function (ReLU here) of the synaptic current input [h(t)]. The current includes both of external input and recurrent feedback. The output z(t) is a weighted readout of the neural responses in the reservoir (a binary choice for the MSI task). The RNN is used here to model the multisensory event-rate discrimination task for the rats [21,28], and is trained by our gBPTT to solve the same audiovisual integration task. The visual and auditory inputs can be either positively (increasing function of event rate) or negatively tuned (decreasing function of event rate). Showing the network both types of tunned inputs could improve the training [29]. The RNN is composed of 150 neurons, whose recurrent dynamics is required to hold a high output value if the input event rate (represented by time-dependent noisy inputs) was above the decision boundary, and hold a low output value otherwise. Neurons are reciprocally connected, and the global statistics of the topology is learned from the training trials. The benefits of multisensory inputs, as known in cognitive science [16], are reproduced by our RNN model trained by gBPTT (Fig. 11). Integrating multiple sources of information, rather than unisensory inputs, helps decision making particularly when the task becomes hard (i.e., around the decision boundary). As the training proceeds, the sparsity of the recurrent layer grows rapidly until saturation, while the sparsity of the output layer grows in the same manner but finally reaching a lower value yet with a small fluctuation (Fig. 12). The recurrent layer becomes sparser with training, demonstrating that the latent dynamics is likely low dimensional, because of existence of some unnecessary degrees of freedom in recurrent feedbacks. In contrast, the goal of the output layer is to decode the recurrent dynamics, and the output layer should therefore keep all relevant dimensions of information integrated, which requires a densely connected output layer. This behavior is consistent with the evolution of the entropy profile. The recurrent layer maintains a relatively higher level of variability, compared with that of the output layer at the end of training (see the continuous entropy computed according to Eq. (14), or the Ξ-entropy in Fig. 13). In addition, the discrete π-entropy decreases with training in the output layer, in contrast to the increasing behavior of the same type of entropy in the recurrent layer (Fig. 13). Let us then look at the distribution profile of hyperparameters (Fig. 14). In the output layer, the distribution of π is U-shaped, and Ξ shows a sharp single peak at zero, which demonstrates that a dominant part of the weight distribution reduces to the Bernoulli distribution with two peaks at 0 and m = 0 respectively. The observed less variability in weight values of the output layer is consistent with the decoding stability. In the recurrent layer, the profile of π-distribution is U-shaped, and the distribution profile of Ξ resembles an L shape. This implies that there emerge VIP and UIP connections in the network. Moreover, a certain level of variability is allowed for the weight values, making a flexible computation possible in the internal dynamics.
Next, we ask whether our training leads to the emergence of neural selectivity, which indeed exists in the prefrontal cortex of behaving animals [24]. The selectivity properties of neurons play a critical role in the complex cognitive ability. In our trained networks, we also find that neurons in the recurrent layer display different types of selectivity (Fig. 15). In other words, neurons become highly active for either of modality (visual or auditory) and choice (high or low), or both (mixed selectivity).
We then explore the detailed patterns of the hyper-parameter matrices, which conveys individual contributions of each connection to the behavioral performance. By inspecting the sparsity matrix [ Fig. 16 (b)], one can identify both unnecessary (π = 1) and important connections (π = 0). We also find that, some neurons prefer receiving or sending information during the recurrent computation. An interpretation is that, the spatio-temporal information is divided into relevant and irrelevant parts; the relevant parts are maintained through sending preference, while the irrelevant parts are blocked through receiving preference.
Target weight perturbation experiments (Fig. 17) show that the VIP weights play a fundamental role in supporting the task accuracy reached by the recurrent computation. Our method can thus provide precise temporal credit assignment to the MSI task, which the standard BPTT could not.
Finally, we remark that a dynamic network with time-dependent specified weight values is able to reach an equivalent accuracy with the network using the mean-field propagation. Note that the overall statistics of the network does not change significantly (Fig. 18), suggesting that the hyperparameters for the the weight statistics are more important than precise values of weights. In fact, dendritic spines in neural circuits, biological substrates for synaptic contacts, are also subject to fluctuation, i.e., a highly dynamic structure [25,26]. Future exploration of this interesting connection would be fruitful, as an ensemble perspective is much more abstract than a concrete topology, while the specified stationary topology is still widely used in modern machine learning. Therefore, the ensemble perspective yielding a  dynamic network in adaptation of external stimuli could shed light on our understanding of adaptive RNNs.

IV. CONCLUSION
In this study, we propose an ensemble perspective for understanding temporal credit assignment, which yields a generalized BPTT algorithm for training RNNs in practice. Our training protocol produces the hyperparameters underlying the weight distribution, or the statistics of the RNN ensemble. In contrast to the standard BPTT, gBPTT highlights the importance of network statistics, which can particularly make a dynamic network changing its specified network weights a potential substrate for recurrent computation in adaptation to sensory inputs. It is thus interesting in future works to explore the biological counterpart of neural computation in brain circuits, e.g., in terms of dendritic spine dynamics [25].
Our SaS model has three types of hyperparameters with distinct roles. m tells us the mean of the continuous Gaussian slab, while Ξ controls the variance (fluctuation) of the slab. In both computational tasks we are interested in, we find that the Ξ-distribution profile is L-shaped. In other words, the peak at zero turns the SaS distribution into a Bernoulli distribution, while the tail at finite values of variance endows a computational flexibility to each connection, which may be critical to recode the high-dimensional sensory inputs into a low-dimensional latent dynamics. It is thus interesting to establish this hypothesis, by addressing precisely how the ensemble perspective helps to clarify the mechanism of low-dimensional latent dynamics encoding relevant spatio-temporal features of inputs.
The spike probability π tells us the sparsity level of the network. We find that a sparse RNN emerges after training. In particular, the recurrent layer is sparser than the output layer, reflecting different roles of both layers. The sparseness allows the recurrent layer to remove some unnecessary degrees of freedom in the recurrent dynamics, making recurrent feedbacks maintain only relevant information for decision making. In contrast, the output layer is averaged over ten independent runs. much denser, being thus able to read out relevant information both completely and robustly (note that most of Ξ values are zero). It is worth noticing that the π-distribution profile implies the existence of VIP weights, which we show has a critical contribution to the overall performance of the network. Our method thus provides a practical way to identify the critical elements of the network topology, which could contribute to understanding temporal credit assignment underlying the behavior of the RNN.
Working at the ensemble level, the constituent neurons of our model also display neural selectivity of different nature in response to the task parameters. Some neurons show uni-selectivity, while the others show mixed selectivity. The selectivity property of neurons, shaped by the recurrent connections, is also a key factor impacting the behavioral output of the network, and thus deserves future studies about the structure basis of the neural selectivity.
Taken together, our model and the associated training method are able to provide insights toward a precise understanding of temporal credit assignment, not only in engineering applications (e.g., MNIST digit classification), but also in modeling brain dynamics (e.g., multisensory integration task).