Supervised Contrastive Learning-Based Modulation Classification of Underwater Acoustic Communication

Modulation parameters are very significant to underwater target recognition. But influenced by the severe and time-space varying channel, most currently proposed intelligent classification networks cannot work well under these large dynamic environments. Based on supervised contrastive learning, an underwater acoustic (UWA) communication modulation classifier named UMCSCL is proposed. Firstly, the UMC-SCL uses a simply convolutional neural networks (CNN) to identify the presence of the UWA signals. Then, the UMC-SCL uses ResNet50 as an encoder and updates the network by supervised contrastive learning loss function, which can effectively use the category information and make the eigenvector distribution of the same category more concentrated. Then, the classifier uses the feature vector output by the encoder to distinguish the final modulation categories. Finally, extensive ocean, pool, and simulation experiments are done to verify the performance of the UMC-SCL. Without any prior information, the average classification accuracy for MPSK and MFSK can reach 98.6% at 0 dB and is increased by 6% compared to the benchmark algorithm under low SNR.

Introduction
With the development of UWA communication technology, more and more ocean applications have installed UWA communication equipments. Through modulation classification can explore the influence of ocean multipath and Doppler effect and more effectively assistant target identification, signal identification, interference identification, and spectrum management.
In general, conventional modulation classification algorithms can be divided into two categories: likelihood-based and feature-based methods [1]. The likelihood-based method requires a large amount of prior information and computation, which makes it unsuitable for harsh noncooperative UWA communication. On the contrary, feature-based method has gradually become the mainstream method due to its low computational complexity and no dependence on prior information.
Feature-based methods consist of two parts: feature extraction and classifier. In [2], multiscale reverse dispersion entropy and grey relational degree features are used to improve the classification performance of ship-radiated noise. In [3][4][5], support vector machine (SVM) is used to distinguish wireless signals. In [6], high order cumulant features are put into SVM based on mixture kernel function to classify the digital signals. Wei et al. [7] use a SVM based on hybrid features, cyclostationary, and information entropy to classify the modulation types, including BPSK, QPSK, 2FSK, 4FSK, and MSK. By this means, the parameter extraction process is complicated, and the capacity is low. Even if more training data is added, the classification performance cannot always be improved [8]. For recent years, deep learning [9][10][11][12][13][14][15][16] has shown excellent performance in image feature extraction, speech recognition, and natural language processing and has been successfully used on acoustic signal sets [17,18]. However, in modulation classification area, it is mainly used in the electromagnetic communication.
In [9], long-short term memory (LSTM) is used to classify the modulation schemes for a distributed wireless spectrum sensing network. Li et al. [10] use the I/Q data to classify signal directly through deep neural networks (DNN). In [11,12], adaption of deep learning to the complex temporal signal domain is studied, and first proposed a CNN-based classifier to solve the problem of excessive parameters in DNN. In [13], AlexNet and GoogLeNet are used to classify the constellation of the signal samples. Huang et al. [14] introduce a novel cascaded CNN that cascade two-block CNN to identify MPSK and MQAM hierarchically. Wang et al. [15] propose a hierarchical CNN scheme to more accurately classify the higher-order QAM signals. Liu et al. [16] combine CNN with long short-term memory (LSTM) architecture into DNN and increase the accuracy rate by 13.5% compared with original CNN. In these classic end-to-end neural networks, cross-entropy loss is the most widely used loss function to achieve the purpose of updating network weights. However, the cross-entropy loss function also lacks robustness to noisy tags [19,20] and may have marginality [21,22], leading to reduce generalization performance. The traditional end-to-end supervised training methods focus on the final classification accuracy rather than the quality of the features extracted from the UWA data. As a result, when the signal-to-noise ratio (SNR) becomes low, the accuracy of traditional methods will drop sharply and cannot work well. In recent years, the renaissance of contrastive learning has led to major advances in self-supervised performance learning [23][24][25]. When there is no available label, the data is augmented through its own cropping and flipping, and the encoder is updated through the self-supervised loss function. Although it can alleviate the disadvantages of traditional networks to a certain extent, it cannot learn from the other samples in the same category. As a result, self-supervised contrastive learning methods are not suitable for UWA data with different SNR.
In this paper, from the perspective of representation learning, we extract features with high discrimination through supervised contrastive learning [26] to support the normal classification tasks in harsh UWA channel and propose a novel classification framework named UMC-SCL. We first distinguish between valid signal and ocean noise through a simply CNN. Then, the supervised contrastive learning module will learn from the valid modulation signal and update the encoder network by supervised contrastive learning loss function. Go through this module, the features of the same category are as close as possible, and the features of different categories are as far away as possible. Therefore, we can achieve the purpose of classification only by using a fully connected layer. Finally, we verify the superiority of the proposed method through extensive ocean, pool, and simulation experiments and use principal component analysis (PCA) to visualize the output features for interpretability. Compared with the known traditional supervised networks, the proposed method greatly improves the classification accuracy under low SNR without any prior information and parameter extraction process.  hðtÞ is the energy normalized impulse response of UWA channel, sðtÞ is the original signal, and nðtÞ is the ocean noise. aðtÞ is related to SNR. Node 1, Node 2, and Node 3 communicate with each other. The listener can intercept their communication signals from the sea water. SL is emitting sound source level, TL is propagation loss, and NL is the background noise level [27].

UWA Data Sources.
In order to make the research result more applicable, we have constructed a complete data set through actual ocean experiments, pool experiments, and simulation experiments that are close to the reality.
2.2.1. Ocean Data. The ocean data are collected in Wuyuan Bay, Xiamen, China. As shown in Figure 2, the sound source T x1 and the receiving hydrophone R x1 are placed in the shallow sea near the footpath, with a depth of 5 m and a communication distance of 60 m.
We send and receive signals at four different times of the day. During the experiment, there are some activities such as yachts, fishing boats, and other activities that introduce a lot of man-made noise. Besides, dozens of plank road bridge piers between the sending and receiving ends make the reflection effect more significant.

Pool Data.
Ocean experiments are costly, and the data acquisition is difficult. In order to increase the richness of the dataset, we further conduct pool experiments. The pool is located in UAC laboratory in Xiamen University. Figure 3(a) is the photo of the pool. The pool has a length of 25 m and a width of 5 m. It is divided into deep water area (depth = 1:5 m) and shallow water area (depth = 1:15 m). Figure 3(b) is the distribution of transmitter and receivers for pool experiment. T x is the sound source, and R x is the hydrophone. The distances between R x1 , R x2 , R x3 , and T x are 3 m, 6 m, and 12 m, respectively, and the depth is 1 m. When T x sends a signal, the sound rays will be attenuated by water and reflected on the pool wall.    Figure 4 shows the sound ray propagation in a shallow sea channel. The sound ray will be reflected by the sea surface and bottom during propagation. Moreover, the speed of sound in seawater changes with temperature, salinity, and water depth, causing sound rays to be refracted. The speed of sound can be described according to the following formula [27].
where T is temperature in, S is salinity in ppm, and Z is the depth of seawater in m.
In UWA communication, the impulse response can be assessed by beam tracing for typical acoustic communication frequencies. The basic path loss of the received signal that traveling through the UWA channel is given by [28].
where A 0 is a scaling constant, l is the traveling distance of sound ray, k is the spreading factor, and α is the absorption coefficient which is closely related to the frequency of sound waves and can be obtained by Thorp's empirical formula as where the units of α and f are dB/km and kHz, respectively. The impulse response of the multipath channel can be expressed as the summary of the transfer function of each path where Γ p , τ p , and l p are, respectively, the cumulative reflection coefficient of the surface and bottom, propagation delay, and the propagation distance of the p-th path. Generally speaking, an ideal surface can be modeled by a reflection coefficient γ s = −1, while the bottom reflection can be modeled by where θ p is the grazing angle associated with the p-th propagation path and ρ and c are the nominal density and the speed of sound in water (ρ = 1000 kg/m 3 and c = 1500 m/s). ρ p and c b (calculated by Equation (1)) are the density and the speed of sound in bottom. The propagation delay of p-th path can be simple calculated as where l 0 is the direct distance from the sender to the receiver. In order to get a tractable, simple channel model, we examine an approximation to the function. Taking p = 0 as the reference path and H 0 ðf Þ as the impulse function corresponding to l 0 , the impulse function of the receiving end can be further expressed as

Supervised Contrastive Learning-Based Modulation Classification
A large number of studies have proved that DNN is superior to SVM. In the field of UWA modulation classification, the application of DNN is still scarce and all use end-to-end supervised methods. However, when the SNR becomes low, the accuracy will drop sharply. In response to this problem, we use supervised contrastive learning to narrow the feature distance between the same category and expand the distance between different categories, so as to improve the classification accuracy of modulation schemes under low SNR.
3.1. Classification System Model. As shown in Figure 5, in Step 1, the signals received by the receiver may be useful signal or useless ocean noise. In Step 2, the input signals are recognized through a simple two convolutional layers and a fully connected layer. Conv11 × 32 means the channel number is 32, and the size of convolutional kernels is 11 × 11. If the input signal is useful signal, it will be transported to supervised contrastive learning module for further classification; if it is ocean noise, it will be discarded. In Step 3, supervised contrastive learning loss function is used to update the backbone network (ResNet50) to extract features from UWA data and then put the features into classifier for classification. By this means, the influence of ocean noise can be effectively eliminated, and the classification accuracy at low SNR will be significantly improved. In the following content, the specific network architecture will be given in details.

Backbone
Network. The backbone network of supervised contrastive learning in this paper is ResNet50. ResNet50 is a residual CNN with 50 layers. It directly skips several layers and introduces the output of a certain layer into the input part of the following data layer, which overcomes the problems of low learning efficiency and ineffective improvement of accuracy due to the deepening of the network. Another two important operations in the network are batch normalization and ReLU. Batch normalization is aimed at converting the input data to an output data distribution with a variance of 1 and a mean of 0 to improve the speed of network optimization. ReLU is a nonlinear activation function. It makes the output of some neurons be 0, so as to improve the sparsity and avoid the overfitting phenomenon of the network.
In traditional supervised end-to-end CNN, as shown in Figure 6, the output of the classifier is used as the only indicator to update the network. The most widely used loss function is the cross-entropy loss function, and the expression is where N is the number of samples and M is the number of label categories. If the true category of sample i is equal to c, then y ic = 1; otherwise, y ic = 0. p ic is the predicted probability that the sample i belongs to the corresponding category.
3.3. Supervised Contrastive Learning. Supervised contrastive learning effectively utilizes the category label information, making the feature points from the same category closer than the points from different categories. Different from self-supervised learning [24], the positive samples are other samples in the same category. As shown in Figure 7, the progress is divided into two training stages. The first stage focuses on the training of the encoder and uses the supervised contrastive learning loss function to update the encoder. The second stage focuses on the training of the classifier using the feature output by the encoder and using the cross-entropy loss function to update. In self-supervision, the function of the two converters is to flip or crop the input picture so that the two newly generated images can be used as the positive samples. Due to the high complexity of UWA data, cropping or flipping the time domain signal will destroy its original characteristics. Since the label information is known, the supervised contrastive learning takes all the samples from the same class in the batch as positive samples and compares them with the negative samples in the rest of the batch. The loss function becomes where where i is the blind UWA data and z i represents the feature generated by the backbone network. z j represents the feature that comes from the same category with data i, and z k represents the feature generated by backbone network that is different from data i. τ is a scalar temperature parameter larger than 0.ỹ i is the category label of i. To update the network parameters under the constraint of the loss function, the  The classifier in the second stage is a simple fully connected layer. It uses the 2048-dimensional standardized feature output by the encoder to classify the modulation schemes. It should be mentioned that the parameters of the encoder are frozen in the second stage. Therefore, whether the encoder can obtain excellent features after training plays a decisive role.
Algorithm 1 describes the update process of the supervised contrastive learning.

Experiments and Results
In this section, the details of the experiments are explained. We also evaluate the modulation classification performance of the proposed method and compare it with the existing methods. In order to analyze the algorithm performance more intuitively, we use PCA to visualize the features to provide the interpretability of the proposed method.  Table 1 shows the parameter setting of different modulation schemes. The ocean noise is actually collected in the Wuyuan Bay sea area. After passing through the ocean channel, the pool channel, and the simulation channel, the data with the characteristics of multipath fading and Doppler frequency shift is obtained. On this basis, Gaussian white noise with different SNR is superimposed on the obtained data through MATLAB. In this paper, the intraband SNR is used to evaluate the performance of the proposed algorithm. It can be calculated as Step 1 Step 3 Step 2

Wireless Communications and Mobile Computing
where F s is the sampling frequency and B s is the bandwidth of the signal.
In Step 2, it is aimed at distinguishing the ocean noise and the useful signals. The train set consists of 2,000 ocean noise samples and 2,000 modulation signal samples with different SNR. The corresponding test set is 800 samples per category. In Step 3, the training set of supervised contrastive learning consists the data with different SNR after noise pollution. Among them, 550 samples of each modulated signal are generated from -9 dB to 9 dB every 2 dB, 250 samples of which are used as the training set and 300 samples are used as the test set. Therefore, the training set contains 15,000 samples with different SNR, and the test set of each SNR contains 1,800 samples.

Experimental Implements.
In the ocean and pool experiments, NI USB-6259 Pinout capture card is used to convert the digital signal to analog signal at the transmitter and convert the analog signal to digital signal at the receiver. JYH500A power amplifier and Type-2692-0S2 charge amplifier are used to amplify the transmitted signal and the received signal, respectively. WBT22-1107 transducer which can convert the analog electrical signal to acoustic signal is used to send and receive signal in the water. Besides, the experiments are performed on computing server equipped with an Intel(R) Core(TM) i7-9700K 3.6GHz CPU, a NVI-DIA GeForce RTX 2060 SUPER GPU, "Pytorch" and "Python" programming language, the CUDA 10.1 and CUNDD software. The optimizer of ResNet50 is "Adam," and the learning rate is 0.05 and decays to 10% of the original learning rate every 30 epochs.

Simulation Results.
The simulation experiment is carried out under the simulation UWA channel. In the noise distinction stage, the distinction between ocean noise and useful signals is obvious, especially in the frequency domain. Even when the SNR is -6 dB, the classification accuracy can still achieve 100%. Therefore, it can be explained that the simple convolutional network of Step 2 can well eliminate the influence of marine noise. In the Step 3, Figure 8 gives the classification accuracy of six modulation schemes. In general, the classification accuracy of six modulation signals increases with the increase of SNR and can achieve an average accuracy of 98.84% at 0 dB. When the SNR decreases to -6 dB, the recognition of 8PSK is the most difficult, and the confusion of modulation categories is mainly concentrated on QPSK and 8PSK.

Actual Ocean and Pool Experiment
Results. Due to the difficulty and high cost of obtaining ocean data, in practical experiments, we mix pool data with the ocean data to increase the richness of training set, so that the trained encoder and classifier can better fit the distribution characteristics of UWA data. The result of Step 2 in practical experiments is the same as mentioned in the previous simulation part. In Step 3, using the feature output by the encoder, the classification accuracy of the single fully connected layer is shown in Figure 9. For MPSK, its information is modulated in phase, so its characteristics in the time domain are not as obvious as MFSK. When the SNR is -6 dB, the average accuracy of MPSK is 79.7%, while MFSK can achieve a high   Classification performance results of six modulation categories at -6 dB and 0 dB are presented using confusion matrix in Figure 10. In each modulation category, 300 tests are implemented. When SNR is -6 dB, BPSK, 2FSK, 4FSK, and 8FSK have achieved high classification accuracy through supervised contrastive learning. However, since QPSK and 8PSK are relatively similar in modulation phase, they are easy to be confused. There are 102 QPSK samples that are mistaken for 8PSK and 70 8PSK samples that are mistaken for QPSK. When the SNR reaches 0 dB, except for the slightly larger classification error of QPSK, the recognition accuracy of other modulation schemes almost reaches 100%.

Accuracy Comparison.
To verify the superiority of the proposed method in this paper, the performance is investigated by making comparisons with four relevant algorithms in recent years; the comparison algorithms are as follows: (1) Algorithm 1 based on ResNet50 using constellation density as feature [29] (2) Algorithm 2 based on AlexNet using 3-channel image as feature [13] (3) Algorithm 3 based on VGGNet using original gray image as feature [30] (4) Algorithm 4 based on SE-Net using the features in time domain, frequency domain, and timefrequency domain [31] Figure 11 presents the average classification accuracy of five algorithms versus SNR. The average accuracy is obtained by averaging the classification performance of six modulation categories. As shown in Figure 11, the following observations can be made.
(1) For all five algorithms, the modulation classification performance improves with an increasing SNR value (2) Given the same SNR, in addition to the proposed algorithm, the other four algorithms will have a sharp decay on the classification accuracy when the SNR becomes low (3) The proposed supervised contrastive learning algorithm has strong adaptability to low SNR UWA modulation signals and outperform all other algorithms. When the SNR is -6 dB, the accuracy of our proposed method is 6% higher than the benchmark algorithm [29] 4.3.4. PCA for Interpretability. PCA can reduce a set of n -dimensional vectors to k-dimension through orthogonal transformation. That is, k unit orthogonal basis is selected, so that the original n-dimensional data is represented by this group of basis. For high-dimensional data, first make the mean of the input vector to 0 and then use the covariance to represent the correlation between vectors a and b. The covariance is calculated as For mn-dimensional vectors fa 1 , a 2 , ⋯a m g, the matrix X is composed of

Wireless Communications and Mobile Computing
The covariance matrix C is It can be seen that the diagonal of the matrix C is the variance of the vectors, and the other elements are the covariances between different vectors. Supposing Y = PX is the vector of the original data X projected to the low-dimensional space, P is the transformation matrix, and D is the covariance matrix of Y, there is the following equation In order to enable the transformed low-dimensional vectors to represent more original information, we hope that they are not correlated with each other; that is, the covariance is equal to 0. Therefore, the matrix D should be a diagonal matrix. According to the relevant knowledge of linear algebra, the matrix P should be the eigenvector matrix of matrix C, and it should be arranged from top to bottom according to the size of the corresponding eigenvalues. Select the matrix P k composed of the first k rows of matrix P, and obtain a matrix Y k with k-dimensional vectors. Taking k = 3,  Wireless Communications and Mobile Computing the high-dimensional feature outputs by the network are presented in a 3-dimensional plane. Figure 12 shows a 3dimensional space cross-sectional view of the feature point distributions extracted by different networks. It is easy to see that the features extracted by the supervised contrastive learning method have a higher degree of discrimination and better classification effect under low SNR. When SNR is -6 dB, the features extracted by ResNet50 [29] are overlapped. In contrast, the features extracted by the proposed method, except that the features of QPSK, 8PSK, and 8FSK, have some overlap; the feature distributions of the other three modulation signals are concentrated and easy to distinguish. What is more, with the increase of the SNR, the feature point distribution boundaries of different modulation schemes become clearer and clearer.

Conclusion
In this paper, we are the first to propose a novel modulation classification scheme based on supervised contrastive learning. Firstly, the useful signals and ocean noise will be distinguished in the first module. Secondly, the encoder ResNet50 in the supervised contrastive learning module will learn the input UWA data under the guidance of the supervised contrastive learning loss function to update the network. By this means, the distance between feature vectors in the same category but with different SNR will be minimized, and the distance between feature vectors of different categories will be expanded as much as possible. Then, the classifier recognizes the modulation scheme according to the feature output by the encoder. Finally, the ocean, pool, and simulation experimental results verify the superiority of the proposed method. Compared with the existing researches, the experimental verification in this paper is more complete. The proposed method eliminates the complex parameter extraction process and does not require any prior information. When the SNR is 0 dB, the average accuracy can achieve 98.6%. Com-pared to the benchmark algorithm, the accuracy at -6 dB is improved by 6%. Moreover, we use PCA to visualize the feature distribution, which can intuitively analyze the superiority of the proposed algorithm.

Data Availability
The data used to support the findings of this study were supplied by Daqing Gao under license and so cannot be made freely available. Requests for access to these data should be made to Daqing Gao (dqgao@stu.xmu.edu.cn).

Conflicts of Interest
The authors declare that they have no conflicts of interest.


s2fieldsofstudy:
Computer Science
Education

text:
Construction of College Physical Education MOOCS Teaching Model Based on Fuzzy Decision Tree Algorithm

With the continuous development of the MOOCSmodel in college physical education, the corresponding teaching evaluation has also been widely concerned by the community. e development of a traditional education mode in college physical education cannot meet the current teaching requirements. In order to solve the problem of narrow application and insucient accuracy in traditional education, on the basis of the Kohonen fuzzy decision tree algorithm and the MOOCS fuzzy decision tree algorithm, a fuzzy evaluation model of P.E. teaching is proposed. e results show that the fuzzy ID3 algorithm can achieve high accuracy in the four databases, and the classication accuracy of the A-D database is 75.9%, 62.9%, 76.6%, and 95.1%, respectively. Except for database C, the classication accuracy of the minimum classication uncertainty algorithm is lower than the other two clear decision tree algorithms. Compared with the minimum classication uncertainty algorithm, the fuzzy ID3 algorithm has obvious advantages in classication rules. e classication rules of A-D database are 18, 12, 16, and 10, respectively. When the authenticity threshold is about 0.8, the fuzzy ID3 algorithm has the highest classication accuracy.e proposed MOOCSmodel for college physical education based on the fuzzy decision tree algorithm has strong practicability and high accuracy. is paper studies the MOOCSmodel of college physical education and introduces the fuzzy decision tree algorithm to evaluate the MOOCS model of college physical education. It solves the problem that traditional sports cannot satisfy the needs of “Internet plus education” at all. Compared with the traditional sports model, it has better applicability and higher accuracy.

Introduction
Massive open online course (MOOCS) has become a current online education teaching curriculum mode. It provides diverse and personalized courses to the learning group through the characteristics of convenience and technological innovation. It can help learners get comprehensive and wide information and resources [1]. With the rise of three-course system, udacity and coursera, the goal of students' systematic learning has certain feasibility. Although the online education platform can meet the needs of all kinds of people, the current online education evaluation method still needs further exploration [2]. It is worth noting that decision tree classi cation has become a very important evaluation classi cation method, and it is still e ective for the uncertain classi cation problem decision tree algorithm.
is fuzzy decision tree has achieved certain research results in many industries such as medical, educational, and energy [2]. In view of the fact that traditional physical education has completely failed to meet the needs of "Internet plus education," a MOOCS model of college physical education is studied and a fuzzy decision tree algorithm is introduced to evaluate the MOOCS mode of physical education in colleges and universities, aiming at providing suggestions for the future development of MOOCS mode in college physical education.
In order to realize intelligent transportation in developed cities, Balta et al. proposed to optimize tra c signals of the SDN (software-de ned network) city by using a three-level fuzzy decision tree. e performance test results show that the three-stage fuzzy decision model has better performance than xed time signaling and webster equation [3]. Nancy et al.
used a fuzzy decision tree and dynamic feature selection to detect the intrusion of wireless sensor networks. e results show that the method can effectively reduce the false positive rate, energy consumption, and delay. At the same time, the system can improve network performance by improving the packet delivery rate. Syryamkin et al. proposed a fuzzy rule set automatic generation algorithm for the fuzzy decision tree and analyzed its performance with the network parameter adaptive hybrid algorithm. e technical efficiency results show that the former has higher performance [4]. Nalinipriya et al. picked up a priority-oriented scheduling algorithm, an enhanced priority scheduling algorithm, which uses job priority measures to generate the fuzzy decision tree. Compared with other scheduling methods, the method improves the execution time and shows better performance [5]. In order to study a general model suitable for prediction and risk, Zhang et al. proposed a generalized membership function model of fuzzy set in fuzzy decision tree and extended it to the fuzzy random forest method. e test results show that the model is practical and effective [6]. Teekaraman et al. pointed out that the fuzzy set and decision tree were used to classify the credibility group of publishers of Book Social Network. e model pruning rules were authenticity, and the evaluation indexes such as classification accuracy, precision, recall rate, rule generation number, and time complexity showed that the proposed learning model had better performance [7].
Ghobaei-Arani et al. proposed a large infrastructure which can support the peak load of the game in order to ensure the QoS requirements of highly variable concurrent players. e workload predicted by the fuzzy decision tree algorithm and user SLA (service level agreement) is used to estimate the appropriate resources allocated to each layer. e test shows that the performance and accuracy of the method are better [8]. Das and Padhy proposed to use fuzzy decision tree to forecast commodity future index in view of the difficult situation of financial time series data analysis and prediction. e experimental results show that the model is very accurate and has high feasibility [9]. Zhai et al. have successfully completed the motivational factors of MOOCS learners in nanotechnology and nano sensors. e results show that the three motivations are occupation, individual, and education, and the college-affiliated students and ordinary participants are all due to common interests and personal growth needs [1]. Lim et al. proposed a semantic network model to measure the different lexical association between teachers and students and the degree of students' participation in MOOCS. According to the data score of the MOOCS forum, the degree of students' participation in MOOCS and students' learning situation showed a positive correlation [10]. Xing et al. comprehensively evaluated some students of computer basic courses for MOOCS by means of investigation and analysis. e test results show that the teaching quality and learning effect have been improved obviously [11].
In conclusion, the fuzzy decision tree algorithm has been widely used in the teaching mode, and the evaluation results are well received by most users. Most scholars have very mature views on the fuzzy decision tree algorithm and the MOOCS model of physical education in colleges and universities. At present, the research data of the MOOCS model of physical education in colleges and universities by using the fuzzy decision tree algorithm are relatively small, and it has not achieved a relatively ideal evaluation and recommendation effect. In view of this, this study proposes a MOOCS model for college physical education based on the fuzzy decision tree algorithm, aiming to make contributions to the future.

MOOCS Model of College Physical Education
Online and offline hybrid teaching can integrate the group learning advantages of the offline class teaching system with personalized e-learning. It is an effective carrier for the indepth integration of information technology and education and teaching. is teaching model needs to comprehensively use different learning theories, technical means, and application methods, which puts forward higher requirements for teachers' teaching resource construction, teaching design, and teaching management. Among them, instructional design is the key factor for the success and effectiveness of online and offline hybrid teaching. e integration mode of "MOOCS" and school physical education is shown in Figure 1. In view of the characteristics of school sports technology teaching, MOOCS cannot completely replace traditional school sports teaching. With the advantages of computer technology and Internet platform, it optimizes and integrates the functions of Internet in college sports resources so as to realize the indepth integration of MOOCS teaching and school sports teaching. MOOCS and college physical education integration is a hybrid teaching form, which achieves the online and offline teaching chain mode. e online and offline centers are questions and tasks, respectively, which are embodied in question answering, communication, learning, evaluation, feedback, and interaction. e MOOCS teaching platform of college physical education can not only complete the evaluation of learning enthusiasm, answering difficult questions, skill video teaching, and online sports knowledge but also ensure that students can actively carry out classroom communication, which greatly reduces the teaching time of classroom teachers and makes students become the real main body of the classroom. Teachers need to design rich and colorful course videos in advance, prepare online interactive Q and A, record students' online learning achievements, carry out formative evaluation, master different students' mastery of skills, and try their best to make students have a certain learning ability of motor skills [12]. For students, in addition to self-learning motor skills and watching videos, they need to participate in offline learning and actively participate in interaction and sharing. e schematic diagram of MOOCS integration into the school physical education teaching process is shown in Figure 2, covering the production and design of MOOCS video, autonomous online learning, teachers' offline teaching, knowledge consolidation, and practice after class. e course video needs to conform to the theory of cognitive load, reduce the frequency of interference information, and emphasize the relevant content with markers when it appears so as to improve the students' effective learning time. e best video time is about 6 minutes. e time and place of online autonomous learning should be determined before the next face-to-face teaching. First of all, the food and study are watched, and the questions and related test items set by teachers in advance are completed. On the basis of this, we can have a happy discussion with students. e key step of online classroom teaching is the task-driven method, which can stimulate learners' interest and give full play to students' main role and teachers' leading role. Combined with online and offline mixed teaching, students can truly realize the automation of motor skills and learn to repeat and practice after class.

Fuzzy Decision Tree Algorithm.
Decision tree is a widely used machine learning algorithm. Finding the optimal classification features and corresponding classification eigenvalues in the dataset is the key to solve the problem of decision tree, and the division is based on information entropy and information gain. At present, the common decision tree algorithms are CLS, ID3, and C4.5 (classification and regression tree, cart). ID algorithm is a greedy search method from top to bottom in decision tree space. Firstly, the algorithm calculates the information gain of each attribute. By comparing the information gain, it selects the attribute with the maximum value as the splitting node and then forms a branch with the splitting node according to different attribute values. en, it calculates the information increment of the remaining attributes for each branch according to the above method, calculates and selects the attribute with the maximum value as the splitting node, and then according to each branch, it calculates the information increment of the remaining attributes, generate another branch with different attribute values, and repeat the above steps until the decision tree that can classify training samples is generated. e following is the specific method. For information entropy, it also becomes a classification system, assuming that it is a set of data samples; x is the number of X, C is the attribute category variable, the value is C � C 1 , C 2 , C 3 , ... C n , n is the total number of categories, and x i is the number of samples contained in C i . en, the probability of sample belonging to C i is p i � x i /x, and the information entropy of   sample classification is I( . . X n , and x ij is the number of samples belonging to C i in the subset X j ; then, the information entropy of Y is e information gain of attribute Y is as follows: Because the ID3 algorithm is complex, it takes up the running memory of the machine and wastes resources and reduces the operation efficiency, this paper proposes to optimize the information entropy formula by using the power expansion of the function, that is, to transform the calculation formula of information entropy into an approximate calculation formula containing the addition, subtraction, multiplication, and division algorithm [13]. e improved process is as follows. According to the demand, p i can be set as the following equation: Further processing formula (3) gives the following formula: (4) e higher the power of the exponent, the higher the accuracy of the formula, because in the actual operation, only the size is compared, and the approximate calculation formula of the value is as follows: In the above formula, 8/3 ln 2 is a constant, which can be ignored in the calculation of comparison size. On this basis, the decision tree is created and generated. For continuous value partition, the expression of information gain can be extended to the equation as follows: In the above equation, Ent (D) is the information entropy. In the research, because the complex multisegment partition can achieve better fitting results, which increases the training cost and prediction cost, multiple function combination values are used to partition. Fuzzy set theory refers to the theory of using the concept of membership function and fuzzy set theory. e fuzzy set on the universe can be expressed as follows: In the above equation, the membership function of the fuzzy set is μ A (x), and the value range is [0, 1], which represents the membership of any element x ∈ X to x ∈ X. When μ A (x) is 0, the element x does not belong to the fuzzy set A. When μ A (x) is 1, the element x belongs to the fuzzy set A. ere is no clear membership relationship between the set and the element, which is expressed by the membership function. Intersection operation and union operation refer to membership functions of intersection and union of fuzzy subsets A and B, respectively, which are shown in the equation as follows: e related definitions of fuzzy decision tree are as follows: the finite set is represented by U, the set of fuzzy subsets is represented by F(U), in which the fuzzy subset A can be represented by μ Second, a set of fuzzy subsets Γ, then Γ � Ω i ∩ B, and 1 ≤ i ≤ m are composed of all the children B of nonleaf nodes. Each leaf node of a directed tree has one or more classification decision values. All fuzzy subsets correspond to one attribute, and the construction process of fuzzy decision tree is shown in Figure 3.
In the process of constructing fuzzy decision tree, we need to consider the authenticity threshold β 0 and significance level α. Under the significance level, the fuzzy set can be expressed as follows: It can be seen that the higher the α value, the less fuzzy the data training will be, but the larger the value of the training set will become an empty set. Generally, the value range is (0, 0.5]. If the training results meet the requirements, the data can be fuzziness without α. Otherwise, α is needed to improve the training effect of the model. β 0 is an indicator used to evaluate whether to stop dividing the nodes. If the node authenticity is greater than β 0 , it indicates that the node partition can be stopped [14]. Generally, β 0 is greater than 0.75. e reality degree of a node represents the maximum value of the example in the node in the membership degree of each category, and the expression of the example in the node for the membership degree of the category is as follows:

MOOCS Model of College Physical Education Based on
Fuzzy Decision Tree Algorithm. e common fuzzy decision tree algorithms are the fuzzy ID3 algorithm and the minimum classification uncertainty algorithm. e segmentation attribute criteria of the former are fuzzy information gain and fuzzy information entropy. Taking the sample set X � x 1 , x 2 , . . . , x N as an example, the attribute l of the nonleaf node B can be represented by A (1) , A (2) , . . . , A (l) , in which each fuzzy condition attribute has k s fuzzy semantic value e fuzzy classification entropy of each attribute value T (s) i in nonleaf nodes can be expressed by formula e average fuzzy classification entropy of each attribute on nonleaf nodes can be expressed by In above formula, the weight of the second attribute of the i attribute is denoted by ω i . e minimum classification uncertainty algorithm divides the attributes into minimum classification uncertainty. e former is chosen as the classification of the MOOCS model applied in college physical education. In order to construct the MOOCS model of college chysical education teaching based on the fuzzy decision tree algorithm, firstly, we need to calculate k center points by the Kohonen feature mapping algorithm and discretize the continuous attributes by fuzzy semantic value. en, the triangular membership function is used to fuzzify. e parameter of triangular membership function is the neutral point of attribute. e membership function formula of each semantic value is shown in the equation as follows: When the values of k are 2 and 3, the corresponding membership functions are shown in Figures 4(a) and 4(b). e evaluation of the MOOCS model in college physical education teaching is based on students' learning behavior, including teaching resources browsing, forum data, online notes, and online test. According to the final test times and comprehensive score, it can be divided into four levels: A, B, C, and D. e behavior description of the students is shown in Table 1. In view of the attributes that affect the effectiveness of the model in the integrated information, it is necessary to reduce the attributes. Finally, the input data are student number and four kinds of final learning behavior data.
Finally, the evaluation structure of the MOOCS mode of college physical education teaching oriented to fuzzy decision tree is shown in Figure 5, and the classification model represents a series of if-then rules [15]. If a new data is introduced, the root node testScore (A4) is used to judge the attribute and content of the test data, and then browsertime (A3), onlinenotes (A2) and bbspost (A1) are used to get the final evaluation result. e learning behavior of the educated is a series of vectors, and finally, four scoring rules are obtained. e evaluation criteria of the classification model are classification accuracy and the scale of decision tree model [16].

Evaluation and Analysis of MOOCS Model in College Physical Education
In the experiment, the rules extracted from the fuzzy decision tree are predicted, and the data examples are shown in Table 2. e membership degrees of A11 and A12 are 0.2 and 0.8, respectively, the membership degrees of A21, A22, A23, and A24 are 0.0, 0.7, 0.1, and 0.2, respectively, the membership degrees of A31 and A32 are 0.1 and 0.9, respectively, and the membership degrees of A41, A42, and A43 are 0.0, 0.7, and 0.3, respectively. e matching result of instance and rule is shown in Figure 6. e two rules of instance classification B are rule 2 and rule 5, respectively. e maximum matching membership is 0.3, and the authenticity of the corresponding rule is 89%. ere is only one rule in the other case classification. e matching membership and rule true degree of A are 0.0 and 86%, respectively, the matching membership and rule true degree of C are 0.1 and 89%, respectively [17], and the matching membership and rule true degree of D are 0.2 and 88%, respectively.
In the experiment, the database table of the corresponding system of university A entity is selected to evaluate the classification performance of the MOOCS mode. e database includes three parts: comprehensive scoring, data integration management, and query statistics. When the learning rate is set to 0.5 and the number of center points is 2 and 3, respectively, the center points of the four attributes of the dataset are shown in Figure 7. It can be seen that there is a big difference between the four major attributes under different numbers of center points [18]. e maximum and minimum number of center points of browsertime are 7.1 and 9.58, respectively, the maximum and minimum number of center points of onlinenotes are 0.1 and 43.75, respectively, the maximum and minimum number of center points of bbspost are 25.37 and 43.85, respectively, and the   maximum and minimum number of center points of test-Score are 0.57 and 43.75, respectively. e fuzzy results of the university data set are shown in Figure 8. ere are great differences between browsertime's fuzzy long and short attribute membership degrees, the biggest differences are 0.997 and 0.006. ere are great differences between onlinenotes's fuzzy long and short attribute membership degrees, the biggest differences are 0 and 1, and there are seven groups of data. ere is a big difference in the membership degree of bbspost's fuzzy long short attribute, the biggest difference is 0 and 1, and there are three groups of data. ere is a big difference between the membership degree of fuzzy long and short attributes of testScore, the biggest difference is 0 and 1, and there are 8 groups of data.
e experiment further selects the system database table of the other three schools B-D for the MOOCS mode classification evaluation. e algorithms used are C4.5, cart, fuzzy ID3, and minimum classification uncertainty. e classification accuracy is shown in Figure 9. e classification model constructed by the fuzzy ID3 algorithm can get   Mathematical Problems in Engineering the highest accuracy among the four databases. e classification accuracy of A-D database is 75.9%, 62.9%, 76.6%, and 95.1%, respectively. erefore, the algorithm has strong practicability. e accuracy of the minimum classification uncertainty algorithm is far lower than that of the fuzzy ID3 algorithm, and the classification accuracy of A, B, and D databases is lower than C4.5 and cart. e least classification uncertainty algorithm does not get higher classification accuracy than the clear decision tree, which may be related to the probability of the distribution of the research results.
C4.5 and cart have an average classification accuracy of 74%, which can process data continuously and have better classification accuracy. e accuracy of database D classification is higher than that of the other three groups of databases, and the four algorithms have obtained more than 90% of the data in the database, which shows that there are significant differences among different individuals in the database. e accuracy of database B classification is about 60%, which shows that there is no difference between different individuals in the database.    Figure 10. e number of rules is closely related to the efficiency of decision tree. Too many rules can easily lead to over fitting. Compared with the minimum classification uncertainty algorithm, the classification rules of the fuzzy ID3 algorithm have obvious advantages. Except for database A, the rules generated by the fuzzy ID3 algorithm are less, which shows that the fuzzy ID3 algorithm has higher applicability. e cart algorithm generates the least number of rules, and the number of classification rules of database A-D is 3, 3, 7, and 8, respectively. Moreover, the algorithm is quite different from other algorithms, which is closely related to the principle of the algorithm. e experiment takes the database D with a strong classification effect as an example to explore the influence of authenticity threshold on classification accuracy and classification rules. e results are shown in Figures 11(a) and 11(b), respectively. With the increase of authenticity threshold, the classification accuracy first increases and then gradually decreases, and the maximum classification accuracy is about 0.8. e number of classification rules of the two algorithms increases with the increase of the authenticity threshold. erefore, it is necessary to select the appropriate authenticity threshold in the construction of the MOOCS classification evaluation model.
At the end of the experiment, the MOOCS model of physical education teaching in colleges and universities was evaluated and analyzed, including online theoretical assessment, offline skills assessment, platform learning, students' learning ability, progress, and growth. e letter E-I was used to refer to each aspect. e full score of each aspect was 10 points, and the evaluators were 10 professional MOOCS researchers. e overall situation is shown in Figure 12. It can be seen that all the 10 evaluators have a high evaluation on the five aspects of the MOOCS mode of college physical education, the score of each item is about 9 points, the total score of five items is more than 80 points, and the total score of platform learning is the highest, with a value of 90 points.

Conclusion
e classification and evaluation of MOOCS mode in college physical education teaching is a topic of common concern of many scholars and experts in the field of education. In view of the problems existing in the traditional teaching evaluation, this paper puts forward a classification evaluation model of the MOOCS mode of college physical education oriented to the fuzzy decision tree algorithm, constructs the fuzzy ID3 algorithm combined with fuzzy theory and decision tree algorithm, and applies it to the classification evaluation of the MOOCS mode of college physical education. e classification accuracy of the minimum classification uncertainty algorithm in the four databases is lower than that of the fuzzy ID3 algorithm, and the fuzzy ID3 algorithm has higher practicability than the minimum classification uncertainty algorithm and the other two clear decision tree algorithms. e average classification accuracy of C4.5 and cart is about 74%. In the four databases, the cart algorithm generates the least number of rules, and the classification rules of the A-D database are 3, 3, 7, and 8, respectively. Except that database A generates fewer rules, the fuzzy ID3 algorithm has obvious advantages over the minimum classification uncertainty algorithm. e number of classification rules of the fuzzy ID3 algorithm is positively related to the authenticity threshold, and the classification accuracy is obtained when the authenticity threshold is about 0.8. In view of the fact that the cart algorithm has a small number of classification rules, it can be combined with other methods to improve the accuracy of MOOCS classification evaluation in college physical education. However, this paper does not discuss the classification accuracy of the minimum classification uncertainty algorithm in the database. e minimum classification uncertainty algorithm has obvious advantages and is not obvious enough, which makes the research need to be further improved.

Data Availability
e data used to support the findings of this study are included within the article.


AI in Asset Management and Rebellion Research

On October 30th, 2021, Rebellion Research's CEO announced in a Q3 2021 Letter to Investors that Rebellion's AI Global Equity strategy returned +6.8% gross for the first three quarters of 2021."It's no surprise", Alex told us,"Our Machine Learning global strategy has a history of outperforming the S&P 500 for 14 years". In 2021, Rebellion's brokerage accounts can be opened in over 70 countries, and Rebellion's research covers over 50 countries. Besides being an AI asset management company, Rebellion also defines itself as a top-tier, global machine learning think tank. Alex planned to build a Rebellion ML&AI ecosystem. Should Rebellion stay in the asset management area or jump into other areas? How could the Rebellion strategically move towards a more broad area? What were Rebellion's new or alternative business models?

2007, based on Bayesian machine learning, successfully predicted the 2008 stock market crash and gave a rating of F on Greek bonds in September 2009. At that time, Fitch's rating was still A, and Rebellion downgraded Greek bonds one month ahead of the official.
In recent years, Rebellion has also begun to gradually apply more technologies to asset management, such as Natural Language Processing (NLP) based market sentiment analysis, 13F data and ML (also known as fundamental quantitative analysis), bitcoin and ML, Reinforcement Learning (RL) for Portfolio Allocation, Machine Learning Portfolio on 13F. "However, ML is our heart", Alex said, "In addition, RL is more useful than Deep Learning (DL) for us".

AI Technologies in Asset Management
Background Turing (1950) proposed the well-known Turing Test, which set off the first wave of artificial intelligence (Turing and Haugeland, 1950). In 1956, at the summer academic seminar held at Dartmouth University, McCarthy, Minsky, and Shanno formally gave the term Artificial Intelligence (AI) (McCarthy et al., 2006). However, due to the limitations of computer computing power and symbolism theory, the popularity of artificial intelligence faded rapidly.
In the 1980s, artificial intelligence rose again, and the breakthrough originated from abandoning symbolism and using statistical thinking instead. In 1991, IBM Deep Blue defeated chess player Kasparov, which set off the second wave of artificial intelligence. However, artificial intelligence at that time was limited by data volume and test environment and did not have much practical value.
The third wave of artificial intelligence originated from the concept of deep learning put forward by (Hinton, Osindero, & Teh, 2006. At this time, the development of the Internet industry has formed massive amounts of data, and the innovation of the graphics processing unit (GPU) has dramatically improved the computing power of computers. With the maturity of these two conditions, artificial intelligence has achieved rapid development. Taking Google AlphaGO's victory over Go player Lee Sedol in 2016 as a landmark event, the concept of AI has been deeply rooted in people's hearts, the theory has become practice, and scientific research achievement has gone to the market.
Similar to the three waves of artificial intelligence, the application of information technology in the financial field could be divided into three stages. Since the 1950s, technology has gradually entered the e-finance period. The development of information systems had promoted the electronization and automation of financial services, improved the service capacity of outlets, and reduced the transaction costs of deposits, loans and remittances. The main commercial applications were credit cards, ATMs, POS machines, CRM systems, etc. In the 1990s, with the generation of massive user data, the scope and depth of financial services were greatly extended.
In addition, the enhancement in cyber security transformed financial service mode, resulting in online banking and mobile payment. With the emergence of new technologies since 2016, mainly AI, Blockchain, Cloud Computing, Big Data, the intelligence and personalization of the financial industry was further promoted. Information asymmetry and transaction costs were reduced significantly. The most affected area was likely asset management and fundamental analysis was considered to be the cornerstone of asset management (Bartram, Branke, & Motahari, 2020). Mittermayer (2004) proposed NewsCATS, a system that classified news and did trading based on the support vector machine (SVM). Choudhury et al. (2008) used text information for communication on Engadget to predict stock price. Results showed that the model could predict the stock price trend of blog-related stocks in the future effectively (the accuracy rate is 78% to 87%). Schumaker and Chen (2009) proposed an SVM-based method for financial news article analytics. It was found that the prediction results of the model including both the terms of the article and the stock price at the publication time were close to the real future stock price (with an MSE of 0.043), the price movement trend was the same as the real future price (with an accuracy of 57.1%), and the maximum return (2.06%). The Arizona Financial Text System (AZFinText), based on machine learning, was created by using a comprehensive method of language, finance, and statistical technology to deal with the problem of discrete prediction of stock prices. The trading return rate of this system was 8.50%, which was 2% higher than the best performing quantitative fund (Schumaker & Chen, 2009a). Li (2010) used Naive Bayes to verify the relationship between the information in the company's public document and the company's stock price, and the results showed the average tone of forward-looking statements had a positive correlation with future earnings. Groth and Muntermann (2011) used news reports and stock price data to determine that the disclosures provided in text data led to the most extraordinary risk exposures. Compared to the nearest neighbor (KNN), neural network (NNnet), naive bayes, the results showed that SVM had the highest AUC (0.642) and accuracy (78.96%). Hagenau et al. (2012) used market feedback as a part of word selection and constructed an SVM model with an accuracy of 65.1% for automatically predicting stock prices using unstructured text information. Li et al. (2014) used the eMAQT model based on the SVM to confirm the relationship between investor behavior and news information. Bogle and Potter (2015) built a stock price prediction model SentAMaL based on ML and emotion analysis and studied the mixed prediction model of emotion expressed on Twitter and machine learning to predict the US index. The decision tree model reached 100% accuracy, and the performance of the decision tree 5 model in this task was better than the SVM and artificial neural network. Piñeiro-Chousa (2017) uses the Logit model to analyze the performance of investors on social media and its influence of text information on the market. The results showed that the combination of different factors did have an impact on the market, which depended on the type of investors. Bollen et al. (2011) conducted the Granger causality test and the self-organizing fuzzy neural network, and found that specific public emotion could significantly improve the prediction and found that daily news could only predict stock returns for days of one or two. And positive news would increase stock returns rapidly, while negative news had a long-term reflection period. Most of the response delayed to news occurred near the announcement of subsequent earnings. Kar et al. (2017) used support vector regression and long short-term memory network (LSTM) to establish the application task of target-dependent fine-grained text emotion prediction in the financial field. Ghosal et al. (2017) proposed a hybrid model based on convolution neural network (CNN), LSTM, vector averaging, and feature-driven multilayer perceptron to improve the accuracy of target-dependent text emotion analysis tasks in the field of stock investment. Zhang et al. (2017) proposed a predictable method for the rise and fall of stocks. In the aspect of feature extraction, besides stock price information, it also extracted emotional features related to 6 stocks based on news extraction. It was found that compared with the SVM classifier based on price features, the proposed method has an improvement of more than 5% in the stock rise and fall prediction. Lee and Soo (2018) proposed a recursive convolution neural network that combines the advantages of convolution, sequence model, and knowledge extraction, and can capture information from the trained convolution filter by embedding pre-trained words in the corpus. The results showed that the recurrent convolutional neural network (RCN) model combined with technical analysis indicators predicts that four of the five targets have lower RMSE (TSMC: 28%, Grape Gold: 38%, CSC: 30%, Taiwan Stock Exchange: down 53%). Tan et al. (2019) proposed the application of the tensor-based eLSTM model in financial news, using tensor instead of cascade vector to model market information, and using the event-driven mechanism to balance the heterogeneity of different data types in LSTM. Tan et al. (2019) conducted experiments on the annual data of China's securities market, which showed that the eLSTM method was superior to existing algorithms such as AZfinText, eMAQT, and TeSIA. Araci (2019) introduced the state-of-the-art BERT into the finance area, and the proposed FinBERT outperformed ML models. Moody et al. (1998) put the recursive reinforcement learning algorithm model (RRL) and in the tests of the S&P 500 Index and US stocks, the reinforcement learning model with Sterling ratio as the objective function has the highest return. Moody and Saffell (1999) applied RRL to the fields of single stock and asset portfolio. They tested the intraday foreign exchange market (USD/GBP), S&P 500, short-term US treasury bonds, and other financial assets. With the yield as input and the differential Sharpe ratio as the objective function, the experiment was carried out under the condition of transaction cost of 5‰. The return of RRL strategy was higher than Q-learning strategy and buy-hold strategy, and the transaction number was lower than Q-learning strategy. Moody et al. (2001) added the short-position wait-and-see action Ft ∈ {-1, 0, 1} on the basis of RRL, and Ft=0 means suspending trading and reducing risks within a certain period of time. In addition, the downside deviation ratio was used instead of the Sharpe ratio as the objective function to test the return status of the model when the market goes down. This was the first time that RRL was applied to foreign exchange high-frequency trading between sterling and US dollar. The comparison between RRL and Q learning shows that RRL was superior to Q learning strategy in many aspects, which also proves that RRL was more suitable for high-frequency trading. Gold (2003) proposed to replace the single-layer neural network with the multi-layer neural network in the RRL model. Gold tested it in 25 different high-frequency foreign exchange markets. The test results showed that both single-layer RRL and multi-layer RRL could achieve profitability, and the performance of multi-layer RRL was worse than that of the single layer. Gorse (2011)

Summary of AI Technologies in Asset Management
In this session, we discussed the frontiers of the AI technologies (ML, DL, RL) that Rebellion Research mainly applied. We can find that the usage of AI in asset management, such 8 as portfolio management, trading, portfolio risk management, is an emerging field that attracts both academia and industry. As Alex said, AI technology can effectively discover information that humans cannot or are difficult to capture, process high-latitude information that is difficult for humans to process, and analyze unstructured data (such as news, social platform data, pictures, etc.) to make scientific and rational decision-making.
In addition, we find that the application of AI technology in asset management was closely related to the development of AI technology. The application of AI in the financial field has existed since this concept was put forward, but it was not widely used due to objective factors such as hardware. Since Hinton et al. proposed

Can Do vs Should Do
In the current AI Hype, people are trying to apply AI-based solutions to various fields, and the asset management industry is also actively exploring the use of AI systems, and taking advantage of its data insight capabilities to provide better services.
We believe that in the field of asset management, the current AI systems do provide data insights far beyond human beings' ability. They can better capture the detailed information in the data and automatically discover and learn the laws of market changes caused by this information.
Therefore, we believe that technologically, though the AI systems are costly and complex, the performance of some pioneers in this area has proven the value of these systems. The use of AI systems to optimize asset allocation is a "can do" decision.
But on the other hand, the risk of AI systems in the asset management field has not yet been fully tested, and customers' trust in a "black box" system that is difficult to explain the logic of decision has not yet been fully established. Therefore, we believe that the application of AI and asset management is a "should do" decision, but we are more vigilant about its systemic risks.

Rebellion's Vision
Rebellion Research was the first mover in AI in asset management. "We are more than AI in asset management -providing AI financial advisors and hedge funds", Alex pictured, "we are a global machine learning think tank". AI Investing. Rebellion Research's technology enabled it to process an extremely diverse set of information, and its analysis was based on macroeconomic, fundamental, technical, and traditional factors, such as growth, value, momentum, and so on. Rebellion used its knowledge of volatility and interrelationships among stocks to create portfolios that balance risk and expected returns. Bayesian statistics were the backbone of Rebellion's artificial intelligence-based investment software, providing a flexible framework that enabled Rebellion to automatically combine new data available every day with previous market knowledge to predict stock performance. Rebellion was founded by three mathematicians who believe they can find alpha by using machine learning for long-term investment, covering 53 countries to individual stock prices, to create an optimized portfolio that beats the global stock market with 24/7 service.

AI Stock Advising.
Rebellion's AI-based global equity strategy has been managing assets for clients since January 1, 2007. Since its establishment in 2007, it has performed significantly better than the S&P 500 (see Exhibit 1). This strategy held a diversified portfolio consisting of 90-120 global stocks. Rebellion's absolute return strategy was a 0 beta, low volatility strategy consisting of 15-20 ETFs (see Exhibit 2). Rebellion provided cash alternatives for customers, Education. Rebellion also had close relationships with many high-profiled universities in AI in asset management education, such as UCB Haas MFE, MIT Sloan MFin, Cornell SC Johnson, NYU Courant Finmath, etc., in providing high-quality guest talks and co-op internships.

Challenges & Risks
Technologies. Although AI had been very powerful in asset management and other finance areas, it had its limitations. DL-based AI systems have the ability to learn massive amounts of data that humans cannot process and capture detailed information that traditional methods cannot use, but based on the current technology, deep learning models are still a "black box" for users.
As human financial advisors, when we constructed investment portfolios, we could explain the reasoning and logic behind them. However, robo advisors, DL-based especially, were poor at interpretation, the evaluation of the model's effect strongly relies on historical data, and we cannot predict whether the model's output is still valid under unconventional input, which reduced the liability, These feedbacks may be coupled with other AI models in the market and cause further market volatility. Additionally, despite the fast development of hardware, it was still hard to process hundreds of tons of finance news and transaction data in real-time. This response time of the system could be acceptable when the market is relatively stable, but the reliability of AI-based systems in a crash or crisis is still waiting to be valid.

Threats from Fast-movers.
Rebellion was faced with many competitors, not only new AI asset management companies, but also previous traditional hedge funds which were transforming to AI. The former was usually featured and founded by previous famous fund managers or university researchers. The latter, with hundreds of billions of AUM and AI experts from IBM Waston, Microsoft Research, Google Brain, was generally leading and taking over the AI asset management market.

Rebellion's Next Move
In light of Rebellion's previous success in the 2008 financial crisis, 2009 greek debt crisis, 2014 oil price crash, 2019 COVID-19, etc., Alex was quite confident in Rebellion Research's AI system. Alex's ambition was far more than the finance area, "we also routinely collaborate with US government and military experts, covering domains such as drone image recognition and retail inventory analysis." On the technology side, ML, more accurately, Bayes, was the heart of Rebellion. With the new development, breakthroughs, and trends in DL, RL, and Auto-ML, should rebellion reconstruct the existing system with more deep methods?
On the business side, should Rebellion keep its main focus on the finance area or explore more broad areas in AI? How could rebellion survive the fast-movers?


extfieldsofstudy:
Computer Science
Physics

s2fieldsofstudy:
Computer Science
Mathematics

text:
Neural Inference of Gaussian Processes for Time Series Data of Quasars

The study of quasar light curves poses two problems: inference of the power spectrum and interpolation of an irregularly sampled time series. A baseline approach to these tasks is to interpolate a time series with a Damped Random Walk (DRW) model, in which the spectrum is inferred using Maximum Likelihood Estimation (MLE). However, the DRW model does not describe the smoothness of the time series, and MLE faces many problems in terms of optimization and numerical precision. In this work, we introduce a new stochastic model that we call $\textit{Convolved Damped Random Walk}$ (CDRW). This model introduces a concept of smoothness to a DRW, which enables it to describe quasar spectra completely. We also introduce a new method of inference of Gaussian process parameters, which we call $\textit{Neural Inference}$. This method uses the powers of state-of-the-art neural networks to improve the conventional MLE inference technique. In our experiments, the Neural Inference method results in significant improvement over the baseline MLE (RMSE: $0.318 \rightarrow 0.205$, $0.464 \rightarrow 0.444$). Moreover, the combination of both the CDRW model and Neural Inference significantly outperforms the baseline DRW and MLE in interpolating a typical quasar light curve ($\chi^2$: $0.333 \rightarrow 0.998$, $2.695 \rightarrow 0.981$). The code is published on GitHub.

Introduction
Quasars are astronomical sources powered by the accretion of matter falling onto the black hole in a galaxy's center. The accretion onto the black hole is a turbulent process, so the brightness varies intrinsically over time. This variability not only allows for studying the region around the black hole [1] but also enables one to trace the evolution of the Universe through strong gravitational lensing [2]. The former requires an interpretable model of radiation variability, whereas the latter requires interpolation of a time series with irregular time sampling.
Due to the accretion turbulence, the variability of quasar radiation is usually modelled by a stochastic Gaussian process called "Damped Random Walk" [DRW;3]. This model has two main problems: the first problem is that the DRW can model the quasar spectrum on time scales of years but not on the scales of months, where the quasar spectrum steepens.  and CDRW (orange) introduced in this paper. Error bars and χ 2 are derived from MCMC sampling [11; 12]. CDRW better describes the power spectrum at high frequencies. [4; 5; 6; 7]. The second problem is associated with the maximum likelihood estimation (MLE), which is used to infer the parameters of a DRW: this method has convergence issues, high computation complexity, and, sometimes, poor 2σ constraints on the inferred parameters.
In this work, we aim to solve these two problems of conventional DRW modelling of quasar light curves. In Section 2.1, we introduce a new physically-interpretable model Convolved Damped Random Walk (CDRW) that well describes the quasar spectrum on time scales, from days to decades. In Section 2.2, we propose the Neural Inference method that aims to solve problems of maximum likelihood estimation. Our approach is task-specific in contrast to the generalized approach in [8]. In Section 3, we present the data. In Section 4, we present the experiments. We conclude in Section 5.

Convolved Damped Random Walk model of quasar radiation
The conventional physical model of quasar light curve F (t) is given by the lamp-post radiation g LP and thin-disk reverberation P TD : where τ and ρ are the characteristic times of radiation from accretion onto the black hole and from its re-radiation by the outer disk, respectively [13]. Usually, F (t) is modelled as a DRW, but this model fails to describe the power spectrum at high frequencies (Fig. 2). This discrepancy can be mitigated if the driving lamp-post radiation g LP (t) is assumed to be DRW instead. In this case, convolution with thin-disk reverberation kernel P T D suppresses the power spectrum at high frequencies. The spectrum of this kernel is well fitted by a softened powerlaw. Such a derivation brings us to a new model of quasar radiation that we call Convolved Damped Random Walk (CDRW) with power spectrum: where ω is the circular frequency 2πf . Given τ ≥ ρ, τ is correlation time. In turn, the reverberation time ρ defines the smoothness of the curve. This model well describes the power spectra of real quasars on all frequencies (Fig. 2). The newly introduced smoothness of ρ results in better interpolation performance of the model (Fig. 1). Moreover, in border cases, it reduces to the main types of stochastic processes: white noise, Matern ν = 1.5 noise, DRW, and Random Walk (red noise).

Neural inference of Gaussian processes
Without prior knowledge of the quasar flux F observed at times t obs , the baseline approach to the inference of a Gaussian process is MLE, that is, maximization of likelihood p(F (t obs )|µ, σ 2 , τ, ρ). In the context of numerical methods, MLE requires matrix inversion that has computational complexity O(N 3 ). For light curves with a high number of observations N , such an inference requires much computational power, moreover, the numerical errors in matrix inversion can jeopardize the inference results. In the context of the theory of optimization, MLE faces convergence issues due to orders of magnitude differences between gradients on different parameters. Additionally, there are degeneracies between the parameters and biases of modes of the posterior distribution -e.g. Bessel correction for inverse Gamma posterior of sample variance [14].
To mitigate these negative aspects of Gaussian process inference, we introduce a novel method that we call Neural Inference. We use a neural network to predict the final correlation time τ and an initial guess of reverberation time ρ. In this case, maximization of the likelihood p(F (t obs ), τ |µ, σ 2 , ρ) leads to significantly more accurate parameters. Moreover, these estimations become more stable with respect to the stochastic realisation of the studied Gaussian process.

Data
We carry out experiments on synthetic data that mimic real observations. The observations of quasars have 135 ± 25 days-long annual gaps due to occultation by the Sun and bad weather. Within the observation season in the n-th year, we model time sampling as a stochastic process: where P oissonP is a discrete Poisson process [15], that models short observation gaps, and is time in the night. The statistical form and parameters of the observation schedule correspond to the COSMOGRAIL dataset [16].
Finally, τ min ≈ 11 to a minimal relevant DRW break reported in the work [19]. The curves are normalized to (-1,1). A 2-year-long sample from a mock light curve is presented in Fig. 1.  Neural network. For training, we simulate a dataset of 300,000 labels log(τ ) and ρ according to eq. 4 and split them in train/validation/test in ratio 0.8/0.1/0.1. Both stochastic observation times t obs and radiation F (t obs ) are sampled in the runtime of training to avoid overfitting of random realisations. The architecture of the network is presented in Fig. 3. In the neural network, the data is interpolated with B-splines on the grid (10,365), then processed by three ResNet blocks [20] with 64,128,256 filters. It is followed by linear reduction of the filters to 64 and the attention layer, which is applied by the Learnable Dictionary Encoding (LDE) [21; 22] with 64 clusters. Finally, the encodings are linearly transformed to predictions of log(τ ) and ρ.

Results
The neural network is trained with MSE loss for four epochs until convergence. It reaches RMSE 0.15 for log(τ ) and 0.98 for ρ on the test set. On Tesla P100, an epoch takes half an hour, given that sampling of the data is done on the GPU. We carry out data sampling using Jax [23], the network itself is implemented in PyTorch [24].
Experiments. To compare our methods to baseline approaches, we simulate 1000 mock quasar light curves with different stochastic realisations of t obs and F (t obs ) (eq's. 3,4), but the same spectral parameters τ and ρ for all these curves. First, we pick τ = 540 as the median correlation time for the observed quasars [19]. Second, we pick ρ = 4.77, so it results in the break on a month scale, which is midway between DRW ρ = 0 and the maximal reported break of two months.
For the first experiment, we compare the performance of Neural Inference (Fig 3) and baseline MLE optimization [TNC; 25; 26] for the inference of log(τ ) and ρ. The resulting metrics are presented in Table 1. In this experiment, Neural Inference outperforms baseline MLE in every metric.
For the second experiment, we compare performance in the interpolation of the same curves on a grid with a regular cadence of one day. In this experiment, the baseline approach to quasars is interpolation with DRW, inferred with MLE. In turn, the introduced approach is interpolation with CDRW, inferred using the neural inference technique. The resulting metrics are presented in Table 2.
The difference between interpolation means µ(t) (MAE, RMSE) is marginal, but the χ 2 of the new method is significantly better and predicts much more reliable uncertainties than the baseline.

Conclusion
In this work, we introduced the new Convolved Damped Random Walk (CDRW) model of quasar radiation (Sec. 2.1) that well describes spectra of quasars on all the time scales. Additionally, we introduced a new method of inference of Gaussian processes, which we call Neural Inference (Sec. 2.2). This method uses neural networks to mitigate several problems of conventional MLE optimization. A combination of these two novelties results in significant improvement in inference and interpolation of a model of a typical quasars light curve (Tables 1, 2).

Impact statement
Impact on Astrophysics: The new CDRW model decouples the temporal scales of accretion onto a black hole from that of reverberation of the disk surrounding it. The introduced reverberation time ρ is the direct tracer of the characteristic radius of the disk R λ , that traces the mass of the black hole, Eddington luminosity, and accretion efficiency. Therefore, the CDRW model introduces an explicit constraint on those parameters, which is designed to enhance our understanding of galaxy physics.
On the other hand, ρ defines the short-scale characteristic time of quasar evolution. The distribution of this parameter across quasars can help optimize the observation schedule to maximize the obtained information while minimizing the load on the telescope.
In turn, the Neural Inference approach significantly elevates the quality of inference and interpolation of mock quasar light curves. The improvement in inference results in more accurate information about the physics of quasars. The improvement in interpolation quality reveals itself in the studies of gravitationally lensed quasars. The combination of CDRW and Neural Inference provides better uncertainties that result in the more accurate inference of lensing-induced time delays and, therefore more strict constraints on the Hubble rate.
Impact on other domains: Time series phenomena permeate our everyday lives -from finance and healthcare to climate science and astrophysics, since the mathematical formulation of the task is often similar. The Gaussian process, especially DRW, is a baseline approach in many fields. Moreover, people face the same problems with MLE-based inference in all these fields. Therefore the introduced Neural Inference technique can be applied out of the box to constrain the kernel of a Gaussian process for many different types of data. Additionally, the convolutional term in the CDRW model can be well-adjusted to any data in order to account for domain-specific short-term interaction.