extfieldsofstudy:
Computer Science
Biology

text:
Recurrent Brain Graph Mapper for Predicting Time-Dependent Brain Graph Evaluation Trajectory

Several brain disorders can be detected by observing alterations in the brain's structural and functional connectivities. Neurological findings suggest that early diagnosis of brain disorders, such as mild cognitive impairment (MCI), can prevent and even reverse its development into Alzheimer's disease (AD). In this context, recent studies aimed to predict the evolution of brain connectivities over time by proposing machine learning models that work on brain images. However, such an approach is costly and time-consuming. Here, we propose to use brain connectivities as a more efficient alternative for time-dependent brain disorder diagnosis by regarding the brain as instead a large interconnected graph characterizing the interconnectivity scheme between several brain regions. We term our proposed method Recurrent Brain Graph Mapper (RBGM), a novel efficient edge-based recurrent graph neural network that predicts the time-dependent evaluation trajectory of a brain graph from a single baseline. Our RBGM contains a set of recurrent neural network-inspired mappers for each time point, where each mapper aims to project the ground-truth brain graph onto its next time point. We leverage the teacher forcing method to boost training and improve the evolved brain graph quality. To maintain the topological consistency between the predicted brain graphs and their corresponding ground-truth brain graphs at each time point, we further integrate a topological loss. We also use l1 loss to capture time-dependency and minimize the distance between the brain graph at consecutive time points for regularization. Benchmarks against several variants of RBGM and state-of-the-art methods prove that we can achieve the same accuracy in predicting brain graph evolution more efficiently, paving the way for novel graph neural network architecture and a highly efficient training scheme.

Introduction
Latest neuroscience studies have emphasized the importance of personalized treatments for brain disorders that can significantly improve patient's recovery [1]. Brain disorders such as mild cognitive impairment (MCI) can be easily reversed if diagnosed at an early stage before evolving into irreversible Alzheimer's disease (AD) [2]. As such, recent landmark studies [3,4,5,6] proposed using the breadth of machine learning to predict brain connectome evolution trajectory. For instance, [3] has proposed a learning-based framework to predict the longitudinal development of cortical surface and white matter fibers. To do so, first, they used multiple atlases to generate a spatially heterogeneous atlas that mimics the cortical surface of the target subject. Second, they predicted spatio-temporal connectivity features from neonatal brains using low-rank tensor completion. Furthermore, [4] proposed a deep learning model that jointly classifies and predicts the evolution trajectory of the brain from a single acquisition point. At the baseline, they identified the landmarks, namely regions of interest (ROIs), and utilized supervised and unsupervised learning to predict the evolution trajectory at each landmark.
However, such studies were conducted only on brain images that solely consider local neighborhood connectivities undeniably overseeing the brain global connectivity pattern. To properly diagnose brain connectivity disorders, the brain must be viewed as a large interconnected graph [7] where each ROI represents a node, and each pairwise connectivity between two ROIs represents an edge. To address this limitation, we set out a more challenging problem, which is predicting the evolution of brain graphs from a baseline observation. In this context, recent works [5,6] leveraged generative adversarial networks (GANs) [8], where they proposed the first graph-based GAN specialized in AD evolution trajectory prediction. Namely, GANs [8] address the problem of unsupervised learning by training two neural networks: generator and discriminator. The generator takes randomly distributed data as input and generates synthetic data that mimics a real distribution. On the other hand, the discriminator inputs the generated data and predicts whether it is real or fake. Both generator and discriminator compete in an adversarial way. Hence, the overall quality of the generated data improves with training epochs. [5] used gGAN to learn how to normalize a brain graph with respect to a fixed connectional brain template (CBT). Their gGAN architecture is made up of a graph normalizer network that learns a high-order representation of each brain in a graph and produces a CBT-based normalized brain graph. Furthermore, [6] proposed EvoGraphNet that connects series of gGANs each specialized in how to construct a brain graph from the predicted brain graph of the previous gGANS in the time-dependent cascade.
Although gGANs were proven to be successful for predicting brain connectome evolution trajectory given a single observation, there is still a considerable amount of computational complexity. Since gGANs contain two graph neural networks (i.e., generator and discriminator), they require a significant amount of computational power and training time. Also, these architectures require a sequential framework where each gGAN has to generate a time point t i to get the following time point prediction t i+1 .
This challenge raises the following question: can we mimic the prediction power of the above-mentioned sequential gGANs without the need to use highly complex sequential networks?
According to [9], brain connectivities do not evolve randomly. Their evolution follows a temporal scheme that aims to satisfy the patient's needs for each given age and health condition. In that regard, recurrent neural networks (RNNs) [10] are known for their temporal pattern recognition. Therefore, in this paper, we propose to power our model with RNNs. We term our proposed model Recurrent Brain Graph Mapper (RBGM), the first framework to predict brain connectome evolution while efficiently reducing complexity and training time consumption. Our model uses a shallow graph convolutional neural network architecture to predict brain disease's evolution trajectory given a baseline graph. To do so, for a given time point, t i each mapper uses the ground-truth from the previous time point t i−1 to predict the brain connectivity scheme at the time point t i instead of starting from the initial time point. To do so, we apply the teacher forcing method [11], which is known for its quick and efficient training recurrent-based models.
We propose preserving the topological consistency between the predicted and ground-truth brain graphs at each time point. To do so, we integrate a topological loss measuring the topological discrepancy between the predicted brain graph and its corresponding ground-truth brain graph. Furthermore, we leverage a l1 loss to minimize the sparse distance between two serialized brain graphs to capture time-dependency between two consecutive observations. We also investigate the effect of Kullback-Leibler divergence (i.e., KL-divergence), where we enforce the preservation of node distribution between predicted and ground-truth brain graphs over time. We articulate the main contributions of our work as follows: 1. On a methodological level. Our proposed RGBM is the first RNN-based geometrical deep learning framework that predicts the time-dependent brain graph evolution trajectory from a single observation. 2. On a conceptual level. Our model reduces the complexity of the geometrical deep learning framework by speeding up the training while maintaining similar performances to the state-of-the-art methods. 3. On clinical level. Our RGBM can be used to prevent and reverse the onset of neurological diseases.

Proposed Method
This section introduces the key steps of our RBGM for predicting brain graph evolution from a single observation. Table 1 displays the mathematical notations that we use throughout our paper. We denote the matrices as boldface capital letters (e.g., X) and scalars as lowercase letters (e.g., m). The transpose operator is denoted as X T . The graph neural network takes the input brain graph X tr ti−1 ∈ R nr×nr for a given time point t i , where n r is the number of ROIs to learn node embedding V l = [v l 1 , v l 2 , . . . , v l nr ] T that captures the node-to-node relation and visualizes it in vector form. Then a transformation layer takes these node embeddings V l and computes the pairwise absolute difference to predict the brain graph at the time point t i given byX tr ti ∈ R nr×nr . First V l is repeated horizontally n r times to obtain R ∈ R nr×nr×nr . Next, we compute the absolute difference between R and its transpose. Finally, the resulting tensor is sum along z-axis to obtain the predicted brain graphX tr ti for the time point t i .
Overview of Recurrent Brain Graph Mapper for predicting brain graph evolution trajectory from a single baseline. Our proposed RBGM is composed of m mappers for m time points, as shown in Fig. 1. Each mapper can predict a brain graph for a given time point t i using its corresponding ground-truth brain graph at the time point t i−1 as an input. The recurrent graph convolution enables each mapper to capture temporal changes in the brain connectivity pattern between consecutive time points. Also, it increases the prediction power of each mapper, hence using fewer convolutional layers. Furthermore, Table 1: Mathematical definitions.

Mathematical notation
Definition m number of time points ns number of training subjects mr number of edges nr number of ROIs in brain S tr i node strength vector of ROI i in the ground-truth brain grapĥ S tr i node strength vector of ROI i in the predicted brain graph X tr t i training brain graph connectivity matrices ∈ R nr ×nr at tî X tr t i predicted brain graph connectivity matrices ∈ R nr ×nr at ti Mi the neighborhood containing all the adjacent nodes of node i F l edge-conditioned filter Θ l learnable edge-based parameter for dynamic graph convolution vi l node embedding of ROI i at layer l ∈ R d t W l weight parameter b l bias term R horizontally replicated brain connectivity matrix ∈ R nr ×nr ×nr R T transpose of horizontally replicated brain connectivity matrix ∈ R nr ×nr ×nr hi hidden state matrix at ti ∈ R mr ×mr W ih input to hidden weight for recurrent filter ∈ R 1×mr W hh hidden to hidden weight for recurrent filter ∈ R mr ×mr b h bias term for recurrent filter ∈ R mr ×mr we apply the teacher forcing method [11] to quickly and efficiently train our RGBM.
To enhance our method's robustness, we propose using l1 loss thanks to its resilience against outliers to enforce the connectivity consistency across time points. Thus, we express the l1 loss for each subject tr using the predicted brain graphX tr ti−1 from the mapper M i−1 , and its corresponding ground-truth brain graph at t i as follows: This acts as a regularizer over time and aligns with the sparse nature of brain connectivity evolution. In addition to the l1 loss, we propose a second loss to preserve the topological consistency between predicted brain graphs and their corresponding ground-truth at each time point. To frame the topological loss, we define a node strength vector measuring the topological strength for each node in a given graph. Since our brain graph is fully-connected (each node has a same number of edges), we chose node strength as a centrality measures. We compute the node strength vector by adding the weights of all edges connected to a node of interest. As such, S = [S , S , ..., S ] T represents the node strengths for all ROIs where n r is the number of ROIs. The following equation gives the topological loss: The full loss. We combine the previous losses to train our RGBM as follows: where λ 1 , and λ 2 are hyperparameters adjusting each corresponding loss.
The mapper network architecture. Each mapper m uses our proposed recurrent graph convolution (RGC) function. We leverage the teacher forcing method [11] to speed up training and increase the overall performance in training. Namely, the teacher forcing is a common method that speeds up training and improves the quality of recurrent-based models. It enforces the recurrent model to use ground-truth samples to predict the brain graph at the following time point. According to this method, for a given time point, t i the mapper takes the ground-truth X tr ti−1 from the time point t i−1 instead of taking the predicted brain graphX tr ti−1 from the preceding mapper M ti−1 to make prediction for the time point t i in training phase. Therefore, the need to start from the initial time point is eliminated during training.
Dynamic edge-filtered convolution. Each mapper in our RBGM uses the dynamic graph convolution with edge-conditioned filter proposed by [12]. Let G = (V, E) is a directed or undirected graph, where V is the set of n r ROIs and E ∈ V × V is a set of m r edges between each ROI. Let l be the layer index. For each layer l ∈ {1, 2, ..., L}, F l : R dm → R d l ×d l−1 represents a filter-generating network that generate edge weights for the message passing between ROIs i and j given features of e ij . d m and d l are dimensionality indexes. This operation is expressed as follows: where v l i is the node embedding for the ROI i at layer l. N (i) denotes the neighbors of ROI i. F l is the neural network that maps R dm to R d l ×d l−1 with weights W l . Θ l is the dynamically generated edge specific weights by F l . The b l ∈ R d l is the bias term. We note that F l can be any type of neural network.
We draw inspiration from the image-based recurrent neural network architecture, which shows outstanding performances on time-series data prediction [13,14]. This type of network can remember the former information and process new events accordingly thanks to its hidden state, which holds the former information (i.e., learned information in the previous layer). Each RNN cells takes two distinct inputs: (i) the input brain graphs from the current time point, and (ii) hidden state value from the brain graphs at the previous time point, then updates the hidden state, which holds the representation of the knowledge from the prior time point.
Proposed graph recurrent-filter. We propose the first edge-based recurrent graph neural network Fig.2 by re-designing the edge-conditioned filter [12] in the graph convolution layer as a graph recurrent-filter so that each mapper can capture temporal changes on brain connections over time, as shown in Fig.  2. Therefore, unlike [6], the need to start from the initial time point is eliminated during training. Our proposed graph recurrent filter can process past information when generating messages between each ROI to capture temporal changes of brain connectivity. To do so, it takes the set of edges e ∈ R mr×1 for a given time point t i and the hidden state matrix from the previous time point t i−1 given by h ti−1 ∈ R mr×mr , which acts as a memory and processes past information. Then it updates the hidden state matrix h ti ∈ R mr×mr for the current time point t i . In order to avoid the vanishing gradient problem which makes the network's gradients tend to zero (i.e., hard to learn parameters), we need a function which can bound the gradient and eliminate the risk of divergence during the training.
To do so, we use tanh [15] as an activation function in our graph-recurrent filter since it allows the state values to update by bounding in the range of [−1, 1] compared to other activation functions such as sigmoid. The equation for our recurrent edge-filtering function F l (e ti , h ti−1 ) is expressed as follows: where W ih ∈ R 1×mr and W hh ∈ R mr×mr are learnable parameters for inputto-hidden weight and hidden-to-hidden weight respectively. b h ∈ R mr×mr is bias term.
The transformation layer architecture. Let X tr ti−1 ∈ R nr×nr be the input brain connectivity matrix at a given time point, t i where n r is the number of ROIs. After obtaining the output node embeddings V l = [v l 1 , v l 2 , ..., v l n ] T of RGC layer from a given input X tr ti−1 , we construct predicted brain graphX tr ti at t i by computing pairwise absolute difference of learned embeddings [16]. To do so, first V l is replicated with respect to the horizontal axis n r times to obtain R ∈ R nr×nr×nr . Then, we compute the absolute difference between R and its transpose R T . Finally, the resulting tensor is the sum along z-axis to obtain the predicted brain graphX tr ti ∈ R nr×nr for the time point t i .

Results and Discussion
Evaluation dataset. We conducted experiments on OASIS-21 longitudinal dataset with 113 subjects [17]. This set contains longitudinal collection of 150 subjects aged between 60 to 96. Each subject's brain scans were acquired 3 times one year apart. For each subject, we construct a cortical morphological network derived from cortical thickness measure using structural T1-w MRI as proposed in [18]. Each cortical hemisphere is parcellated into 35 ROIs using the Desikan-Killiany cortical atlas. We construct our RBGM with PyTorch Geometric library [19]. To inherit the previous layer embedding V l−1 ∈ R nr×nr , we multiply V l−1 by dynamically generated edge specific weight Θ l . Finally, V l is computed by combining the previous layer embedding and the average message passing between ROIs. (B) Recurrent graph convolution. First, the graph recurrent-filter network creates the message between ROIs i and N (i) by taking hidden state value h ij in contradiction to the conventional edge-conditioned filter graph generation. To inherit previous layer embedding V l−1 ∈ R nr×nr , we multiply V l−1 by dynamically generated edge specific weight Θ l . Finally, V l is computed by combining the previous layer embedding and average message passing between ROIs.
Parameter setting. In Tab 2, we report the mean absolute error between ground-truth and synthesized brain graphs at follow-up time points t 1 and t 2 .
In Tab 3, we publish the required training time for each comparison method respectively. We set hyperparameters of each mapper as follows: λ 1 = 1, λ 2 = 10. We used AdamW [20] optimizer and set the learning rate at 0.0001 for each mapper. Finally, we trained our model by using 3-fold cross-validation for 200 epochs using an NVIDIA Tesla V100 GPU. Comparison Method and evaluation. Due to the lack of RNN-based comparison methods that consider time-dependency for brain graph prediction, we benchmarked our RBGM against some of its variants. We call the first benchmark method: RBGM (w/KL), where we replaced topological loss with KL-divergence loss between the predicted graph at t i and its corresponding ground-truth brain graph. The KL-divergence minimizes the discrepancy between ground-truth and predicted connectivity weight distribution at each time point t i . The second benchmarking method is against the current state-of-the-art EvoGraphNet [6] in order to assess the power of topological loss and the recurrent graph convolution. In Tab 2 we report the mean absolute error (MAE) between ground-truth and predicted brain graphs for consecutive time points t 1 and t 2 for each comparison method. Our proposed RBGM outperformed baseline methods at t 1 by achieving both the lowest mean MAE (averaged across the 3 folds) and the overall best MAE as shown in Tab 2. However, for t 2 , EvoGraphNet achieved both the best MAE and mean MAE results. Notably, results show that our RBGM closely matches the best results at time point t 2 by an error difference of 7 × 10 −3 in mean MAE yet outperformed EvoGraph-Net in time consumption by achieving 46% less training time. To the best of our knowledge, such time/complexity/error is a delicate compromise to make. Under such a compromisation paradigm, we can fairly judge the outperformance of our proposed RBGM is matching state-of-the-art results given less complexity and time consumption. Overall, our RBGM performs almost similar to the state-of-the-art method EvoGraphNet while speeding up the training and reducing the complexity Tab 3 since it consists of fewer convolutional layers. However, our RBGM has a few limitations. So far, we have worked on brain graphs where the single edge connects only two ROIs. We aim to generalize our RGBM to handle brain hypergraphs where multiple edges can link two ROIs in our future work. This will enable us to better model and capture the complexity of the brain as a highly interactive network with different topological properties.

Conclusion
In this paper, we proposed the first edge-based recurrent graph neural network RBGM that uses a novel recurrent graph convolution to predict the brain connectivity evolution trajectory from a single time point. Our architecture contains m number of mappers for m time points. We proposed a time-dependency loss between consecutive time points and a topological loss to preserve topological consistency between predicted and ground-truth brain graphs at the same time point. The results showed that our time-dependent RBGM achieved a similar prediction accuracy compared to the state-of-the-art EvoGraphNet while reducing the training time and complexity. The RBGM is generic and can be used to predict brain graphs for any given time point. In future studies, we aim to generalize our RBGM to using hypergraphs and account for brain hyperconnectivity.

Acknowledgements
This work was funded by generous grants from the European H2020 Marie Sklodowska-Curie action (grant no. 101003403, http://basira-lab.com/normnets/) to I.R. and the Scientific and Technological Research Council of Turkey to I.R. under the TUBITAK 2232 Fellowship for Outstanding Researchers (no. 118C288, http://basira-lab.com/reprime/). However, all scientific contributions made in this project are owned and approved solely by the authors.


s2fieldsofstudy:
Computer Science
Physics

text:
Research on pattern recognition technology for a highperformance optical fiber vibration sensor

In this paper, the optical fiber vibration sensor based on Mach-Zehnder Interference (MZI) principle is designed and researched, which can improve the ability to recognize the physical intrusion events for underground power pipelines by means of pattern recognition technology. Through data collection, analysis and processing, the vibration event database of key intrusion events in typical environment is established. By using coherent optical beat frequency, vibration recognition algorithm, machine learning and other technologies, the frequency domain diagram and time domain diagram of different vibration signals are analyzed and judged. In the end, the feature values of optical fiber vibration sensor such as frequency range, amplitude and waveform density are extracted.

Introduction
The buried power cables are developing towards high density and long length with the accelerating pace of urban reconstruction. However, shallow buried power cables installed on both sides of urban roads are easily damaged by various external forces, which causes great threat to life, property and social economy. The traditional monitoring system analyzes and judges the types of intrusion events with low accuracy rate and high false alarm rate, which brings great challenges to the prevention of external intrusion in underground power pipelines [1][2][3].
In this paper, the abnormal vibration signals along the underground power pipelines are detected using the optical fiber vibration sensor based on M-Z optical fiber interference principle, acquired and processed using coherent optical beat frequency technology and vibration identification algorithm. And the frequency domain diagram and time domain diagram of vibration signals are analyzed to extract feature values. Then, the intrusion event identification model is established using machine learning method, which can improve the recognition ability of optical fiber vibration sensor to physical vibration intrusion event [4][5].

Composition and Principle of the Optical Fiber Vibration Sensor
The optical fiber vibration sensor uses three single-mode fibers in a fiber cable laying along the same channel as a sensor [9]. The fiber 1 and fiber 2 are used to test the vibration signal of the fiber, and the fiber 3 is used to transmit the signal. When external vibration occurs, the refractive index and length of the fiber core will change, resulting in the change of phase in fiber. Then, through the data acquisition, analysis, processing and feature value extraction of vibration signals in intrusion events, the vibration event database is established to recognize the physical vibration intrusion. The principle is shown in Fig. 1  The laser source launchs two beams of light. When the two beams are subjected to external shock or vibration during transmission, the refractive index of the fiber at the vibration point will change, resulting in the change of the optical wave phase. Due to the different spatial position, the phase modulation will be different, which changes the phase difference between the two beams. When the two beams reach the coupler 2, the phase difference will cause the change of interference fringe intensity. Then this signal can be detected by photoelectric conversion, data acquisition and signal processing. After demodulation, the detector restores the vibration wave signal, which basically reflects the waveform of the event. Compared with the waveform of the existing event library, the type of vibration source can be distinguished.

Coherent Beat-frequency Technology
When two beams of light whose wavelength difference is far less than the magnitude of wavelength (the frequency difference is very small) propagate in the same direction, the light with short wavelength will modulate the light with long wavelength, and form an envelope of the light intensity change. This phenomenon is called optical beat-frequency. The principle is shown in Fig. 2, Figure 2. Principle diagram of optical beat-frequency Optical fiber vibration sensor adopts coherence beat-frequency technology. It means that two lasers with approximate wavelengths are used to generate coherence, and the beat wavelength is controlled by slightly changing the wavelength difference below, which is easy to control the coherence sensitivity and achieve large dynamic range of unsaturated detection. The detection signal is unified to obtain the linear reduction of weak mechanical vibration waveform and the minimum signal can be detected by optical phase modulation and coherent detection [6][7].

Collection and Analysis of Data
The collection of data can be achieved in the acquisition module of the optical fiber vibration sensor, after wavelet filtering to suppresse the background noise, the upper computer processes relevant data information by algorithm. Then, set a threshold to determine whether to alarm. When the data value is greater than the threshold, the system alarms. When the data value is less than the threshold, the system does not alarm.

Extraction of Feature Value
The vibration signal of intrusion event is in the ' time domain ', that is, x axis is time ( seconds or minutes ), and y axis is the measured vibration amplitude ( displacement, velocity, acceleration ). Based on this concept, displaying vibration waveform is a great accurate method. This method can show the actual vibration form of intrusion event and analyze its various vibration parameters. Compared with the vibration spectrum collected in the ' frequency domain ', this vibration waveform can more accurately locate the duration, vibration amplitude and the number of vibration behaviors in the same period of an intrusion event. Moreover, it is not necessary to take the average value of the high-frequency p-art of the spectrum from the system, and then compare it with the data of the event database to make a decision [8][9][10].
The vibration waveform is analyzed by waveform amplitude, amplitude threshold, duration and other parameters to distinguish between false alarm and threat intrusion. Although the waveform caused by knock vibration of accidental events such as branches, birds, small animals, hail, wind and rain and the waveform caused by human invasion are similar, after verification and analysis by machine learning and AI recognition system, the two waveforms are different in terms of the overall waveform amplitude, duration and the number of events generated at the same time.
Optical fiber vibration sensor uses time domain/frequency domain hybrid analysis method and combined with artificial intelligence recognition technology to analyze the phase of coherent signals in order to achieve accurate classification and recognition of vibration source signals, which greatly improves the accuracy of vibration source recognition.
Based on the research on the linear reduction of distributed optical fiber vibration sensing signal and the identification of vibration source type, the method proposed in this paper takes the power communication cable connected to the optical fiber vibration sensing instrument as a long-distance sensor, perceives the vibration signal along the line, collects the GPS coordinate data along the line, and draws the GIS electronic map of the line ( several lines ).
Through the collected signals of a variety of vibration sources, such as power cables and pipelines drilled by manual tools, power cables and pipelines operated by large machines and the lifting of pipe covers, the vibration event database of power cables and pipelines is established and enriched based on the linear reduction of optical fiber micro-vibration distribution signals and the vibration event data ( waveform diagram ) of vibration source types, and the classification and data archiving of different vibration events are formed.

Establishment of Recognition Model
Any complex signal is composed of multiple short-time simple signals of complex algorithm, and the amplitude related to time is broadened and compressed.
 Constructe the data mode by the maximum and minimum values of the signal which are selected according to the data, recognition effect and the number of segments.
 Calculate the Euclidean distance between the data and the model, and the model with the shortest Euclidean distance is regarded as the optimal data model.
 Input the parameters of all the optimal models to the system as the criteria for machine learning and learning.
 Compute multiple sets of data of various types by the established data model.  Train the input signal and the expected output signal by machine learning method. When the minimum variance of the input signal and output signal after training meets the requirements, the training is stopped.
The system collects the vibration signal of physical intrusion events, and converts the photoelectric signal into the corresponding vibration event waveform. Through analysis and research, various intrusion events in typical environments are simulated, and the corresponding vibration waveform dia-

RESULTS
Our company tested the 110KV and 10KV transmission cable lines from Tianxin substation to public service station, and from Tianxin station to Pingshi station, in Huadu District, Guangzhou, in the morning of July 12,2019.
When the transmission line is subjected to external forces, such as hand digging, machine mining, cable along with the transmission line will provide vibration signals to the host computer room, sending warning information to make the staff timely to the scene to stop the destruction.
After the actual monitoring of the signal that produced by opening well cover, hand digging, heavy machinery operation or rainfall by the optical fiber vibration sensor, the waveform of the open well cover in Fig. 4, the waveform of manual tool excavation in Fig. 5, the waveform of heavy machinery operation in Fig. 6 and the waveform of heavy rain in Fig. 7 are obtained.   Figure 6. Waveform of the signal produced by rainfall

DISCUSSION
From the analysis of Fig. 3, Fig. 4, Fig. 5, Fig. 6, we can found that the vibration amplitude caused by different reasons is quite different. Through the analysis of waveform amplitude, amplitude threshold, duration, waveform density and other parameters, we can distinguish whether the vibration signal is an intrusion with false alarm or a threat. At the same time, the relevant parameters are used as the basis for pattern classification.
In order to verify the above analysis, 100 tests were carried out on the invasion behaviors of open well cover, hand digging, heavy machinery operation and rainfall. The test results are shown in Table  1. From the test results in Table 1, it can be seen that the recognition rate of the pattern recognition technology of the optical fiber vibration sensor based on the M-Z fiber interference principle is greater than 96 %. The algorithm, model and feature values that related to pattern recognition and the waveform discrimination standard of intrusion events, which meet the requirement of optical fiber vibration sensor and can be used as the basis for pattern recognition of optical fiber vibration sensor.

CONCLUSIONS
In this paper, an optical fiber vibration sensor based on the principle of M-Z optical fiber interference is designed. The direct buried single-mode communication cable is used as the detection sensor of the optical fiber vibration sensor. Optical fiber vibration signals are analyzed by coherent beat-frequency technology. The frequency domain and time domain diagrams of different vibration signals are analyzed and judged. And the characteristic values, such as waveform amplitude, duration and waveform density, are extracted to distinguish different intrusion events. Through experiments, we can found that the pattern recognition rate of the invasion event of the power buried pipe gallery can reach more than 96 % by using the optical fiber vibration sensor, which meets the requirements of the pattern recognition of the optical fiber vibration sensor. And the related technologies can be used as the basis for the pattern recognition of the optical fiber vibration sensor. In summary, the pattern recognition technology of optical fiber vibration sensor based on M-Z optical fiber interference principle provides an effective method, which can reduce the false alarm rate of buried pipe gallery against external damage.


extfieldsofstudy:
Computer Science
Medicine

text:
Virtual Cohorts and Synthetic Data in Dementia: An Illustration of Their Potential to Advance Research

When attempting to answer questions of interest, scientists often encounter hurdles that may stem from limited access to existing adequate datasets as a consequence of poor data sharing practices, constraining administrative practices. Further, when attempting to integrate data, differences in existing datasets also impose challenges that limit opportunities for data integration. As a result, the pace of scientific advancements is suboptimal. Synthetic data and virtual cohorts generated using innovative computational techniques represent an opportunity to overcome some of these limitations and consequently, to advance scientific developments. In this paper, we demonstrate the use of virtual cohorts techniques to generate a synthetic dataset that mirrors a deeply phenotyped sample of preclinical dementia research participants.

INTRODUCTION
In 2019, the World Health Organization reported that around 50 million people had dementia and there are nearly 10 million new cases every year (Alzheimer's disease facts and figures, 2020). Dementia, a disease that affects an ever increasing proportion of the world's population, has ubiquitous deleterious effects on individuals which range from important deterioration of individuals and family members' well-being, to huge economic costs and considerable demand of public health services (Prince et al., 2014) making the case for advancing our understanding of dementia a critical priority for society. Scientific progress often results from synergetic interactions and joint efforts of academia, industry and lay members of society. Yet, in dementia, these efforts and interactions have not yet succeeded in the development of an effective treatment or cure for the disease.
Observational studies and clinical trials are important resources in research. Observational studies are mostly used to better understand the epidemiology of the disease and randomized clinical trials are the gold standard for testing efficacy of new drugs. In the context of dementia research, features of these studies have been discussed in the literature, and researchers have highlighted some challenges and methodological limitations of both types of studies (Ritchie et al., 2015). For instance, amongst other important features, limitations of the studies include their deficient design, the erroneous selection of individuals into the studies and focusing on erroneous outcomes. In fact, it has been argued that some of these features may partially explain the lack of success in the development of disease-modifying treatments for dementia (Anderson et al., 2017).
Importantly, both types of studies heavily depend on the participation of individuals who generously donate their time, data and specimens for research. In recent years, positive developments in research practices have resulted in increased engagement of research participants in most aspects and stages of research studies. However, the burden on participants can become substantial, and key barriers still exist that discourage some individuals from becoming involved in dementia research (Cardona-Morrell et al., 2017). For instance, concerns about data confidentiality and individuals' privacy are important worries that prevent some individuals from engaging in research (Grill and Karlawish, 2010). Furthermore, in some contexts such as low and middle income countries, where the number of dementia cases are expected to increase at a rate faster than in high income countries research and recruitment of study participants is even more challenging for various factors, including the lack of economic resources and cultural barriers (Ferri and Jacob, 2017). Consequently, the recruitment of individuals into dementia studies can be a challenging task for researchers, and as a result, studies may not have the optimal design to answer questions of interest.
Various practices further limit access and use of existing data sources by researchers interested in conducting research, although recent initiatives are making efforts to change practice. Limitations include, amongst others, limited data documentation, possessiveness of data and even limitations for data transferring and storage (Deetjen et al., 2015).
Virtual cohorts are digital non-identical, yet highly similar, synthetic data records that preserve the statistical properties of the original data, that pose new opportunities to advance dementia research. For instance, they may be used for the simulation of clinical trials with specific designs (Koval et al., 2019), to augment the size of datasets to improve the power to detect effects of interest, or of under-represented groups in a study (for example, ethnic minorities). They may also be used when access to person-level data is challenging, offering additional opportunities to analyse data that preserves the data features whilst not using the real data. Indeed, virtual cohorts have been used in other research areas such as heart models (Niederer et al., 2020), and molecular and clinical analysis of cancer virtual cohorts (Guan et al., 2020).
Yet, virtual cohorts are not incorporated in mainstream dementia research. The aim of this paper is to demonstrate the use of a virtual cohort to answer a commonly posed question in dementia research about factors associated with amyloid positivity using data from the European Prevention of Alzheimer's Dementia Longitudinal Cohort Study [EPAD LCS, Ritchie et al. (2016)].

EPAD LCS
The European Prevention of Alzheimer's Dementia (EPAD) project was initiated in January 2015 and is funded by the Innovative Medicines Initiative (Ritchie et al., 2020). The EPAD LCS had as its primary objective to be a readiness cohort for the EPAD Proof of Concept Trial and its secondary objective was to use the data generated for disease modeling. Based on this secondary objective, we selected this dataset to illustrate our methodology.
Research participants were eligible for inclusion if they were over the age of 50 and should not have a diagnosis of dementia. They must have also been deemed suitable in principle for later inclusion in a clinical trial and therefore should not had any medical or psychiatric disorders which would normally exclude people from such trials.
At baseline and subsequent assessments, research participants completed the entire protocol of assessments, which included the assessments of various domains including basic demography, fluid biomarkers, genetics, lifestyle, clinical, and psychiatric assessment, neuropsychiatric symptoms, function, cognition, and neuroimaging. For full details about the study protocol, please see Solomon et al. (2019).
Specifically, sociodemographic information included data on participants' age, sex, and education (years). The only biomarkers data collected in the EPAD LCS study are CSF (cerebrospinal fluid) ABeta_1_42, Tau and Phosphorylated Tau (p-tau and ttau). Using data from CSF ABeta_1_42, a threshold of 1,000 pg/ml was agreed upon to define amyloid positivity.
With regards to cognition, participants completed the Repeated Battery for the Assessment of Neuropsychological Status [RBANS, Sullivan et al. (2018)] and they were all tested for ApoE status. Sampling preparation and storage details can be found in the EPAD lab manual (available online at http:// www.ep-ad.org). Similarly, participants were asked about their family history of dementia, and their height and weight were also measured.

Analytical Methods
From the original EPAD analytical database, the EPAD cohort of 1,500 patient records (EPAD V1500) were extracted, including age, sex, ethnicity, years of education, height, weight, family history of dementia, as well as the p-tau, t-tau, ABeta_1_42, and RBANS. We performed pre-processing as follows: • ABeta_1_42 in some cases has strings such as ">1,700" or "<200"; those have been transformed to 1,701 and 199, respectively • We created a new binary variable "abeta < 1,000, " with a value of 0 if the ABeta_1_42 has a value <1,000 and 1 otherwise • We computed body mass index (BMI) from height and weight using the standard formula BMI = weight (kg)/height 2 (cm) • We computed a binary indicator of family history of dementia that took the value of 1 if either father or mother had had a history of dementia and they were the biological father or mother (respectively).
Using this cohort, we used Syntegra's synthetic data engine to create a synthetic dataset which has the same schema and validated the statistical fidelity of the synthetic data with the real data. For validation we applied a comprehensive set of methods as described next.

Univariate Marginal Distributions
We compared univariate (marginal) distributions of the variables between real and synthetic, and used the KS-statistics 1 as a goodness-of-fit metric for numeric variables and the KLdivergence 2 as a goodness-of-fit metric for categorical variables. These two statistics are of statistical fidelity between the real and synthetic variables. When the real distribution matches the synthetic distribution, we expect • The KS-statistics p-value to be high (>0.05), and the closer to 1.0 the better; this is a bit non-intuitive at first and is because in this case our null hypothesis is that the two distributions are 2 https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence the same, so a low p-value represents a high likelihood of the non-null hypothesis • The KL-divergence to be low (as close to zero as possible).
Intuitively, KL-divergence reflects the information loss when the distribution of the variable in synthetic data is used to approximate the distribution in the real data, which doesn't lend itself to an easy interpretation in terms of a clear threshold of "strong fidelity" vs. "weak fidelity" in the same way KS-Statistics does; nevertheless, one can compare the KL-divergence of a given variable over multiple synthetic datasets and lower values mean better statistical fidelity.

Pairwise Variable Correlation
We measured pairwise correlation between each pair of variables in our dataset, using the Pearson correlation coefficient, and compared the resulting heatmap of correlations between real and synthetic. Furthermore, we computed the L1-norm of the difference in correlation between real and synthetic cohorts, in a metric defined as: PCD-L1 = Corr real − Corr(syn) 1 . Naturally, a low value of PCD-L1 indicates small average difference in the correlation values between real and synthetic, and thus high statistical fidelity of the synthetic data with the real.

Predictive Model Performance
To measure how well predictive models perform if trained on the virtual cohort, we trained a predictive model for amyloidpositivity (ABeta_1_42 < 1,000) using some of the other variables (excluding other outcome variables) as predictors; we trained two models -one using the real data to train the predictive model, and another using the virtual cohort.
In both cases we used: (1) Gradient Boosted Trees (using the LightGBM 3 python library), a commonly used modeling technique.
We measured the predictive performance of both models using the area under the ROC-curve (AUC-ROC) and compared the models. For high statistical fidelity between the real and virtual cohort, we expect the AUC of the predictive model built with the synthetic data to be similar to that built with the real.
For the GBM models, we further look at the feature importance metric, using SHAP (SHapley Additive exPlanations) values 4 , for our gradient boosted trees model, where we expect to see relative stability in feature ranking (and understanding that this can change due to the inherent randomness of the GBM algorithm). We use normalized Discounted Cumulative Gain 5 (nDCG) to measure the difference in ranking.

Discriminator AUC
Inspired by generative adversarial networks, another effective metric for multivariate fidelity is the "discriminator AUC." Concretely, we built a classification model trained to discriminate between patient records in the real data vs. the synthetic data; using ROC-AUC as a measure of performance for this discriminator model, a virtual cohort with high statistical fidelity to the real cohort will result in a ROC-AUC value for this discriminator model that is close to 0.5 (representing random decision classifier), whereas low fidelity is reflected with ROC-AUC values closer to 1.

UMAP-Based Dimensionality Reduction
UMAP 6 is a modern dimensionality reduction technique that is often used to understand clustering of points in high dimensional space. It is informative to inspect the 2D projection of real and synthetic data after transforming the high dimensional space into two dimensions via UMAP, and visualize the various clusters formed; specifically, we expect high fidelity synthetic data to demonstrate broad coverage, such that in every cluster for the real data, there is enough number of synthetic data points.

Distance-To-Closest-Row (DCR)
We demonstrate privacy of the synthetic data by measuring the distance of each synthetic record to its closest record in the real dataset. Concretely we visualize two distributions: (1) the distance between each synthetic record and its closest real record and (2) the distance between each real record and its closest real record (excluding itself of course). For this purpose, we implement a distance function between two records. Given two records X = (X 0 , X 1 , . . . , X p ) and Y = (Y 0 , Y 1 , . . . , Y p ), we bin event numeric variables into 25 bins (thus, numeric values turn into a categorical value), and compute the distance between X and Y is computed as: When DCR values are close to zero, that means synthetic records are similar to the original records resulting in reduced privacy protection. Importantly, the original data may already include significant similarity between records or even explicit duplicates, and so we visualize DCR of real-vs.-real and real-vs.-synthetic to fully understand any reduction in risk.

RESULTS
Descriptive measures of the set of key variables used in the analyses from the EPAD V1500 sample and the synthetic dataset are shown in Table 1.
We evaluated the synthetic data generated applying the Syntegra methodology to the EPAD V1500 dataset according to the instruments described before. Unless stated otherwise, in all the following experiments, the number of synthetic samples generated is identical to the number of samples in the real dataset, although synthetic data generation allows for generation of any desired number of synthetic records. In all visualizations (if appropriate) we use the color blue to represent real data and the color green to represent synthetic data.

Univariate Marginal Distributions
We generated univariate distributions for selected variables in the dataset. Figure 1 displays the univariate distributions for these selected variables in the dataset; a summary of the KS-statistic (for categorical variables) and KL-divergence (for numeric variables) are shown in Tables 1, 2.

Pairwise Variable Correlation
For the sake of brevity, we present pairwise correlations in graphical form. Figure 2 displays a heatmap of pairwise correlations for real data compared to the synthetic data.
The calculated PCD-L1 is 0.0367, a value that indicates small average difference in the correlation values between real and synthetic, and thus high statistical fidelity.

Predictive Model Performance
We trained a gradient boosted trees classifier to predict ABeta_1_42 < 1,000, which is an indicator for dementia. We tuned the model parameters to achieve the best model performance using 50 iterations of hyper parameter tuning. The resulting Receiver Operating Curve (ROC) and area under the ROC (or C-statistic) are in Figure 3. The ROC curves are demonstrating similar characteristics; we ran this analysis multiple times with varying random initialization each time, resulting in • For GBM: AUC for real data is 0.6385 ± 0.0109 and for synthetic is 0.6745 ± 0.0096 • For Logistic regression: AUC for real data is 0.7232 ± 0.0034 and for synthetic is 0.7032 ± 0.0028.

Feature Importance Ranking
For the Gradient boosted trees model, we compared the feature importance ranking, using SHAP values, of our real model vs. the feature ranking of the synthetic model, as shown in Figure 4. The nDCG metric comparing these two feature rankings is 0.9244.

Discriminator AUC
The calculated discriminator AUC, as described in the "Methods" section above resulted in 0.5320 ± 0.0339 which is close to the desired level of 0.5, reflecting good statistical match between real and synthetic.   In calculating the UMAP visualization, we used the open source UMAP python package umap-learn 7 , with n_neighbors = 7 https://pypi.org/project/umap-learn/ 50 and min_dist = 0.1. Importantly, umap is trained on the real data and both real and synthetic data is transformed to the 2D dimension using the same learned model.

UMAP Dimensionality Reduction
Distance-To-Closest-Row (DCR) Figure 6 also depicts the DCR distribution of the real vs. synthetic data. The ratio between mean-dcr values for the real and synthetic data provides a quantitative measure of privacy. For this dataset the ratio was: 0.9995 which is excellent, although we do see a slight shift to the left in the synthetic distribution as compared to the real.

DISCUSSION
We demonstrated the use of synthetic data generated using the Syntegra methodology to produce a virtual cohort of nonidentical digital records that preserve the statistical properties of the EPAD V1500 dataset. We showed the fidelity of synthetic data generated, reporting a series of commonly used indices that consistently revealed high degree of similarity at the individual level data between the original dataset and the virtual cohort at the individual level data.
Our work, albeit being an application of modern computational approaches without aiming to answer any specific clinically relevant question, is a clear illustration of the advantages that virtual cohorts via synthetic data generation offer to the research community. Virtual cohorts represent quick and cost-effective opportunities to advance research, overcoming several of the existing hurdles that slow down or even impede scientific developments. For instance, by using virtual cohorts researchers would avoid lengthy and time-consuming data transfers, which would become unnecessary and obsolete, whilst having access to data that preserves the properties of the original datasets of interest whilst also preserving confidentiality and data privacy.
Notably, the generation of virtual cohorts by demand, that is, the generation of tailored synthetic datasets generated for answering specific questions, is another advantage of this technique. However, its feasibility needs to be further explored and may require appropriate documentation registering all steps taken for later external data quality examinations.
The aim of our work was to illustrate, via a specific example, how an advanced synthetic data generation technique could produce a dataset that preserves all features of the true cohort. We show the similarities between the real and the synthetic cohort in a cross-sectional setting. Although we do not demonstrate the use of virtual data in longitudinal designs, techniques exist that permit similar work in these more complex designs, and the recent release by Aridhia and EPAD of the EPAD V.IMI, that contains longitudinal data for over 2,000 participants, can be used for this purpose.
In the very near future, researchers developing and using virtual cohorts in research areas such as dementia research will need to engage with regulatory institutions and industry to explore their systematic use and acceptance of results generated using virtual cohorts. Most importantly, the community still needs to engage with participants to better understand their view regarding the use of their data for the implementation of this approach. Clearly, efforts to continue collecting data need to continue and be further supported as new datasets will always be necessary to answer questions that emerge as scientific knowledge advances. We are not advocating for stopping data collections of new data. Instead, we advocate for the exploration of new opportunities that virtual cohorts offer to maximize the utilization of existing datasets whilst overcoming some of the hurdles that currently disrupts science progression.

DATA AVAILABILITY STATEMENT
The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: Synthetic data: https://doi.org/10.34688/fpv7-b321 EPAD data: http://ep-ad.org/erap/.

ETHICS STATEMENT
The studies involving human participants were reviewed and approved by NHS Health Research Authority. For details see: https://www.hra.nhs.uk/planning-and-improvingresearch/application-summaries/research-summaries/epadlongitudinal-cohort-study/. The patients/participants provided their written informed consent to participate in this study.

AUTHOR CONTRIBUTIONS
GM-T drafted the paper and contributed to the discussions on results. OM and ML performed the analysis and contributed to the manuscript. RB contributed to the manuscript and oversaw the project. All authors contributed to the article and approved the submitted version.

FUNDING
The project was supported financially by the COVID-19 Alliance as technical capacity building.


Non-Parametric Online Learning from Human Feedback for Neural Machine Translation

We study the problem of online learning with human feedback in the human-in-the-loop machine translation, in which the human translators revise the machine-generated translations and then the corrected translations are used to improve the neural machine translation (NMT) system. However, previous methods require online model updating or additional translation memory networks to achieve high-quality performance, making them inflexible and inefficient in practice. In this paper, we propose a novel non-parametric online learning method without changing the model structure. This approach introduces two k-nearest-neighbor (knn) modules: one module memorizes the human feedback, which is the correct sentences provided by human translators, while the other balances the usage of the history human feedback and original NMT models adaptively. Experiments conducted on EMEA and JRC-Acquis benchmarks demonstrate that our proposed method obtains substantial improvements on translation accuracy and achieves better adaptation performance with less repeating human correction operations.

Introduction
The quality of the neural machine translation (NMT) system has been significantly improved recently (Sennrich, Haddow, and Birch 2016a;Vaswani et al. 2017;Zhang et al. 2018b;Hassan et al. 2018). However, recent research has shown that machine translation still lags behind human parity on translation quality (Läubli, Sennrich, and Volk 2018;Freitag et al. 2021). In some scenarios where highquality translation should be guaranteed, human-in-the-loop machine translation, i.e., machine translation with human post-editing, is still indispensable. As human corrections on machine-translated text are constantly produced, adapting the NMT model to these corrections could improve translation quality and effectively reduce human efforts, as shown in Figure 1. However, the training of NMT models requires a certain amount of data and consumes dozens or hundreds of GPU hours, which is unavailable in this scenario. Therefore, online learning from human feedback at a low training cost has become a promising research topic in recent years.
A series of studies (Turchi et al. 2017;Kothur, Knowles, and Koehn 2018) propose to on-the-fly update the NMT Figure 1: The workflow of online learning from human feedback in the human-in-the-loop machine translation. Given the source sentence SRC, the NMT System generates the translation MT. The user corrects it and produces PE. Then the corrected sample is used to adapt the NMT system. model right after humans correct the translation. Although these methods improve translation performance, they require extra costs for frequent gradient computations at the inference time. Tuning parameters only on the new-coming sentences also brings the risk of catastrophic forgetting problems. Another research line (Gu et al. 2018;Zhang et al. 2018a) augments the NMT model with the translation memory to cache the human-corrected sentences. For each sentence to translate, cached sentences with similar context are retrieved to help the decoding process of the NMT model. Memory-augmented NMT avoids updating parameters onthe-fly, but it requires careful network architecture design to model the retrieved translation memories. Also, extra training overhead is inevitable to optimize the memory network.
Recently, Khandelwal et al. (2020) proposed KNN-MT, a non-parametric and model-agnostic method for domain adaptation. It equips the pre-trained NMT model with a knearest-neighbor classifier over a datastore of cached context representations and corresponding target tokens. The final translation probability is the interpolation between the probability calculated by the NMT model and the KNN module. KNN-MT provides a simple but effective way to exploit human feedback by adding human-corrected translations into the datastore. However, the success of KNN-MT is owed to enough in-domain samples pre-cached in the datastore. In the human-in-the-loop translation, to learn the cor-rected translation, human translators have to make repetitive corrections on the same mistake to provide enough samples for KNN-MT. Biasing the interpolation to the KNN module could accelerate this learning process, but it takes the risk of being interfered by the noise in retrieved neighbors when retrieved items are not relevant enough. Therefore, it is crucial to balance the usage of the probability from the NMT model and the KNN module when applying KNN-MT.
In this paper, we propose KNN-over-KNN (KoK), a plugand-play non-parametric approach for online learning from human feedback. KoK equips the NMT model with two KNN modules, Token-KNN and Policy-KNN. Token-KNN is used to model the translation probability as in KNN-MT. Human-corrected sentences are incrementally added to the datastore of Token-KNN to improve the translation quality of proceeding sentences. Policy-KNN is introduced to model the balance mechanism between the Token-KNN module and the NMT model. It interpolates between the two modules with a Bernoulli distribution. The distribution is estimated by a k-nearest-neighbor classifier over a datastore. For building the datastore, we extract features from the retrieval results of Token-KNN as the key, and heuristically induce a binary flag for each result as the value (1 means using this result and vice versa).
We conduct experiments on translating domain-specific documents with various lengths. Oracle references are used to mimic human feedback. KoK obtains significant improvements (up to 12.9 BLEU improvement and 9.7 TER reduction) compared to the Pre-Trained NMT model. KoK also achieves consistent improvements over existing online learning methods on documents at all lengths. We further show that KoK can adapt to human feedback with less repeated corrections, which reduces human effort. Our code is open-sourced at https://github.com/wangqi1996/KoK.

Background Neural Machine translation
Neural machine translation systems formulate the translation task as a conditional probability model p(y|x), which defines the translation process from source sentence x = {x 1 , x 2 , ..., x m } into target sentence y = {y 1 , y 2 , ..., y n }. The representative works decompose it in an auto-regressive manner from left to right : where y <t = {y 1 , y 2 , ..., y t−1 } denotes the prefix tokens. Then the probability of each target token is defined as: where h t is the representation for context (x, y <t ), and W , b are the trainable parameters to mapping the dimension of h t to the vocab size.

Online Learning from Human Feedback
In the human-in-the-loop translation, a complete translation step is: the NMT model generates machine-translated text, and then the human translator makes revisions on it (human feedback). Online Learning from human feedback follows such a paradigm: after the human completes the revision of the current sentence, the machine translation system is adapted incrementally by taking this sample into account. Concretely, given a document D = {x 1 , x 2 , .., x |D| }, where x i represents i th sentence, the translation for x i is generated asŷ is the NMT model adapted by x <i sentences and f 0 is the pre-trained model. The human translator correctsŷ i , and then produces the reference y i . The model f i is acquired by adapting f i−1 over the new sample (x i , y i ): The adaptation process could be tuning the model parameters, or updating the external translation memory. In the rest of the paper, we omit the superscript i for simplicity.

KNN-MT
Recently, Khandelwal et al. (2020) proposed KNN-MT, a non-parametric method. It augments the NMT model with a token-level k-nearest-neighbor retrieval mechanism, allowing it to directly access the cached examples stored in the datastore during inference. It consists of two steps: datastore construction and inference.

Datastore Construction
The datastore consists of a set of key-value pairs. More specifically, given the parallel sentence in the dataset (x, y) ∈ (X , Y), the pre-trained NMT model generates the context representation h t as Equation 1 at every timestep t, then the datastore takes the h t as the key and the corresponding target tokens y t as the value: (K, V) = {(h t , y t ), ∀y t ∈ y|(x, y) ∈ (X , Y)}. Inference The inference process performs in a token-bytoken manner. Given the source sentence x and the generated target prefixŷ <t , the pre-trained NMT model generates the representationĥ t and then predicts a probability p NMT (ŷ t |x,ŷ <t ) for the target tokenŷ t .
Then, the k-nearest-neighbor model retrieves the similar K neighbours R = {(k i , v i ), i ∈ {1, 2, ..., K}} in the datastore according to Euclidean distance toĥ t . The retrieved result is converted into a distribution over the vocabulary by where d(k i ,ĥ t ) represents the Euclidean distance between k i andĥ t , p(k i |ĥ t ) is the probability of i th retrieval neighbor (k i , v i ). T is a hyper-parameter to control the sharpness of the softmax function. The final output distribution is an interpolation between distributions from the NMT model and the KNN retrieved neighbors with a tuned parameter λ ∈ [0, 1] p(ŷ t |x,ŷ <t ) = λ * p KNN (ŷ t |x,ŷ <t )

Amoxil is an
Amasilin is an antibiotic.
Amoxil is an antibiotic.
value key Amoxil antibiotic … Amasilin ist ein Antibiotikum. Amoxil is an antibiotic xŷ t y <t Figure 2: An illustration of proposed KoK.

Proposed Method
In this section, we describe the proposed method, KNNover-KNN (KoK). Based on KNN-MT, we model a balance mechanism between the usage of the pre-trained NMT model and the KNN module cached history human corrections, i.e., we replace the fixed λ in Equation 3 into a dynamic λ t varies at each predictingŷ t : KoK learns to predict p KNN and λ t from history human feedback in a non-parametric way, and can be applied to NMT models with various architectures.
Architecture Figure 2 shows the overall architecture of KoK. It parameterizes the final translation probability (Equation 4) with two KNN modules besides the pre-trained NMT model, in which the Token-KNN module predicts the probability p KNN over target words and the Policy-KNN module predicts an interpolation coefficient λ t . During the translation process of one sentence, for each decoding step t, Token-KNN takes the hidden representation h t as query and retrieves from its datastore to generate the translation probability p KNN . Then features extracted from the retrieval result of Token-KNN are used as the query of Policy-KNN to calculate the λ t value. The final translation probability (Equation 4) is computed as an interpolation between p KNN and p NMT with λ t .

Token-KNN
Token-KNN module predicts the translation probability p KNN with k-nearest-neighbor classifier over the datastore. It is similar to the KNN module in the KNN-MT, but the datastore is built incrementally. During the translation process, with theĥ t as the query, the Token-KNN queries from the token datastore. The translation probability is computed as Equation 2. During the adaptation process, given source sentence x and its human-corrected translation y, we build the key-values pairs and then store them into the datastore.

Policy-KNN
Policy-KNN module calculates the λ t , which expresses the balance mechanism between the usage of the pre-trained NMT model and the Token-KNN module. We introduce a random variable λ which obeys the Bernoulli distribution. λ = 1 represents to use the translation probability of Token-KNN and vice versa for the NMT model. Thus, predicting coefficient λ t in Equation 4 is demonstrated as estimating the conditional probability p(λ = 1|x, y <t ) Policy-KNN estimates this probability with k-nearestneighbor classifier over the datastore as well. In the following, we will describe in detail the datastore construction and inference process of Policy-KNN. At the t th decoding step, we denote the retrieval results from Token-KNN as Zheng et al. (2021a), we construct the key vector s t using two kinds of features extracted from the retrieval results of Token-KNN. One feature is the distance, we denote the Euclidean distance between the context rep- The other feature is the counts of distinct values for all retrieval results, marked as c i . Specially, c i is computed as: where UNIQUE represents the unique elements in the list. s t is constructed by concatenating all the features: Intuitively speaking, the distance features d i stand for similarity, which directly evaluates the importance of each retrieval result. The count features c i represents the consistency of retrieval results, as more distinct retrieved values mean less credible KNN retrieval. Both of these two features are crucial for predicting an optimal value of λ.
We further re-weight the key vector according to the significance of each element. We adopt exponential reweighting on distance features and count features, respectively: Value The retrieval result of Token-KNN does not explicitly come with a golden value of λ indicating whether to use this result or not. Instead, we implicitly induce the value of λ through a simple heuristic: where x and y is the source sentence and its humancorrected translation, respectively.
This heuristic is actually mathematically meaningful. To find the value of λ t to maximize the probability of KNN-MT (Equation 4), we need to solve the optimization problem below: Inference During the translation process, at the t th decoding step, given the retrieval result R tok t of Token-KNN, we build the query featureŝ t as in Equation 6, then Policy- .., K}}, from its datastore. The retrieval result λ t is calculated by: The final translation probability is calculated as Equation 4.

Implementation
Overall, KoK is performed as an online learning paradigm. It takes three steps when translating a source sentence x:

Experiments Experimental Setup
We evaluate the proposed method by equipping the generaldomain pre-trained NMT model with KoK and use it to translated in-domain documents with various document sizes. We use the oracle reference as the human-corrected translations to simulate the real human-in-the-loop scenario.  (Tiedemann 2009), which consists of sentencealigned documents focusing on medical products.
(2) JRC-Acquis corpus (Steinberger et al. 2006), which contains the European Union laws applicable to the EU member states.
Following common practices, we use the Moses toolkit 2 to tokenize the documents, remove the same sentences in a document and then segment the words into subword units (Sennrich, Haddow, and Birch 2016b) with the bpecodes provided by the pre-trained model. We validate our method on documents with various lengths. Specifically, we divide the documents into five buckets based on their length (0-50, 50-100, 100-200, 200-500 and 500-1000). We randomly select some documents so that the total document length of each bucket is upper than 1000. Detailed for EMEA/JRC dataset statistics are shown in Table 1 (2018), we train the model 5 steps on every single sentence with the adam optimizer. The hyper-parameters setting follows the pretrained models, except the learning rate is set to 1.0 × 10 −7 and fixed during the inference process. • KNN-MT: After every sentence is translated, we add the human-corrected sentences to the datastore of KNN-MT. Similar to our method, the K is 8. The λ value for the EMEA dataset and the JRC-Acquis dataset are 0.2 and 0.3, respectively. • Adaptive KNN-MT (Zheng et al. 2021a): A variant of KNN-MT, which introduces a network to dynamically determine the number of K for each target token. We use the model pre-trained on the IT domain (provided by the paper) and incrementally update the datastore as in the KNN-MT.
Evaluation. We use SacreBLEU 5 (Post 2018) to measure the result with case-sensitive detokenized BLEU (Papineni et al. 2002). We concatenate the translations of all the documents in each bucket and calculate the corpus-level BLEU. For the completeness of the results, we also report the TER (Snover et al. 2006) metric to compute the edit distance between the reference and system translation.

Main Results
We can see that consistent improvements (up to 12.9 BLEU improvement and 9.7 TER reduction) are achieved by KoK on different document lengths. As a comparison, Online Tuning achieves relatively minor improvement on long documents, probably because that several thousand samples are still not enough for a gradient-based approach. KNN-MT fails to attain similar improvements as KoK on long documents because of the small but fixed λ value. This indicates the benefit of adjusting λ adaptively. Instead, the per-5 https://github.com/mjpost/sacrebleu formance of adaptive KNN-MT deteriorates when the length of a document is relatively small. As the training of the adaptive KNN-MT is based on a static datastore built from all indomain training data, it is not compatible with the human-inthe-loop machine translation scenario that the human feedback comes in a streaming way. Overall, the results demonstrate the effectiveness and generalization of our method on different document conditions. We also conduct experiments on the JRC-Acquis dataset, which contains documents related to the law domain. All the results are listed in Table 3. KoK consistently outperforms all the baselines, which demonstrates that our approach can be generalized to other domains. We show the performance on the full test sets in Appendix.

Comparing With Translation Memory
We also compare our method with the NMT model augmented by the translation memory (TM-NMT) (Kuang et al. 2020;Bapna and Firat 2019). Due to the training data of the WMT19 NMT model is unavailable, we implement the TM-NMT model and the NMT model of KoK on the WMT14 DE-EN dataset for a fair comparison, and evaluate them on the EMEA dataset. For the NMT model, we use the transformer big architecture, and the training detail follows the introduction of fairseq 6 . For the TM-NMT, we follow the source similarity method in Cai et al. (2021), and the architecture is the same as the transformer big model.
The result is illustrated in Table 4. As seen, KoK still achieves improvements on all the document sizes. However, TM-NMT fails to exceed the pretrained NMT model except on the 200-500 bucket, which may be resulted from the inability to handle the low quality of retrieval memories (Cai et al. 2021).   Table 5: Decoding latency of different models on the EMEA dataset. We also list the speedup w.r.t the pre-trained NMT model.

Latency
In the human-in-the-loop translation scenario, human translators are sensitive to decoding latency. Therefore, we also evaluate the latency of the proposed KoK. We simulate the Computer Aided Translation(CAT) task setting and measure the decoding latency sentence-by-sentence 7 . The latency is calculated on the EMEA dataset, and we report the average latency in milliseconds. The speed-up ratio is also computed by comparing it with the Pre-Train Model following the previous practice (Zheng et al. 2021a). The result is summarized in Table 5. Our method slightly slows down the inference time, but significantly improves translation quality, which is tolerable for users.

Analysis
In this section, we perform several analyses and ablation studies for the proposed KoK.

Can KoK Learn Optimal λ
KoK equips KNN-MT with a Policy-KNN to adaptively decide between adopting knowledge from human feedback or keeping decoding results from the pre-trained NMT model on every token. In this section, we demonstrate that KoK indeed learns an optimal policy to make the decision. We first compare KoK with KNN-MT which chooses different λ values. The results are shown in Figure 3. We try λ ∈ [0.0, 1.0) with 0.1 as the step size, and report average BLEU scores on the various document length buckets on the EMEA dataset. KNN-MT is sensitive to the choice of λ value (BLEU varies from 18 to 43), which indicates the importance of selecting a proper λ value. KNN-MT achieves the best performance when λ is around 0.2, but still lags behind KoK. This experiment shows that it is crucial to decide the λ value token-by-token adaptively. 7 The latency is measured on a single GeForce GTX 1080-Ti GPU.
We further evaluate whether KoK could learn an optimal λ. Given oracle target y, for each y t ∈ y, we compute the Token-KNN probability of y t as p knn (y t ), and the corresponding λ t predicted by Policy-KNN. To show the relationship between λ t and p knn (y t ), we divide all the Token-KNN probability values into five buckets and compute the average λ values in every bucket. The result is shown in Figure 4. As seen, the λ value becomes high when the Token-KNN probability gets bigger, and vice versa. This is in line with the motivation that KoK should bias to the usage of Token-KNN probability when retrieved human feedbacks indeed help the translation. We also evaluate the effectiveness of K in the Appendix.

Zero-Shot and Few-Shot Ability of KoK
In machine translation with the human-in-the-loop scenario, users not only care about the quality of the translation model, but also care about how fast the model can learn from their feedback to avoid repeatedly making the same mistakes they already corrected before. Therefore, in this section, we evaluate the adaptation speed of KoK to user feedback.
We follow the R-indicator used in Simianer, Wuebker, and DeNero (2019), which measures the translation recall of words with different occurrence times in users' feedback. We denote R i as the recall of tokens that have appeared i times in the previous corrected sentences: where H j represents unique words in the j th machinegenerated translation and R i,j represents unique words in the j th reference that are their (i + 1) th occurrence in the whole document. Specifically, R 0 evaluates the tokens that first appear in the translating document and R 1 considers those that have appeared once.
We conduct experiments on documents with [200, 500] bucket from EMEA. R 0 , R 1 , R 2∼5 , R 5∼9 and R 9+ are computed for KoK and other baselines as Equation 9 respectively (R m∼n is defined as the micro average of R m , R m+1 , ..., R n ). The results are shown in Figure 5. KoK achieves the comparable R 0 , but the R i values improve quickly and outperform all the other methods. It indicates KoK's ability to reduce interference from unrelated retrieval results of Token-KNN, and to adapt to helpful human feedback faster. Adaptive KNN-MT achieves a lower R 0 , which may due to the same reason about worse performance in main experiments.   Figure 4: Average λ in different ranges of reference Token-KNN probability. We compute the Token-KNN probability and its corresponding λ value by Policy-KNN for each words in the reference.

Related Work
Online Learning from Human Feedback. Online updating (Álvaro Peris and Casacuberta 2019; Turchi et al. 2017;Weng et al. 2019) the NMT model has been proved to be an effective and efficient method to adapt the model to the target domain. Kothur, Knowles, and Koehn (2018) proposed the single-sentence adaptation, which performed online adaptation one sentence at a time. Simultaneously, they introduced the dictionary adaptation to address the problem of translating novel words. Karimova, Simianer, and Riezler (2018); Domingo et al. (2019bDomingo et al. ( , 2020 presented a user study to prove the advantages of online adaptation. Domingo et al. (2019a) applied online adaptation technology in a production environment. Mueller and Lal (2019) proposed to finetune the model over a subset of similar sentences extracted from the training set. However, these methods require updating the model frequently, which brings a non-negligible computation cost in the human-in-the-loop translation scenario, and is prone to catastrophic forgetting problems.
1 Figure 5: Results on documents with [200, 500] bucket from EMEA dataset for the proposed R-indicator.
(2018) proposed to retrieve samples similar to the source sentence and then encoded retrieved source-target pairs using key-value memory networks. Cai et al. (2021) proposed to store the target-language sentence in the memory and used a cross-lingual retrieval model. Cao and Xiong (2018) Khandelwal et al. (2020) generated the translation with the nearest neighbor classifier over a large datastore of cached examples. Zheng et al. (2021a) proposed to dynamically determine the number of K for each target token. Zheng et al. (2021b) further extended the capability of KNN-MT on unsupervised domain adaptation setting.

Conclusion
In this paper, we propose KNN-over-KNN (KoK), a plugand-play non-parametric method for online learning from human feedback. KoK introduces two KNN modules, one to memorize the corrected sentences by the human and the other to balance the importance of the in-domain corrected sentences and the general-domain pretrain NMT model. In the experiments, our method significantly improves the performance and achieves better performance than state-of-theart baselines on one-shot or few-shot learning for domainspecific lexical items. As KoK achieves competitive performance compared with existing online learning methods, applying our approach to other generation tasks is a promising direction for future work.

The Creating/Updating Process of Datastore
At the beginning of the inference process, the datastore is empty. For each source sentence x and the human corrected sentence y, the adaptation process is: • The pre-trained model generates the hidden representation h by teacher forcing. • With h as the query, the retrieval result of Token-KNN is R tok . • The items for the datastore of Policy-KNN are extracted from R tok as described in the subsection Policy-KNN in page 3. • The items for the datastore of Token-KNN are h and the corresponding target token y. • These items are added into the datastores of Token-KNN and Policy-KNN respectively.
The KNN models are non-parametric and don't need pretraining. We will make it clearer in the next version.

The Performance on The Full Test Sets
We calculate the average BLEU score over the buckets in the main paper, and re-calculate the BLEU scores on the full test sets. The scores are listed in Table 1. We will add these results in the next version.