Re_Trans: Combined Retrieval and Transformer Model for Source Code Summarization

Source code summarization (SCS) is a natural language description of source code functionality. It can help developers understand programs and maintain software efficiently. Retrieval-based methods generate SCS by reorganizing terms selected from source code or use SCS of similar code snippets. Generative methods generate SCS via attentional encoder–decoder architecture. However, a generative method can generate SCS for any code, but sometimes the accuracy is still far from expectation (due to the lack of numerous high-quality training sets). A retrieval-based method is considered to have a higher accurac, but usually fails to generate SCS for a source code in the absence of a similar candidate in the database. In order to effectively combine the advantages of retrieval-based methods and generative methods, we propose a new method: Re_Trans. For a given code, we first utilize the retrieval-based method to obtain its most similar code with regard to sematic and corresponding SCS (S_RM). Then, we input the given code and similar code into the trained discriminator. If the discriminator outputs onr, we take S_RM as the result; otherwise, we utilize the generate model, transformer, to generate the given code’ SCS. Particularly, we use AST-augmented (AbstractSyntax Tree) and code sequence-augmented information to make the source code semantic extraction more complete. Furthermore, we build a new SCS retrieval library through the public dataset. We evaluate our method on a dataset of 2.1 million Java code-comment pairs, and experimental results show improvement over the state-of-the-art (SOTA) benchmarks, which demonstrates the effectiveness and efficiency of our method.

Introduction
Source code summarization (SCS), also named code comment, is a term coined by Haiduc et al. [1]. It is a natural language description of programming fragments. Program maintenance is the most expensive and time-consuming stage in the software life cycle [2]. High-quality SCS is essential to program comprehension and maintenance, which can help developers save time spent on navigating source code and understand programs quickly. Unfortunately, with the rapid update of software, most SCS is mismatched, outdated, and missing. Hence, SCS generation has been researched extensively and has made lots of remarkable achievements [3][4][5][6][7][8][9][10][11][12][13].
SCS generation is a hot field that emerged more than a decade ago. Its methods can be divided into three categories: manually-crafted template, information retrieval-based (IR-based), and deep-learning-based (DL-based). The manual template methods usually extract keywords from source code to generate SCS [4,14,15]. However, they miss a lot of potential information of the source code. The IR-based methods are widely used in SCS generation. They usually generate SCS by searching keywords from the given code or code comments of the code that are most similar to the given code. For example, Haiduc et al. [1,3] analyzed source code using the vector space model (VSM) and latent semantic indexing (LSI) methods, producing natural language description of the classes METEOR, and ROUGE) in code summarization tasks. Furthermore, we also perform time-consuming experiments to confirm the efficiency of our method.
The main contributions of this paper are as follows: • We propose a Re_Trans system by combining retrieval and generative methods and adopt the suitable SCS generation model for a given source code. • We use non-leaf nodes of the AST to build a directed graph and enhances the edge information thought data flow and control flow. To the best of our knowledge, this is the first time that such an efficient structure representation mode has been used in an SCS task. • We perform extensive experiments on a public real-world dataset. All results confirm that the Re_Trans is effective and outperforms the SOTA methods.

Overview
The workflow of our proposed method (Re_Trans) is shown in Figure 1. Re_Trans mainly contains three steps: (1) Data representation (see Section 2.2): Re_Trans parses the source code into AST and source code sequence and processes code summarization by a plain text that is composed of tokens (i.e., variables). (2) Model training design (see Section 2.3); Re_Trans includes a retrieval-based model, a generative model, and a discriminator (see Section 2.5). (3) Model test design (see Section 2.4).

Data Processing
In this paper, we use large public dataset <Java code, comment> pairs. Our data processing method is available in various programing languages. We represent the Java code as parsed AST and code sequence and process comments into plain text.
For one sample, we show the source code structure information in Figure 2. Initially, we use the javalang (https://github.com/kangyujian/JavaMethodExactor, accessed on 20 August 2020) toolkit to parse source code into an AST and remove the leaf nodes of AST. There are two reasons for removing the leaf nodes: (1) To avoid repeated processing because the leaf nodes correspond to the source code text, which has been processed in code sequence information. (2) Non-leaf nodes represent the source code structure information to a certain extent, and this structure saves much traversal time. Furthermore, we enhance source code semantic information by adding data flow and control flow to AST referring to the work of Wang et al. [28]. Considering our AST without leaf nodes, in data flow information, we connect a node to its next brother node (from left to right). It solves the problem that graph neural networks do not consider the order of nodes. For example, the green arrows in the red dotted box (aa) in Figure 2 connect three sibling nodes of "Modifier", "FormaParameter", and "ForStatement". The added edges are (Modifier, FormaParameter), (FormaParameter, and ForStatement). In control flow information, we select "IfStatement", "WhileStatement", "ForStatement", and "BlockStatement" nodes. "BlockStatement" is the root node of the code block when executing source code sequentially. According to the characteristics of each node, we connect their child nodes to form new edges.
Finally, we utilize depth-first traversal to obtain the edge set of the directed AST. The edge set is e = (e 1 , e 2 . . . e n ), where n is the number of edges. Compared with undirected AST, directed AST can represent the sequence structure information more accurately. We take the edge information as an initialization vector and input it into GCN for semantic extraction of the source code.
The source code sequence information is shown in Figure 3. Firstly, we treat each source code as a plain text (as shown in the blue box); each word corresponds to a unique identifier (token id) by dictionary mapping. Then, we add row and column position information to the source code. For the row position information, we assign integer values starting from zero to each row of Java function sequentially (as shown in the red box). For the column position information, we assign integer values starting from zero in the word order of each code (as shown in the green box). Hence, each word in the source code sample has three feathers: token id, column position, and row position. We concatenate the initialized vectors of these features as the initial vector for each word. In this way, we complete the vector initialization of the source code and utilize the self-attention mechanism to obtain the source code sequence sematic vector. We take "static" as an example. From Figure 3, we can see that its token id is 35, column position is 1, and row position is 0. Firstly, we perform vector initialization on 35, 0, and 1, respectively, and obtain the corresponding embeddings Emb 1 , Emb 2 , and Emb 3 . Secondly, we concatenate the three embeddings to obtain the initial vector of "static", Emb = cat(Emb 1 , Emb 2 , Emb 3 ). At last, we input Emb into the self-attention mechanism to obtain the final semantic vector of "static". Let the word dimension be d, then Emb 1 , Emb 2 , Emb 3 ∈ R 1×d , Emb ∈ R 1×3d .

Model Training Design
Re_Trans contains two SCS generation models: retrieval model and transformer. We    In the retrieval model, when comparing the codes' plain texts, it is difficult to judge whether they are similar because different programming texts may implement the same function. For example, both "forstatement" and "whilestatement" can implement the loop function. The semantic similarity measures the difference between tokens based on the similarity of their meaning or semantic content rather than the similarity of dictionaries. It often uses statistical methods, such as vector space models, to associate words and textual contexts from corpora. Therefore, we compare the code' semantic similarity in the retrieval-based model. For a sample, we use GCN to process the enhanced AST and BiGRU to deal with the enhanced code sequence. The semantic vector of a sample is the results concatenation of GCN and BiGRU, namely Emb_S. We use the n-dimensional Euclidean distance formula that is shown as (1) to find the code in the retrieval library that is most similar to the sample, namely Emb_Simi, and its SCS. The construction process of the retrieval library will be detailed in Section 3.1. 2 (1) where Emb_S = (x 1 ,x 2 , . . . ,x n ), Emb_Simi = (y 1 ,y 2 , . . . ,y n ), and i ∈ [1, n] The goal of the transformer model is to generate a new SCS for each input function. For a sample, we also use GCN to extract its structure information. As for the syntactic information, we directly utilize the transformer's encoder. The sample semantic vector is the results concatenation of GCN and the self-attention mechanism. We use the transformer's decoder to convert the sample semantic vector to its SCS.

Model Test Design
In this section, we show the test flow in Figure 6. To make it easier to understand, we introduce the test flow through a running example that is shown in Figure 7. We first input the Java code (a) into the retrieval-based model introduced in Section 2.3 and obtain the semantic vector of Java code (b) and target summarization (b). Even more, (b) is the most similar Java code to (a). Then, we input the semantic vectors of (a) and (b) into the trained discriminator, and the sim_label is one, so we obtain the target summarization (b) as the final summarization result, which means it "generates the most likely state predictions for the sequence". Specifically, the trained discriminator's training process is detailed in Section 2.5.

Discriminator
In particular, it is stated that the role of discriminator in this paper is different from that of the discriminator in the adversarial generative network. The purpose of our proposed discriminator is to judge whether the Java code summarization produced by the retrieval model is optimal, which is used in both training and test phases. Furthermore, the discriminator's working approach is detailed in its training and test process.
In order to train the discriminator, we randomly sample 200,000 samples from the dataset in Section 3.1 and divide them into training and testing sets in a ratio of 8:2. The training process is shown in Figure 8. First, we assign the label to all samples. For a sample: (1) We use the retrieval model in Section 2.3 to obtain its S_code and S_RM and use the transformer model to generate its SCS (S_GM). 2) When analyzing the similarity between two feature vectors, the cosine similarity can avoid the large distance caused by different sequence lengths and only consider the angle between two vectors. Therefore, we utilize the cosine distance to calculate the similarity between S_RM, S_GM, and the target SCS (T_SCS), respectively, which are shown in Formulas (2) and (3), where S_RM = (s_rm 1 , s_rm 2 , . . . , s_rm m ), T_SCS = (c 1 , c 2 , . . . , c m ), S_GM = (s_gm 1 , s_gm 2 , . . . , s_gm m ), and m represents the dimension of code summarization. If S_RM is better, we set the sample as <code, S_code, 1>. Otherwise, the sample is <code, S_code, 0>. We repeat these two steps, and we set all discriminator data in the form of <code, S_code, label>, where the label represents zero or one.
Second, we train the discriminator with 160,000 samples, and the parameters are shown in Section 3.2. Especially, we utilize Multilayer Perceptron (MLP) to calculate the semantic similarity between SCSs. Finally, we use the remaining 40,000 samples to test the trained discriminator.

Dataset Analysis
The dataset contains around 2.1 million <Java code, comment> pairs [29], which are widely used in lots of SCS generation tasks [10,20,30]. We analyzed the dataset from two aspects: (1) statistical length distribution of source codes and their comments (see Figure 9); and (2) a count of the scale of Java code numbers with the same comment (see Figure 10).   From Table 1 and Figure 9, we can see that the comment length distribution is relatively uniform, ranging from 3 to 13. A short comment helps people understand the code function quickly. The code lengths are distributed between 1 and 100, which is approximately normal distribution. When code length is larger than 70, the number is almost unchanged, so we set 70 as the optimal input length parameter. In order to train a retrieval library, we remove invalid data whose comment corresponds to only one function. In Figure 10a, the function of scale two is close to 160,000, far exceeding the number of other scales. We denoise these functions and use them as a retrieval library. From Figure 10b, we find that the number of scales over 80 is almost 1.

Parameter Settings
In this section, we introduce the main parameter settings in all experiments as shown in Table A1. In the transformer model, we use the Adam optimizer and set epsilon to 1 × 10 −9 and (β 1 , β 2 ) to (0.9, 0.98). The parameters of thetransformer model are N = 4, h = 4, and dim = 256), where N is the number of encoder layers, h is the number of multi-headattention, and dim is the embedding dimension. Particularly, we use the NoamOpt to obtain the learning rate dynamically, where the warmup is 200 and the factor is 1. The batch_size is set to 256, and the epoch is 40. In the retrieval model, we also use the Adam optimizer, and set the GCN and BiGRU layer to two. Moreover, we set the leaning rate to 1 × 10 −4 , and the epoch to 30. We conduct all the experiments on a workstation with two Intel(R) Xeon(R) Gold 6154 CPU@3.00 GHz, 128 gb RAM, and two Titan XP GPUs. It is necessary to train on GPUs with 64gb VRAM due to the large size of our model and dataset.

Baselines
To demonstrate the effectiveness of our method, we compare it with the SOTA methods from recent years. The baselines are described as follows: LeClair et al. [10] (2019) proposed a method called ast-attendgru that was an attentional encoder-decoder architecture to generate SCS. Ast-attendgru enhanced the SBT and AST flattening procedure proposed by Hu et al. [7,23] and showed a higher performance. Hence, we only compare against this approach.
Xu et al. [31] (2018) proposed an approach called graph2seq which was a general neural encoder-decoder architecture that solved the graph-to-sequence problem. It achieved SOTA results on an SQL-natural language task using BLEU-4 metric. The open-source code of graph2seq is convenient to conduct comparative experiments.
LeClair et al. [24] (2020) adopted code+gnn+BiLSTM to generate SCS. Different from the flattened AST, they took the AST as a graph, and it performed well, which was closer to our method in terms of code structure information extraction.
Ahmad et al. [13] (2020) proposed the transformer-based method that was the first to apply the transformer model to source code comments. They incorporated relative positional encoding and copied an attention mechanism into the transformer model to improve the SCS quality. We also use the transformer model as the Re_Trans' generative model.
Zhang et al. [11] (2020) proposed a novel retrieval-based neural approach called Rencos. Rencos retrieved the two most similar code snippets in a given code from aspects of semantic and syntax, respectively. Rencos enhanced the accuracy of the SCS by fusing the retrieved results into the generative model.
Wei et al. [32] (2020) proposed an approach to enhance the SCS' accuracy, namely Re2Com. Similar to Rencos, Re2Com also used a retrieval-based method to enhance the SCS' accuracy. For a given code, Re2Com retrieved its most similar code and the SCS pair. Then it took the given code (its code text and AST sequence), the most similar code, and SCS as input to the encoder. Experiments demonstrated the effectiveness of this method.

Results and Analysis
Our research objective was to determine that the Re_Trans outperforms current baselines. We also wanted to demonstrate the efficiency of RGSGS and the effectiveness of the retrieval library built by us. Notably, all methods were trained on the dataset described in Section 3.1 and evaluated by the widely-used metrics BLEU [33], METEOR [34], and ROUGE [35], detailed in the Appendix A.2. The metrics' scores were in the range [0, 1] and reported in percentages in this paper. We answer the following research questions (RQs) to explore these situations: RQ1: What is the performance of Re_Trans compared to the baselines? RQ2: Why does the Re_Trans approach perform well? RQ3: Where is the high efficiency of Re_Trans reflected? RQ4: What is the quality of the SCS retrieval library we built?

Re_Trans vs. Baselines
Among baselines, ast-attendgru, graph2seq, code+gnn+BiLSTM, and transformerbased belong to GM. Rencos and Re2Com are methods that combine retrieval and generative techniques. According to the parameter settings in Table A1, we show the experimental results of Re_Trans and baselines in Table 2. From Table 2, we find that the effect of the Rencos method is relatively poor. The reason may be that it does not matter whether the retrieved similar code is actually similar to the input one or not. When taking them as input of the encoder, Rencos may produce biased SCS. Although ast-attendgru, Rencos, and Re2Com use flatted AST to represent code structural information, the AST sequence is a linear problem in nature. The effect of code+gnn+BiLSTM, transformer-based and Re2Com are close to Re_Trans. The Re_Trans and code+gnn+BiLSTM use a similar source code semantic extraction method, AST graph, and source code sequence, but the Re_Trans performs better. One reason is that our method enhances the AST and code sequence, which can extract the source code semantic information more fully (explained in Section 4.2). LeClair et al. [30] mentioned that the decoder with an attention mechanism is less effective than a transformer. We also find that the BLEU-N of the transformer-based method is on average about 7% higher than code+gnn+BiLSTM. The poor performance of graph2seq is because it only considers the structural information of source code but ignores the syntactic information in the SCS task.
In Table 2, the Re_Trans performs best; the main reasons are the following: (1) We combine the code sequence-augmented and AST-augmented to characterize source code.
(2) The generative model is the transformer model. Transformer is generally better than the seq2seq architecture in all tasks. In addition, the BLEU-N gradually decreases as the N increases. It shows that there is still a lot of room for improvement in the long sequence matching between the generated SCS and the target. Furthermore, it also indicates that the current SCS generation model still needs to be further studied and improved.

Ablation Study
An ablation study is often used to reduce some improved features on the model proposed in the paper in order to verify the necessity of corresponding improved features. Ablation is a very labor-saving way to study cause and effect. In this section, we will illustrate why the Re_Trans method works well through source code representation ablation experiments. The source code semantic representation methods mainly include source code sequence information (Seq), AST-augmented (ast_aug) combined with Seq, and ASTaugmented combined with code sequence-augmented information (Seq_aug). However, we also test the SCS effect of preserving leaf nodes of AST-augmented (ast_leaf_aug) combined with Seq_aug. For these different semantic extraction methods of source code, we use Re_Trans to generate SCS and show the ablation experiments results in Table 3. Table 3. The ablation experiment results (B-n represents BLEU-n). In Table 3, we find that the SCS quality has been improved after the Seq combining with the ast_aug information. It is because pure sequence information ignores the potential and complex structural information of source code, and ast_aug preserves the structural information of source code. As we all know, the self-attention mechanism ignores position information when encoding sequence information. Therefore, we add position information to source code sequences to solve this problem. As shown in Table 3, the effect is significantly improved when we use "ast_aug+Seq_aug" to represent source code. The BLEU-1 is improved by 10.2%, and BLEU-4 is improved by 9.9%. Moreover, we also find that the effect of "ast_leaf_aug+Seq_aug" is better than "ast_aug+Seq_aug", but the gap is slight. In Section 4.3, we demonstrate that the time efficiency of the latter is much higher than that of the former. Therefore, we choose the AST without leaf nodes to characterize the source code structure information.

High Efficiency
High efficiency is an important advantage of Re_Trans compared to other SCS methods. It is mainly reflected in two aspects: the efficiency of source code semantic extraction and the efficiency of Re_Trans' generation model.
(1) Efficiency of source code semantic extraction: The AST contains a large number of leaf nodes with irregular user-defined identifiers, which makes the data processing time-consuming. From Table 3, we know that the SCS' result of AST with leaf nodes (ast_with_leaf) is close to that without leaf nodes (ast_no_leaf) in our method. In order to demonstrate the efficiency of source code semantic extraction, we randomly select 100,000 samples from the dataset and construct AST structure information with and without leaf nodes, respectively. We calculate the accumulated time (the time unit is seconds) for each 10,000 samples, and show the results in Figure 11a. Furthermore, we randomly select 1000 samples from these 100,000 samples, and test their SCS time through Re_Trans for these two source code structure representations, respectively. We calculate the accumulated time (the time unit is seconds) for each 100 samples processed and show the results in Figure 11b.  In Figure 11a, the ast_no_leaf extraction method we proposed (the green line) takes significantly less time, and the growth rate is slow, which reflects the efficiency of AST without leaf nodes in data processing. In Figure 11b, our proposed AST structure (the purple line) is more efficient than the ast_with_leaf extraction method in the SCS task. Therefore, we use the AST that removes leaf nodes, which not only saves the time of graph traversal, but also avoids the repeated operation of source code sequence information processing.
(2) Efficiency of the Re_Trans generation model: We randomly selected 1000 samples from the dataset. We test the SCS generation time of Re_Tran' generation model (Re_Trans_generative) on these data, Re_Trans, Re_Trans generative model with leaf nodes (Re_Trans_leaf_generative), Rencos and Rencos generative model (Rencos_only_NMT). The time statistic results are shown in Figure 12. From Figure 12, we can see that Re_Trans_generative (the purple line) takes the shortest time and is more efficient than Rencos_only_NMT (the cyan line). Moreover, the Rencos method (the blue line) takes the longest time, and the efficiency of our method (the green line) is higher than Rencos. The main reason is that Rencos requires both IR-based and NMT-based methods for each test. However, Re_Trans only uses the IR-based method, or the IR-based and generative methods for each test. The abovementioned generation model of Re_Trans is more efficient than Rencos. Furthermore, in our test, the Rencos retrieval database has 6648 samples and Re_Trans has 10,000 samples, but their average retrieval time of one test data is about 0.104 s. Obviously, the Re_Trans' retrieval efficiency is higher. In practice, with the continuous expansion of the retrieval database, the average retrieval time will increase accordingly. Because the test data needs to match each item in the retrieval database to find the sample with the highest similarity score.

SCS Library
From the dataset analysis in Section 3.1, we find that the public dataset has a large number of similar functions, which are suitable for building an SCS retrieval library. In order to demonstrate the effectiveness of the retrieval library, we conduct the following experiments: (1) In order to test the effectiveness of our retrieval library, we randomly select samples from the code retrieval library after data processing and use the t-SNE data dimensionality reduction and visualization technology. The final result is shown in Figure 13. In view of the fact that too many samples lead to a large number of repeated points in the picture, which affects the visualization effect, we select two groups of 100 and 200 random samples corresponding to (a) and (b) in Figure 13. From Figure 12, we can see that almost all the two points with the same color overlap or are very close. It indicates that functions with the same summarization still retain the same semantic information after data processing. Furthermore, it also shows the effectiveness of a retrieval library built by us.
(2) In order to test the effectiveness of the Re_Trans retrieval model, we randomly select 1000 samples and calculate the probabilities of Top k (p@ k) between test function and each sample by Euclidean distance, where k is 1, 3, 5, and 10. The Top k is to find the top k numbers from the retrieval library. The higher p@ k, the better matching effect. The test is carried out in 10 groups, and we use the boxplot for visual display. We show the result in Figure 14. From Figure 14, we can see that the p@ k increases gradually with the increase of k. In 10 rounds of testing experiments, when k = 3, the minimum value is 80%, the maximum value is 96%, and the average value is 92%. For any test code snippet, it indicates that the same semantics' code snippets searched from the retrieval library are the similar code snippets with higher accuracy. It also indicates the feasibility of using the SCS of similar code as the SCS of the test code.

Conclusions and Discussion
In this paper, we combine AST-augmented and code sequence-augmented to represent source code semantic information. We propose an efficient and accurate SCS generation system, Re_Trans. It first utilizes a retrieval-based model to obtain the most similar code with regard to semantics and its SCS (S_RM). Then, it feeds the given code and its similar code to the trained discriminator. Finally, it decides to use the S_RM as a result or utilize the transformer model to obtain the new result according to the discriminator' output. Moreover, we conducted a series of contrast and ablation experiments to demonstrate that the Re_Trans outperforms existing SOTA methods. Combined with the recent work and the research of this paper, we suggest some valuable research points for the future: In the future, we plan to expand the SCS retrieval library and pay special attention to the quality of the expansion data. Furthermore, we also plan to further investigate the usefulness of our approach, using it to generate SCS for other program languages without code comments. Moreover, a large-scale language training model will be an inevitable requirement with the increasing daily data. Therefore, it is a meaningful research direction that includes extracting effective semantic information without occupying too many computing resources.
Author Contributions: Conceptualization and methodology design: C.Z.; data curation, original draft preparation, C.Z., M.Q. and K.T.; review and editing, Q.Z. and F.L.; visualization and investigation, C.Z. and L.X. All authors have read and agreed to the published version of the manuscript. In this section, we list our experimental parameter settings in Table A1 and metrics introduction in Appendix A.2.

. Metrics
For one Java function x, suppose that the generated SCS by Re_Trans is y, namely candidate sentences. The ground-truth SCS of x is s, namely reference sentences.
BLEU measures the n − gram precision of candidate sentences that appear in reference sentences. The higher score of BLEU-N (N = 1, 2, 3, 4), the higher quality of y. The formula to calculate BLEU-N is as follows: BLEU − N (y,s) = BP · exp N ∑ n=1 w n · log p n , BP= 1 exp(1 − l r /l y ) l y > l r l y ≤ l r where BP is the brevity penalty, used to punish the short candidate sentences. p n is the precision score of the n − gram matches between y and s. w n is usually the uniform weight of n − gram, w n = 1/N. l y is the length of y, and l r is the length of s.
ROUGE-L utilizes the Longest Common Subsequence (LCS) between y and s. The formula to calculate ROUGE-L is as follows: R lcs = LCS(y, s) len(y) , P lcs = LCS(y, s) len(s) , ROUGE − L = (1 + β 2 ) · R lcs · P lcs R lcs + β 2 · P lcs where R lcs is the recall rate and P lcs is the precision rate,β = P lcs /R lcs . METEOR considers the precision and recall rate based on the entire corpus. It uses WordNet to expand the synonym set, which has a high correlation with human judgment.
However, this metric only used in Java programming language. The formula to calculate METEOR is as follows: P = mapped total y , R = mapped total s , F mean = 10P · R R + 9P , Penalty = 0.5 · ( blocks unigrams ) 3 where total s , total y are the total number of words in s and y, respectively. mapped represents the mapping result of words in y on s, and only retains words that appear at most once in s. unigrams represents a single word, blocks represents how well a phrase is matched.
Penalty is used to avoid the case where only word matching is considered.


A New Fuzzy $H_{\infty}$ Filter Design for Nonlinear Time-Delay Systems with Mismatched Premise Membership Functions

This paper is concerned with the fuzzy $H_{\infty}$ filter design issue for nonlinear systems with time-varying delay. To overcome the shortcomings of the conventional methods with matched preconditions, the fuzzy $H_{\infty}$ filter to be designed and the T-S fuzzy model are assumed to have different premise membership functions and number of rules, thus, greater design flexibility and robustness to uncertainty can be achieved. However, such design will also make the derived results conservative, to relax the result, a novel integral inequality which is tighter than the traditional inequalities derived from the Leibniz-Newton formula is applied, besides, a fuzzy Lypunov function and the information of the membership functions are also introduced. All the design methods are presented in LMI-based conditions. Finally, two numerical examples are given to prove the effectiveness and superiority of the proposed approach.

INTRODUCTION
It is well known that time-delay can cause instability and deteriorate the performance of the systems, also it is inevitable in practical control systems. So the study of control systems with time-delay is critical, and researchers have devoted great efforts to investigate them (Cao and Frank [2000], Jeung et al. [1998], Wu et al. [2004], Wang et al. [2020]). And in this paper, we will mainly investigate nonlinear systems with time-varying delay.
Moreover, filtering technology is playing a crucial role in signal processing, and over the development of several decades, fruitful research results have been obtained , Bernstein and Haddad [1989], Gao and Wang [2003], Kong and Dai [2016], Fridman and Shaked [2001], ), among them, H ∞ filtering (Fridman and Shaked [2001], Lin et al. [2008], Ma [2022], Qin et al. [2021]) has attracted widespread attention, as H ∞ filter has no particular requirement for external noise signal and it is not sensitive to uncertainty. In the past few years, researchers has proposed various techniques to improve the performance of the H ∞ filter, to mention a few, in (Lin et al. [2008]), a delay-dependent fuzzy H ∞ filter design method was proposed for T-S fuzzy-modelbased system with time-varying delay. However, in this paper, the Lyapunov-Krasovskii function candidate was ⋆ This study was partly supported by the National Natural Science Foundation of China (61304108,51307035) chosen as a single Lyapunov function, to obtain more relaxed results, the literature ) adopted a fuzzy Lypunov function to analyze the stability condition. In (Su et al. [2009]), the fuzzy H ∞ filter design approach was improved through estimating the upper bound of the derivative of Lyapunov function without ignoring any useful terms. On the basis of (Su et al. [2009]), literature (Huang et al. [2011]) proposed a technique to obtain more accurate upper bound of the derivative of Laypunov function.
However, when designing H ∞ filter for nonlinear systems with time-delay, all the proposed methods require matching conditions, i.e, the fuzzy filter and the fuzzy model are assumed to have same membership functions, such assumption can facilitate the filter design, but on the other hand, it will also limit the design flexibility and make the designed filter lose robustness to the system with uncertain membership functions. To address this problem, we will apply the imperfect premise matching method (Lam and Narimani [2009], Kong and Dai [2018]) to design fuzzy H ∞ filter, which means the fuzzy filter to be designed and the fuzzy model will be allowed to have different premise membership functions and number of fuzzy rules, thus successfully address the issue of traditional H ∞ filter design methods. Nevertheless, such design will also make the designed filter tend to be more conservative. So in this paper, we will adopt a fuzzy Lyapunov function to derive the fuzzy H ∞ filter, which can increase the number Besides, as H ∞ filter has to ensure the filtering error system is asymptotically stable, the stability analysis is necessary, and in this process, the inequalities derived from the Leibniz-Newton formula is often adopted, like in literature (Huang et al. [2011], Lin et al. [2008], , Qiu et al. [2009], Lin et al. [2007], ). Though the H ∞ filter design issue can be solved, the derived results are conservative, and there is little room left to improve. So in this paper, we aim to use a new integral inequality to substitute the conventional inequalities derived from the Leibniz-Newton formula, and to further relax the designed fuzzy H ∞ filter.
What's more, usually the existing fuzzy H ∞ filter design methods are membership functions independent (Kong andDai [2017, 2016], Huang et al. [2011], Lin et al. [2008], ), to the best of our knowledge, the membership functions dependent H ∞ filter design approach is yet to be thoroughly investigated, so in order to further reduce the conservatism, we will consider the information of the membership functions in the criterion.
For the sake of obtaining satisfying fuzzy H ∞ filter, we are going to employ the mismatching method to design fuzzy filter, and to relax the results, a novel integral inequality which is tighter than conventional inequalities will be applied, besides the information of the membership functions and a fuzzy Lyapunov function technique will be introduced into the criteria as well. And the rest of this paper will be organized as follows: Section 2 systematically describes the problem we are going to solve and gives a relevant lemma about the new integral inequality. In Section 3, the detailed H ∞ design methods are presented. And Section 4 uses three simulation examples to illustrate the effectiveness of the designed methods. Finally, some conclusions are given in Section 5.

PRELIMINARIES
Consider a nonlinear system involving time-varying delay, which is described by the following p-rule T-S fuzzy model: where i = 1, 2, . . . , p. ψ α (t)(α = 1, 2, . . . , m) is the premise variable. M i α is the fuzzy term of rule i which corresponds to the function ψ α . m is a positive integer. And x(t) ∈ R n is the system state, z(t) ∈ R q is the unknown signal to be estimated, y(t) ∈ R m is the system output, w(t) ∈ R p is the noise signal which is assumed to be arbitrary and satisfy given system matrices. Time delay τ (t) is a continuously differentiable function, satisfying the conditions followed: By fuzzy blending, the system dynamics can be presented asẋ where and φ i (ψ(t)) is the normalized membership function; µ M i α (ψ α (x(t))) is the grade of membership function which corresponds to the fuzzy term M i α . In order to eliminate the drawbacks of the conventional PDC methodlogy, in this paper, the fuzzy H ∞ filter to be designed is allowed to have different premise membership functions and number of rules from the fuzzy model. Hence, we assume that the number of the fuzzy filter to be designed is c, and it can be described as: where x f (t) ∈ R n and z f (t) ∈ R q are the state and output of the fuzzy H ∞ filter respectively. And A f j , B f j , C f j are the filter matrices of that will be designed.
Similarly, through fuzzy blending, the fuzzy H ∞ filter to be designed can be presented aṡ where for all j, n j (g(t)) is the normalized membership function.
is the grade of membership functions which corresponds to the fuzzy term N j β . According to (3) and (6), and define the augmented state vector as ζ(t) = [x T (t), x f (t)] T and e(t) = z(t) − z f (t), we can obtain the H ∞ filtering system as follows: where

and
Ergo, the fuzzy H ∞ filter design issue that will be investigated in this paper can be presented as follows: Fuzzy H ∞ filter issue Design a fuzzy filter in the form of (6) satisfying the following two conditions: (1) If w(t) = 0, the filtering system (8) is asymptotically stable; (2) For a given scalar γ > 0, if ζ(t) ≡ 0 for t ∈ [−τ 0 , 0], the following H ∞ performance can be satisfied for all the What's more, the following lemma is useful for the later deduction of the main results.
Lemma 1 (Hong et al. [2015]) It is assumed that x is a differentiable function: , and R ∈ R n×n > 0, the following inequality holds: where

MAIN RESULTS
As fuzzy H ∞ filter has to guarantee the asymptotical stability of the filtering system, we will first analyze the stability of the system (8). (8), the constants h, ρ and γ > 0 are prescribed, it will be asymptotically stable with w(t) ≡ 0, and satisfy the H ∞ performance condition (10)

Lemma 2 For system
> 0, such that the the following inequality is feasible. where proof To relax the result, the Lyapunov-Krasovskii functional candidate can be defined as fuzzy Lyapunov function: where It is necessary to mention that in the Lyapunov function, the matrix M is presented as constant matrix, but N (t) and O(t) are expressed as matrix function, this is because if matrix M is denoted as matrix function, it will be difficult to determine the upper bound of φ (x(t)) .
From the inequality (17), it can be inferred that if for V (t)| t=0 = 0, and V (t)| t=L ≥ 0. The inequality (20) can be easily converted to inequality (10), which implies the H ∞ performance requirement is satisfied.
According to the Schur Complement criterion, we can easily convert the inequality (18) as (12). Then following similar deduction process, as w(t) ≡ 0, and in the light of the inequalities (18), (17), we can deriveV (t) < 0, which means the filtering system (8) is asymptotically stable. Hence the proof of Lemma 2 is finished.
Remark 1 From the proof process of Lemma 2, we can see that a new integral inequality (11) is used to derive the upper bounds ofV (t), usually the work is finished by the inequalities derived from the Leibniz-Newton formula, compared with these inequalities, the adopted inequality (11) is tighter, and thus less conservative result can be obtained. Besides, the LMIs (12) derived from (11) is simpler, which means it will be more realizable in practice. (13), it can be seen that a fuzzy Lyapunov function is adopted, such design will increase the number of the free-weighting matrices, and thus can relax corresponding result. Lemma 3 For system (8), the constants h, ρ, υ and γ > 0 are prescribed, it will be asymptotically stable with w(t) ≡ 0, and satisfy the H ∞ performance condition (10), if there exist matrices
and it can also be denoted as Consequently, if the inequality (29) holds, the inequality (12) will be true.
Then we'll introduce a partition as where M 11 > 0, M 22 > 0, and M 12 is assumed to be invertible via invoking small perturbation if it is necessary.

Assume
and R = diag{S , S , S , S , 1}. Pre and post multiplying (29) with diag{R, S , 1}, then the LMI (25) can be obtained with the changes of variables as (32) From (32), we can obtain: Besides, asM 22 = M 12 M −1 22 M T 12 , through an equivalent transformation M −T 12 M 22 x f (t), we can obtain an admissible fuzzy H ∞ realization as: (34) Thus, we complete the proof of Lemma 3.
Lemma 3 provides a feasible H ∞ filter for system (3), however, it is necessary to point out that the equation (26) cannot be directly applied to fuzzy filter design. To address this problem, we will transfer the conditions in Lemma 3 into a finite set of LMIs. Theorem 1 For system (8), the constants h, ρ, υ and γ > 0 are prescribed, it will be asymptotically stable with w(t) ≡ 0, and satisfy the H ∞ performance condition (10), if there exist matrices N i ∈ R 2n×2n > 0,Õ i ∈ R 2n×2n > 0, such that the following LMIs are feasible. 2, ..., p, j = 1, 2, ..., c. and in this case, the parameters of the fuzzy filter can be presented as proof Since and in terms of (9) and (14), we can derivẽ whereΩ ij is defined in (36).
As a result, ifΩ ij < 0 holds,Ω(t) < 0 can be derived. And according to Lemma 3, we know thatΩ(t) < 0 means the filtering system (8) is asymptotically stable and satisfy the H ∞ performance condition. Hence, the proof of Theorem 1 is accomplished. Theorem 2 For system (8), the constants h, ρ, υ and γ > 0 are prescribed, it will be asymptotically stable with w(t) ≡ 0, and satisfy the H ∞ performance condition (10), if there exist matrices 10n+2) , such that the following LMIs are feasible.
whereΩ ij is defined in (36). And in this case, the parameters of the fuzzy filter can also be presented as (37).
proof In this part, for the convenience of notations, we denote φ i (ψ(t)) = φ i , n j (g(t)) = n j , φ i (ψ(t))n j (g(t)) = d ij , and assume d ij andd ij are the lower bound and upper bound of d ij , respectively.
From Lemma 2 and Lemma 3, we havė so we can derive (44) Consequently, if LMIs (41) holds, we can getΩ(t) < 0, which means both the H ∞ performance condition (10) and the asymptotically stable requirement can be satisfied. Thus, the proof of Theorem 2 is finished.

Remark 3
It can be seen that Theorem 2 includes the information of the membership functions while Theorem 1 does not. As a result, Theorem 2 is less conservative than Theorem 2. Whereas, on the other hand, Theorem 2 also includes complex matrices M ij , Q ij , (i = 1, ...p, j = 1, ..., c), which means it will be more difficult to implement in engineering applications. So both Theorem 1 and Theorem 2 have their own significance in practice.

SIMULATION
In this section, two simulation examples will be provided to demonstrate the effectiveness and superiority of the designed criteria.
From Table 1-3, we can clearly see that smaller minimum attenuation level γ can be obtained with Theorem 3 than the ones obtained with other methods, which implies that   Zhou and He [2015]).

Remark 4
The less conservative results can be obtained with the approach proposed in this paper mainly because of three reasons. First, the new integral inequality (11) is employed to derive stability condition, which is tighter than those derived from the Leibniz-Newton formula. Besides, our criterion is membership functions dependent while the others are membership functions independent. Also, we adopt fuzzy Lyapunov function, we can increase the number of free-weighting matrices, and further relax the result.
Then Theorem 1 and Theorem 2 will be adopted to compute the minimum attenuation level γ, the corresponding results are listed in Table 4-5  Table 4. The minimum attenuation level γ for h = 0.5 method υ = 0.7 υ = 1 υ = 2 υ = 5 υ = 10 υ = 20 From Table 4-5, we can clearly see that Theorem 2 can produce much smaller minimum attenuation level γ than Theorem 1, which means Theorem 2 is more relaxed than Theorem 1. And this is because Theorem 2 takes the information of the membership functions into consideration while Theorem 1 does not. Besides, in the simulation process, Theorem 2 spent much longer time to compute the minimum attenuation level γ than Theorem 1, the reason for this phenomenon is that LMIs (36) in Theorem 1 is simpler than the LMIs (41) in Theorem 2, which also implies Theorem 1 will be more realizable than Theorem 2 in practice.
Remark 5 It is essential to point out that in Example 2, the fuzzy model has 3 fuzzy rules while the fuzzy filter have 2 fuzzy rules, and the membership functions of the fuzzy model and the fuzzy filter are totally different. Such design means the number of fuzzy rules and the membership functions of the fuzzy filter can be freely chosen, thus, we can lower the implementation cost via choosing simpler membership functions, and successfully deal with the situation with uncertainty through avoiding unknown membership functions.

CONCLUSIONS
In this paper, the fuzzy H ∞ filter design problem has been investigated for nonlinear time-delay systems. The T-S fuzzy model has been used to describe the dynamics of the system, and two LMI-based criteria have been derived. Unlike conventional fuzzy H ∞ filter, The designed fuzzy filter has been allowed to freely choose the premise membership functions and the number of rules, ergo, robustness to uncertainty and lower implementation cost can be realized. Besides, to reduce the conservatism of the derived results, a novel integral inequality which is tighter than other existing ones has been introduced, and a fuzzy Lyapunov function and the information of the membership functions have been taken into account, too. Finally, three examples have demonstrated the validity of the designed fuzzy H ∞ filter.


Multi-AI Complex Systems in Humanitarian Response

AI is being increasingly used to aid response efforts to humanitarian emergencies at multiple levels of decision-making. Such AI systems are generally understood to be stand-alone tools for decision support, with ethical assessments, guidelines and frameworks applied to them through this lens. However, as the prevalence of AI increases in this domain, such systems will begin to encounter each other through information flow networks created by interacting decision-making entities, leading to multi-AI complex systems which are often ill understood. In this paper we describe how these multi-AI systems can arise, even in relatively simple real-world humanitarian response scenarios, and lead to potentially emergent and erratic erroneous behavior. We discuss how we can better work towards more trustworthy multi-AI systems by exploring some of the associated challenges and opportunities, and how we can design better mechanisms to understand and assess such systems. This paper is designed to be a first exposition on this topic in the field of humanitarian response, raising awareness, exploring the possible landscape of this domain, and providing a starting point for future work within the wider community.

INTRODUCTION
Humanitarian emergencies are growing in number and scale, and in many cases interact with each other, adding to an increased level of complexity. AI decision support systems and automated decision-making technologies are increasingly being used in the humanitarian sector to address the many pressing challenges of responding to emergency situations, including: early warning and preparedness systems; assessment and monitoring capacities; service delivery and support; and operational and organizational efficiency [16]. Many of these efforts are in their nascent stages, however, there is a growing number that are becoming operational and acting as decision support, or decision-making systems in real-world situations.
Deployed AI systems are used at multiple levels of humanitarian decision-making, from headquarters (HQ) to field operations. However, they are typically considered to act in isolation. For example, HQ organizations may provide maps generated using AI systems [11,14], response teams may use Natural Language Processing (NLP) tools to collect feedback from affected populations to inform operational responses [7], and near and long term future scenarios may be forecasted using AI systems for resource mobilization and contingency planning [6,12]. (For more examples see the following survey [9].) There is also a large movement focused on forecast-based financing which uses forecasting methods to inform funding of crisis prevention, mitigation, and response efforts [8].
With a growing number of entities using AI systems to inform their response efforts, it is natural that these systems begin interacting with each other, despite the fact that inter-dependencies are not always recognised, traceable, or addressed by design. In addition, these systems are often not isolated from the broader environment of AI systems beyond the humanitarian space. There are a multitude of other AI systems which externally impact the work of humanitarian organizations -e.g. social media algorithms which curate content and may effect analyses of social media trends and sentiments, or maps produced by external entities [13,15].
There are numerous works highlighting the risks and potential harms associated with using AI systems in the humanitarian sector. These risks and harms include: questions of biases and fairness which can oppress minority voices; accountability of systems and the lack of legal frameworks; challenges surrounding transparency and human oversight; a lack of expertise in the proper and appropriate deployment and use of AI; data privacy and protection concerns; and the meaningful participation of affected communities in every stage of the AI lifecycle, as well as their ownership over such systems [1,4,16,17]. In an attempt to address some of these concerns, there has been a drive to draft AI ethical guidelines, frameworks, and training materials at the international [20], national, and sector specific levels [10]. These principles generally approach the problem of the ethical use of 'AI systems' in isolation, rather than from a systems perspective -i.e. again assuming such AI systems interact with their environment, but not explicitly considering an environment which may include other AI systems.
The field of multi-agent reinforcement learning (RL) has taken important steps in understanding the technical and theoretical behaviour of certain multi-AI systems 1 . Studies commonly focus on the deployment of multiple reinforcement learning models in a closed environment ('games') to monitor their interactions as they perform certain tasks, optimising for various objectives. Traditionally, RL agents have competed against each other in such games, with recent advances in setups where performance in the competition can be clearly defined (e.g. in games of skill), as well as in the more challenging context of zero-sum games [3,21]. There are also many examples in which agents are not intended to be competing. Cooperative AI is an important consideration in areas such as self-driving cars, in which AI agents should work together and any negative competition could be dangerous (see [5] for a range of open questions, challenges and opportunities around building cooperative AI systems).
In the real world, however, outside these closed game setups, there are erratic behaviours and social dilemmas at multiple scales which AI models interacting with these environments must accommodate. Methods, such as that proposed by Baker [2], have begun to try to address this challenge through the training of agents with randomised uncertain social preferences, updating the rewards of RL agents with uncertainty to mimic complex mixed-motive environments. Further work is still needed to take this into a real-world setting for stress testing.
The example of multi-agent RL is a simplified version of reality for simulating and understanding different behavioural patterns. In this paper, we discuss the broader context of AI systems which may or may not take the form of fullyautomated RL agents, and in which humans also play important roles in decision-making chains and interactions. In the 1 Indeed, multi-AI systems such as those discussed in this paper are occasionally referred to as 'multi-agent' systems in which an 'agent' can have the form of any general AI system. 'Agents' in some literature can also refer to humans or AI systems interchangeably. In this paper, we use the phrase 'multi-AI' for the avoidance of doubt as we are referring to AI systems in general, which may or may not have RL components and may have humans-in-the-loop. In the multi-agent RL literature, agents can also sometimes be considered to be of the same 'class' -i.e. operating under the same or variations of the same set of rules and objectives -whereas in our setting the different actors/models/systems can be very heterogeneous in nature. Further, we also include humans in the information flow networks and feedback cycles referred to in this paper; however, we make explicit reference to them when we do so. Fig. 1. Example of a simple information flow network in a humanitarian response. Purple nodes represent AI-only systems, orange nodes represent human-only decision-makers, while blue nodes represent entities or groups which consist of both humans and AI systems together. The arrows represent information flow following the same color scheme.
field of humanitarian response, there are a large number of actors, with mixed, numerous, and often hard to quantify objectives for which to optimise, working at multiple scales. These systems and actors may work more slowly than traditional RL agents in a sandbox environment, however, still react very rapidly by human decision-making standards.
As we continue to operationally deploy AI systems (which are themselves complex systems) in complex humanitarian environments (which contain other AI systems), such systems become intertwined and interact, creating second-order effects which should be explored and understood. This mirrors the higher-order complexity seen when multiple interacting crises occur -e.g. when a conflict, food crisis and pandemic all exacerbate each other. These multi-AI complex systems can generate new sets of emergent behavior which will not be encompassed by only considering AI systems individually, and therefore requires new considerations vis-a-vis their impacts, risks and harms, ethics and overall assessments. There is much we can learn from the technical literature in the multi-agent RL and other spaces, and multidisciplinary approaches are necessary, and need to be developed quickly, to better understand this rapidly evolving multi-AI humanitarian landscape.

EXAMPLE OF AI SYSTEMS EMBEDDED INTO A HUMANITARIAN RESPONSE PIPELINE
A key starting point for understanding multi-AI systems and ensuring their behavior is understood and controlled is to piece interactions apart by considering such systems as information networks where decision-making nodes can be human, machine, or a combination of both.
It is instructive to start with an example of what such a system might look like given current real-world workflows.
In Figure 1 we see an example of a simple information flow network in a realistic humanitarian response scenario.
When a crisis occurs, such as a flood which affects a large population, it is common to use satellite imagery to assess the extent of the flood and/or damage. The image is usually provided by an external provider -through fully automated interactions -some of whom use AI models to process their imagery (e.g. to select those with minimal cloud cover or correct for cloud cover). This is passed to a human image analyzer who uses AI to help rapidly detect where the flood has occurred. The resulting flood map is then passed to another team who uses it as an input to an AI model for identifying the most affected population, and plan a route to their location. The plan is passed to a response team in order to deliver the emergency aid, and we assume that they also get the underlying flood map which is fed into the route planning model from the image analyzer.
At various stages, feedback may be given based on the results. The affected population might give feedback to the response team telling them they have gone to the wrong place or that there are better ways to get to a given area.
The response team might pass this information back to the route planners; however, if they consider it an error in the flood map, which they also have access to, they might give feedback directly to the image analyzer who produced it.
The route planning team might also see errors in the flood map, based on the feedback from the response team which may or may not have also been provided directly to the image analyzer, and provide this feedback back to the image analyzer themselves. We regard it unlikely in this scenario that the image analyzer would provide feedback directly to the external image provider.
It is important to understand both kinds of information flows: the decision/output and the feedback. AI derived information can be erroneous and can cause the multitude of challenges, risks and associated harms generally elicited through existing AI system risk assessments and enumeration in ethical frameworks. When such erroneous information flows into other systems which then make use of it, such risks and harms can be compounded.
The feedback cycles can also be erroneous and can perpetuate or induce later incorrect results. While the process of model validation and updating based on feedback can be rigorous and systematic at the level of the individual AI system, the process of how, and to whom, an entity provides feedback becomes complex when there are multiple interacting systems. When working within complex systems, it can be hard for those giving feedback to correctly identify the system that generated the data 2 , and if a correction or validation is performed on the wrong system, this can further introduce errors through the feedback process itself.
In the example discussed above, the response team may incorrectly deduce that the flood map has errors, rather than the output of the route planning model, based on their own experience or feedback from the population, and therefore initiate a correction to the image analyzer's model which results in feeding incorrect information back into the analyzer's model rather than the route planner's model. This in turn leads to the image analysis model performance decreasing, which affects the route planning model input and in turn provides the response team with worse information.
Taking this one step further, the response team may then find another error but this time report it to the route planning team when the error was actually due to the flood map.
The above scenario is relatively simple, and humanitarian response workflows in reality can be much more complex with large numbers of actors and only partial feedback cycles. Nonetheless, by playing out this simple scenario, we can see that while each stage of the process may be correct in isolation -for example, each model may have passed ethical risk assessments and have validation and update procedures in place based on feedback -the broader multi-AI system may descend into a state of emergent erratic behavior with models becoming tuned to incorrect data due to an ill-understood landscape of information flow and feedback cycles. Note that this scenario can play out even under the assumption that each individual node which is using an AI system believes their input data to be reliable unless observed directly otherwise. Indeed, such behavior is not the only challenge to handle and understand, more of which will be discussed below. 2 similar to the credit assignment problem in the RL literature

TOWARDS TRUSTWORTHY MULTI-AI SYSTEMS: CHALLENGES AND OPPORTUNITIES
These multi-AI systems can present a range of new challenges, risks and associated harms, as well as opportunities for better understanding complex humanitarian situations more holistically, rather than compartmentally. From the discussion above, it is clear that one risk is that such systems become trained on erroneous data -e.g. with errors from one AI model feeding into another, or human errors in which system is updated and how (incorrect feedback pathways) -but which is thought to be reliable and can result in providing incorrect information to decision-makers during emergency situations and have significant harmful repercussions. Further, these complex interactions also have the potential to significantly enhance the effects of outliers. It is commonly known that AI systems can perform erratically on input data which is in the region of low or no statistics in the training domain; however, the training domain for the system as a whole can now be considered as a complex distribution over many interacting training domains 3 . This can have negative impacts on the inherent uncertainties of the multi-AI system and therefore, if not understood and communicated properly, can result in ill-informed/over-confident decisions being made.
More broadly, by not understanding the complex interactions of these systems, efforts to include community participation in the design and deployment of AI systems becomes increasingly difficult. Indeed, the ability to ensure affected populations have meaningful ownership of AI systems presents further challenges as inputs to any end-user focused system may be unknowingly altered through these interactions.
As mentioned above, multi-AI systems present key challenges to existing AI ethical principles, frameworks and guidelines. Decisions lack traceability unless the one/two-way interactions between systems, and feedback loops with participants, are understood, and this can have repercussions with regards to accountability and responsibility of decision-making, as well as the ability to have oversight over AI systems. For example, in the scenario laid out above, while the route planning team may be theoretically accountable for the decision of which route to direct the response team along, when responding rapidly in crisis situations, the route planning team may be able to legitimately claim that they were acting on information which was correct to the best of their knowledge as the full multi-AI system had not been understood.
Finally, while there are significant challenges, risks and harms which can be associated with such multi-AI systems, especially when they are ill-understood, there is still great potential for their use. Humanitarian situations, along with many other real-world scenarios in which AI systems can and will be used, are growing in their complexity and such multi-AI systems may be able to help explain and interact with a broader collection of crises to understand them more holistically.

DISCUSSION AND NEXT STEPS
Multi-AI systems can be thought of as a complex interacting network of interconnected complex systems. These systems are being increasingly used in many domains, including in humanitarian response, but their behaviours are often not understood in their entirety. However, if safely and successfully deployed, multi-AI systems have the ability to help answer new questions, better understand much of the increasing complexity of our world and provide new solutions to operational challenges.
To understand these higher-order complex systems we need a shift in the modalities of thinking and operating as a community, which includes understanding a broad and evolving spectrum of system interactions at different scales.
We need to expand the traditional assumptions of data generation and transformation, considering how information is shared and where it is coming from, as well as how feedback cycles are created, either formally or informally. In settings such as humanitarian emergencies there are many sets of actors linked through multiple pathways, rather than acting as linear systems, which need to be understood in order for them to function effectively. To address these challenges, we propose several areas of future research.
First, from a methodological perspective, a continuation of the development of frameworks for building and testing multi-AI systems, as well as for understanding emergent behaviours of such systems is essential. Working closely with researchers in other fields, such as the multi-agent RL community, will be essential to bringing these theoretical works into critical real-world domains.
When embarking on projects, mapping the network of information flows to understand the broader multi-AI system environment is key. In the case of multi-AI systems in humanitarian domains, network 'owners' or 'regulators' could be assigned who have the responsibility to oversee the system (even if they do not 'own' or 'control' each individual component). The development of stress tests for these systems as a whole, involving all the entities in the network who are providing inputs to relevant subsystems, can then help understand possible erroneous and emergent behavior which can be mitigated before harm is caused. This can be made easier through clear communication, by AI model-creators and owners, of data inputs and model versioning for systems used. This includes the data inputs received by other AI systems and, at each decision node, one should be able to validate any AI augmentation to the input data which has occurred.
Secondly, a greater degree of cross-pollination of these technical communities with legal, ethics, and humanitarian response experts is needed to create ethical and human rights frameworks appropriate for multi-AI systems. A systematic mapping exercise to understand the potential gaps in existing ethical principles and guidelines in the context of multi-AI systems can be supported with technical exercises to both quantify such principles and guidelines and measure when certain criteria are met at the individual AI system level, and then fail to be met at the multi-AI system scale -i.e. questions such as: 'Under which conditions a multi-AI system is unsafe even if the individual subsystems are safe?' and 'Is a multi-AI system interpretable when every AI subsystem is interpretable?'. The multi-agent systems and AI communities have been exploring some of these concepts, and continuing to broaden the scope of this work into the field of AI safety, fairness and others is essential for ensuring both a theoretical and practical understanding of such behaviours.
Similarly, risk and harms assessments which are currently used to evaluate and mitigate risks of AI systems in humanitarian contexts -both before and during project implementations -should be adapted to work with different scales of complexity of the multi-AI systems. One could draw a parallel to how risks and harms assessments have been adapted to include not only individual privacy assessments but also group privacy assessments -as protecting individual privacy does not necessarily mean protecting a particular ethnic group or vulnerable population [18,19].
Third, from an operational and practical perspective, methods for communicating these concepts to project managers and decision-makers in the humanitarian field are needed to socialise the challenges and risks associated with multi-AI systems. Mapping detailed examples of multi-AI systems is also fundamentally needed. While we have presented a specific case study of a multi-AI system in a humanitarian context, there are many other such examples in humanitarian response. For example, aid based on cash transfer mechanisms might use, in addition to multiple human steps, a number of population estimation methods (some based on indirect measures provided by AI models), AI models for disaster impact assessments, and biometric systems for beneficiary identification. Alternatively, a multi-AI system for emergency communications around mis-and dis-information could be composed of human analysts, communication officers and AI subsystems for data collection, language models with biases for certain languages, and AI-driven alert detectors.
Detailed examples will help build operational guidelines, as well as capacity building exercises, and communities of practice could also support the sharing of knowledge and information around this topic within the context of emergency response.
Finally, more broadly, as a multidisciplinary community, greater collaboration is needed to understand the complexities and associated risks and harms of multi-AI systems, both within the humanitarian domain and beyond. There is a wealth of other fields and literature to be drawn on -including: complex systems, multi-agent modelling, systems engineering, and reinforcement learning -which can be adapted to humanitarian contexts.
As AI continues to be rapidly and widely deployed in the humanitarian domain and beyond, these present and emerging challenges will continue to grow and develop. It is therefore vital that we act before multi-AI systems, many of which are already deployed without proper design and testing, become sufficiently embedded that harm is created and future harms become challenging to prevent.

DISCLAIMER
The authors alone are responsible for the views expressed in this article and they do not necessarily represent the views, decisions or policies of the institutions with which they are affiliated including the United Nations.

ACKNOWLEDGMENTS
United Nations Global Pulse work is supported by the Governments of Sweden and Canada, and the William and Flora Hewlett Foundation. The authors are grateful to Katherine Hoffmann Pham and Akbir Khan for their useful discussions and comments while preparing this work.