In order to better understand the rationale behind model behavior, recent works have exploited providing interpretation to support the inference prediction. However, existing methods tend to provide human-unfriendly interpretation, and are prone to sub-optimal performance due to one-side promotion, i.e. either inference promotion with interpretation or vice versa. In this paper, we propose a multi-level M utual P romotion mechanism for self-evolved I nference and sentence-level I nterpretation (MPII). Specifically, from the model-level, we propose a Step-wise Integration Mechanism to jointly perform and deeply integrate inference and interpretation in an autoregressive manner. From the optimization-level, we propose an Adversarial Fidelity Regularization to improve the fidelity between inference and interpretation with the Adversarial Mutual Information training strategy. Extensive experiments on NLI and CQA tasks reveal that the proposed MPII approach can significantly outperform baseline models for both the inference performance and the interpretation quality. 1 1 Introduction Recently, the interpretability of neural networks has been of increasing concern. In order to break the black-box of neural networks, many works explore the interpretability of neural networks through providing interpretations to support their inference results (Ribeiro et al., 2016; Chen et al., 2018; Liu et al., 2019; Thorne et al., 2019; Kumar and Talukdar, 2020). Although prior works have made some progress towards interpretable NLP, they tend to provide interpretations that lack human-readability. Existing interpretable models usually extract promiWork was done during internship at Microsoft Research 1 Our code is available at https://github.com/ theNamek/MPII.git Premise: People walk through a store. nent features or select input key words as explanations, such as attention distribution (Xu et al., 2015), heatmap (Samek et al., 2017), alignment rationale (Jiang et al., 2021), gradients (Li et al., 2016), magnitude of hidden states (Linzen et al., 2016), etc. Considering readability and comprehensibility for humans, some works turn to generate token-level explanations (Liu et al., 2019; Thorne et al., 2019), which are nevertheless prone to cause ambiguity. Figure 1 shows some prevalent forms of interpretations in NLI task. Obviously, human language interpretations seem more acceptable than those chaotic maps, whether it is heatmap or alignment map. As for the token-level interpretation, several discrete tokens without any logical links are vague and ambiguous. Moreover, Thorne et al. (2019) observed that token-level methods tend to predict common tokens (e.g. people, man, dog) 7074 rather than keywords. Intuitively, human language sentence-level interpretations containing reasoning logic are the best form for human to understand. With annotated natural language interpretation datasets available (Camburu et al., 2018; Rajani et al., 2019), methods of generating sentence-level interpretation have been explored recently. Cam-buru et al. (2018) proposed to first generate interpretation and then predict the label only based on the generated interpretation. Kumar and Talukdar (2020) proposed to first generate sentence-level interpretations with deep pre-trained language models (such as BERT and GPT), then fed those interpretations as extra knowledge to help improve inference performance. We notice that these methods only include one-side promotion: utilizing information contained in interpretation to improve inference, while ignoring the other-side promotion: using inference logic to enhance interpretation. As claimed in Kumar and Talukdar (2020) that their one-side promotion improves predictions' faithfulness to generated interpretations, then the other-side should be able to improve interpretation's faithfulness to inference process. This has aroused our thinking: Can we deeply fuse these two relevant tasks with ingenious combination skills and achieve mutual promotion for inference and interpretation? In this paper, we propose a multi-level Mutual Promotion mechanism for self-evolved Inference and sentence-level Interpretation (MPII). Specifically, from the model-level, we propose a Stepwise Integration Mechanism (SIM) to iteratively update the inference prediction and generate an interpretation token at each decoding step, and deeply integrate hidden representations of the prediction and the token with two fusion modules. In this way, the model learns to refine the inference conclusion as the interpretation proceeds, and the inference procedure can in turn guide the generation of interpretation at each decoding step. From the optimization-level, we propose an Adversarial Fidelity Regularization (AFiRe) to improve the fidelity between inference and interpretation with the Adversarial Mutual Information (AMI) method (Pan et al., 2020), which extends the maximum mutual information optimization objective with the idea of generative adversarial network (Goodfel-low et al., 2014). With this training framework, the model is trained against a smart backward network that learns to reward the inference prediction and interpretation of fidelity, which ensures faithfulness and makes the derived interpretation depict the true profile of how the model works (Jiang et al., 2021). To verify the effectiveness of MPII, we conduct extensive experiments on two inference tasks: Natural Language Inference (NLI) task and Commonsense Question Answering (CQA) task. Experiment results reveal that compared with baseline models, our method can achieve mutual promotion on both model inference performance and sentence-level interpretation quality. Meanwhile, through providing simultaneous inference prediction and human-comprehensible interpretation with deep integration mechanism and adversarial training strategy, our model can perform inference and interpretation of fidelity and generate more robust explanations. Main contributions of this work include:  Different from the previous works that only include one-side promotion, we mutually promote the inference and sentence-level interpretation from both the model-level and the optimization-level. We propose a Stepwise Integration Mechanism to tightly fuse latent prediction and interpretation information at every decoding step, and an Adversarial Fidelity Regularization to further improve the fidelity with the adversarial training strategy. Experiment results show that our method achieves significant improvement in both inference accuracy and interpretation quality compared with baseline models. In this section, we introduce Stepwise Integration Mechanism (SIM) and Adversarial Fidelity Regularization (AFiRe) in details. Utilizing the autoregressive nature of Transformer decoder, SIM enables deep interaction at every decoding step between inference and interpretation. With the adversarial training strategy, AFiRe enables further integration of latent semantic information between inference and interpretation, and also improves the quality of explanation sentences by bringing them closer to human expressions. Transformer model (Vaswani et al., 2017) has been firmly established as the dominant approach in text generation tasks, we therefore adopt the Transformer model as backbone. Given a sequence of tokens as input X = { x 0 , x 1 , ..., x m } (e.g. for NLI: 7075 Output Embedding Masked Multi-Head Attention Add & Norm Feed Forward Add & Norm Multi-Head Attention Add & Norm Linear Softmax MLP Argmax Gate2 Softmax Argmax Gate1 Positonal Encoding Prediction Explanation Input Embedding Multi-Head Attention Add & Norm Feed Forward Add & Norm Positonal Encoding Inputs MLP MLP Figure 2: The overall architecture of our model. Both prediction label and explanation token are generated at every decoding step. Two fusion gates are attached to enable deep interaction of their hidden representations. X = { [ CLS ] + Premise + [ SEP ] + Hypothesis } , for CQA: X = { [ CLS ] + Question + [ SEP ] + Answers } ), Transformer encoder produces a sequence of continuous vectors H enc . Conditioned on H enc , on each decoding step, Transformer decoder takes the embedding of words generated by previous steps as input and predicts the word for current step. With ground truth prediction L and explanation E from human-annotated dataset, the interpretable model is required to generate prediction L  and explanation sentence E  = { e  0 , e  1 , ..., e  n } simultaneously. Prevalent interpretable models share the same encoder and separately adopt a MLP and a decoder to generate predictions and explanations. We analogously adopt the standard Transformer encoder, but apply Stepwise Integration Mechanism to deeply integrate standard MLP and Transformer decoder at every decoding step to simultaneously produce predictions and explanations. As depicted in Figure 2, at decoding step t , decoder takes the last generated token e  t  1 and the predicted label l  t  1 at previous step as input. At the first decoding step, we pass the encoder hidden state corresponding to [CLS] token into MLP to get the l  0 . We project the label l  t  1 with Multi-Layer Perceptrons (MLP) and obtain v pt  1 , which represents the previous step prediction information. We then fuse the prediction information v pt  1 and the explanation token e  t  1 with gate mechanism. The gate probability at t step is computed by: p  t = ReLU( W 1 [ Emb l  t  1 ; Emb e  t  1 ] + b 1 ) (1) p t =  ( W 2 p  t + b 2 ) (2) where ; means concatenation, W 1 , W 2 , b 1 and b 2 are trainable parameters. ReLU(  ) here denotes the ReLU activation function (Nair and Hinton, 2010),  (  ) represents the sigmoid function. We fuse the prediction and interpretation information as below: Emb t = p t Emb l  t  1 + (1  p t ) Emb e  t  1 (3) where Emb t contains the information of prediction and the overall explanation sub-sequence generated in all previous steps. We utilize the stack of masked self-attention layers f sa used in Transformer decoder to compute the decoder hidden states: { h 0 , h 1 ,..., h t } = f sa ( { Emb 0 , Emb 1 ,..., Emb t } ) (4) 7076 The attention vector referring to the source sequence is computed with multi-head attention: v t = f mha ( H enc , h t ) (5) where H enc represents the encoder hidden states, f mha denotes the multi-head attention module. The v t is further passed into a fully connected layer followed with softmax function to obtain the vocabulary distribution of generated explanation token e  t at t step: e  t = argmax ( softmax ( Wv t + b )) (6) where W and b are both trainable parameters. The gate mechanism is then used to integrate the explanation information to update the prediction information: p t =  (MLP 1 ([ Emb l  t  1 ; MLP 2 ( v t )])) (7) where the two MLP(  ) use different parameters. We apply the residual connection (He et al., 2016) here, which is easier to optimize in the scenario of many decoding steps. This is similar to the gate mechanism used in Long Short-Term Memory (LSTM) (Hochreiter and Schmidhuber, 1997) that learns to remember important information obtained on each decoding step. At the last decoding step, the model deduces the eventual decision: L  = argmax ( softmax ( Emb l  n )) (9) where n is the length of the generated explanation E  . With this setting, both prediction and explanation are updated at every decoding step. The step-by-step explanation helps the model to do better inference, and the stepwise inference in turn guides the generation of better explanation. From the level of optimization objective, we further introduce the Adversarial Fidelity Regularization (AFiRe) to improve the fidelity of inference and interpretation. We leverage the Adversarial Mutual Information (AMI) method (Pan et al., 2020) to extend the maximum mutual information objective among input, inference prediction and the generated explanation with the idea of generative adversarial network (Goodfellow et al., 2014). Compared to the maximum likelihood estimation (MLE) objective, maximum mutual information (MMI) objective encourages the model to generate the prediction and explanation that are more faithful to the input (Kinney and Atwal, 2014; Stratos, 2019). The mutual information I ( X, L, E ) among the input X , inference label L and explanation E is formulated as: I ( X, L, E ) = EP ( X,L,E ) (cid:20) log P ( X, L, E ) P ( X ) P ( L, E ) (cid:21) = H ( X )  H ( X | L, E ) where H denotes the entropy. Because of the intractability of directly estimating the mutual information in high-dimensional space, we approximate the optimization objective with a Variational Information Maximization lower bound (Chen et al., 2016b; Zhang et al., 2018; Poole et al., 2019): I(X, L, E) = H(X) + EP ( X,L,E ) [log P ( X | L, E )] = H(X) + EP ( X,L,E ) [log Q  ( X | L, E )] + EP ( L,E ) [ KL ( P ( X | L, E ) || Q  ( X | L, E ))]  H(X) + EP ( X ) EP  ( L,E | X ) [log Q  ( X | L, E )] where KL ( || ) denotes the Kullback-Leibler (KL) divergence between two distributions. P  ( L, E | X ) and Q  ( X | L, E ) denote the forward network (gen-erating L, E conditioned on X ) and the backward network (generating X conditioned on L, E ) respectively. Since the entropy term H ( X ) associates with the training data and does not involve the parameters we optimize, the objective of MMI is equivalent as: max , E ( L  ,E  )  P  ( L  ,E  | X ) (cid:2) log Q  ( X | L  , E  ) (cid:3) where  and  are the parameters of the forward and backward network respectively. L  and E  represent the synthetic prediction label and explanation generated by the forward network. With the MMI optimization objective, the backward network is trained with only the synthetic label and explanation produced by the forward network, and prone to sub-optimal performance if the synthetic text is uninformative. Since the the backward network provides a reward for optimizing the forward network, a biased backward network may provide unreliable reward scores and mislead the forward network optimization. To remedy this problem, we leverage the Adversarial Mutual Information (AMI) method (Pan et al., 2020) to extend MMI with the idea of generative adversarial network (Goodfellow et al., 2014). Specifically, we first bring the min-max adversarial game into training procedure and add an additional objective term Q  ( X | L, E ) to maximize the negative likelihood of Q  when feeding it with the real data: min  max  E ( L  ,E  )  P  ( L  ,E  | X ) (cid:2) log Q  ( X | L  ,E  ) (cid:3)  Q  ( X | L,E ) With this interactive training strategy and regularizing the backward network with both the synthetic data and real data, the forward network will be trained against a smarter backward network that only rewards prediction and explanation of fidelity. Besides, we add an objective term P  ( L , E | X ) of maximize the negative likelihood of P  to balance the positive samples as teacher-forcing algorithm (Li et al., 2017). The final optimization objective is formulated as: min  max  P  ( L , E | X ) + MutualInformation (cid:122) (cid:125)(cid:124) (cid:123) (cid:124) (cid:123)(cid:122) (cid:125) AdversarialTraining E ( L  ,E  )  P  ( L  ,E  | X ) (cid:2) log Q  ( X | L  ,E  ) (cid:3)  Q  ( X | L,E ) As depicted in Fig 3, to encourage the forward network to learn a stronger connection between generated explanations and model predictions, we also add Q  ( X | L, E  ) as negative samples for backward network. This explicitly encourages the backward network to be capable of punishing the P  when it generates unfaithful explanations. We intend to verify the mutual promotion effect of SIM and AFiRe on the inference ability and interpretablity of model. We choose two tasks requiring inference ability: Natural Language Inference (NLI) and Commonsense Question Answering (CQA). We use six datasets as our testbeds: SNLI (Bow-man et al., 2015), e-SNLI (Camburu et al., 2018), CQA (Talmor et al., 2019), CoS-E (Rajani et al., 2019), MultiNLI (Williams et al., 2018), and SICK-E (Marelli et al., 2014). SNLI is a standard benchmark for NLI task, while e-SNLI extends it with human-annotated natural language explanations for each sentence pair. CoS-E 2 dataset extends CQA dataset with natural language explanations for each QA sample. MultiNLI is another large-scale NLI corpus, which includes a diverse range of genres. SICK-e (Sentences Involving Compositional Knowledge for entailment) provides sentence pairs that are rich in the lexical, syntactic and semantic phenomena. The latter two datasets are used for out-of-domain evaluation. NLI: We use e-INFERSENT and Transformer as two baseline models for NLI task. The e-INFERSENT model adds a LSTM decoder into INFERSENT (Conneau et al., 2017) for explanations. The classification module and the explanation generation module are separated but share the same encoder. The Transformer model (Vaswani et al., 2017) adds a MLP layer for making predictions. With this baseline, we aim to test whether vanilla transformer without further interaction can achieve good results. CQA: We use CAGE (Rajani et al., 2019) as the baseline model for CQA task. CAGE adopts the explain-then-predict approach, which firstly fine-tunes a deep pretrained language model GPT (Rad-ford et al., 2019) to generate explanations, then use a classifier to predict the inference label with the generated explanation and source text as the input. To evaluate inference performance, we report Task-specific Accuracy (NLI Accuracy and CQA Ac-curacy). To evaluate the quality of generated interpretation, we report BLEU (similarity between generation and ground truth), PPL (fluency of generated sentences), and Inter Repetition (diversity of generated explanations). Table 1 shows automatic evaluation results on the SNLI and CQA datasets with the annotated explanation from the e-SNLI and CoS-E datasets. Compared with the baseline models, our MPII method can achieve significant performance improvement for both the inference and interpretation on two tasks. It indicates that the inference and interpretation process can be mutually promoted with our proposed method. With the ablation study, we notice a performance degradation of the inference and interpretation if we remove either of them, demonstrating the faithfulness between the generated explanation and the model's prediction. Inference Promotion: We can achieve 11.73 and 2.06 absolute inference accuracy improvements compared to the baselines for the NLI and CQA task, respectively. For the NLI task, with our MPII framework, the Transformer baseline model can improve over 5 absolute accuracy score. The ablation study shows the contribution comes from not only the mutual interaction of inference and interpretation in the Stepwise Integration Mechanism (SIM), but also the adversarial mutual information training objective introduced in the Adversarial Fidelity Regularization (AFiRe). Moreover, with parameters initialized with the pretrained BART model, the accuracy can be further improved by a 4.53 absolute score. For the CQA task, we observe that better performance is still achieved compared Methods MultiNLI SICK-E Transformer 55.92 53.21 Transformer + MPII (w/o AFiRe) 56.42 53.84 Transformer + MPII 58.73 56.54 Table 2: Out-of-domain NLI evaluation results on MultiNLI and SICK-E datasets. with the CAGE baseline model. If we remove the AFiRe, a significant inference degradation would be witnessed. It also indicates the effectiveness of AFiRe for utilizing interpretability to improve the inference ability. Interpretation Promotion: The quality of generated interpretation can also be significantly improved with our mutual promotion method on both NLI and CQA tasks. For the NLI task, combined with our MPII, the Transformer baseline model can provide more accurate, fluent and diverse interpretation with much better results in all metrics. Similar with the inference results, the ablation study shows that both SIM and AFiRe contribute to the performance improvement. With the pretrained BART model, we further improve the BLEU and Inter-Rep performance and get comparable PPL compared with the e-INFERSENT model. For the CQA task, our method performs better in terms of BLEU score and the diversity of generated explanations. We notice that the BLEU scores are pretty low for CQA task, which may stem from the free form of expression for explanations in the dataset, i.e. several different explanations share the same 7079 a park typically does not have a peer or boardwalk Explanation Generation Process 0.00 0.25 0.50 0.75 1.00 P r ob o f T h r ee P r e d i c ti on L a b e l Premise: a couple standing on what looks like a peer or boardwalk Hypothesis: a couple hugging each other at the park Label: Contradiction NeutralContradictionEntailment Figure 4: Visualization for mutual promotion evolution of inference and interpretation. commonsense knowledge. We observe that most of the explanations generated by our method are reasonable enough to interpret the predictions even though the BLEU scores are low. Our method also achieves a smaller Inter-Rep score, which shows that our model can provide more diverse explanations to reveal the inference process of making predictions. As shown in Table 2, we evaluate our method with the Transformer baseline model on two out-of-domain datasets: MultiNLI and SICK-E. The results show that our mutual promotion method enables the Transformer model to be more robust, and achieves about 3 absolute accuracy improvement on both of the out-of-domain datasets without fine-tuning. It is because with our MPII method, the model can generate more reliable and domain-related interpretation, which helps to make more accurate inference prediction. The ablation results demonstrate both the adversarial mutual information training strategy in AFiRe and deep integration in SIM is very effective to improve the model's generalization and robustness. We propose a model-based evaluation metric Critic-Score to evaluate the fidelity between model's inference predictions and interpretations. Inspired by Shen et al. (2017), which applied a trained model to automatically evaluate the text style transfer accuracy in the absence of parallel dataset, we pre-train a well-performed discriminator model to evaluate the fidelity between the predicted label and the generated explanation. The discriminator is a binary classifier f : ( X, L, E ) (cid:55) Yes / No , which shares similar architecture with the backward network in our Adversarial Fidelity Regularization (Section 2.3). The training dataset is constructed based on the e-SNLI and CoS-E corpus. Given a sample  X i , L i , E i  on e-SNLI that serves as a positive sample, we build the negative sample as  X i , L i , E j  , where explanation E j  = E i is selected from another e-SNLI sample that shares either the same premise or hypothesis. With this dataset, the discriminator model is trained to learn the intrinsic fidelity between the label and its corresponding explanation. The trained discriminator achieves 97% accuracy on its test set and is able to serve as a quantitative way of evaluating fidelity. As shown in Table 3, with our proposed mutual promotion method, the Transformer model can achieve significant improvement on Critic-Score between prediction and explanation. The ablation results confirm both the deep interaction design in Stepwise Integration Mechanism and the adversarial training strategy in Adversarial Mutual Information can contribute to the improvement of fidelity and faithfulness. Mutual Promotion Visualization: Figure 4 demonstrates the evolution of the inference prediction as the interpretation proceeds. The input of the model is [CLS] a couple standing on what looks like a peer or boardwalk [SEP] a couple hugging each other at the park, of which the ground truth label is contradiction. We observe that the model draws an initial conclusion that the entailment relationship between the premise and the hypothesis is not entailment, and is not able to tell whether it is neutral or contradiction. As the deliberation proceeds, our model comes to judge that it is con-tradiction with the generated interpretation a park does not have a peer or boardwalk. From the clear split of the red and blue lines when does and not are generated, we can see that the prediction is very sensitive to explanation, which demonstrates the faithfulness (Kumar and Talukdar, 2020). Semantic Similarity Evaluation of Interpretation: To better evaluate the quality of generated explanations, we also measure the cosine similarity between generated explanations and human annotated explanations. The results are presented in Fig 5. The cosine similarity of our method conFigure 5: The distribution of cosine similarity with average sentence embedding between human annotation and generated interpretation. centrates on 0.9 and achieves higher scores than CAGE, which demonstrates the effectiveness of our MPII for generating better interpretation that are closer to human expression. Case Study Table 4 presents examples produced by different models. For the first example, e-INFERSENT fails to make correct prediction and provide reasonable explanation. In contrast, our MPII not only predict the entailment relation correctly, but also produce faithful explanations to 7081 Methods Fidelity-C  Fidelity-W  LAcc  Fluency  NLITask e-INFERSENT 3.16 2.74 3.34 4.23 Transformer 3.30 3.21 3.65 3.68 Transformer+MPII(w/oAFiRe) 4.01 4.12 4.33 4.36 Transformer+MPII 4.17 4.38 4.57 4.51 CQATask CAGE(GPT,ETP) 3.71 3.18 3.52 4.25 BART+MPII(w/oAFiRe) 4.26 4.13 4.05 4.21 BART+MPII 4.37 4.39 4.22 4.30 Table 6: Human evaluation results on Fidelity-C(fidelity between correct prediction and corresponding interpre-tation), Fidelity-W(fidelity between wrong prediction and corresponding interpretation), LAcc(accuracy of selecting correct lables when only given the generated interpretations), Fluency(fluency of interpretation). interpret predictions. For the second example, our MPII and MPII with AFiRe removed still capture the entailment relation well, and explain that at the beach and at restaurant can not be done at the same time. As we can see, these explanations generated by our method are also fluent. Table 5 shows the randomly selected examples generated by different models in the CQA task. For the first example, CAGE makes wrong prediction, and generates explanation that obviously conflicts with common knowledge. In contrast, our method can make correct predictions and generate more reasonable explanations. Similarly for the second example, CAGE seems to directly copy words from the question that do not actually contain meaningful information. Our MPII still explains well, but fails to explain properly with AFiRe removed, even if the explanation contains the correct answer, which reveals the importance of AFiRe for promotion of interpretation. Human evaluation: We conduct human evaluation to further evaluate the effectiveness of MPII. We randomly selected 300 examples from the test set of e-SNLI, and asked 4 well-educated annotators to rate every sample with 4 metrics on a 1-5 Likert scale in a strictly blind fashion (Stent et al., 2005). As shown in Table 6, analogous to automatic evaluation results (Section 3.4), our MPII can generate interpretations with best quality and fidelity to corresponding inference predictions, whether correct or wrong. With the great success of natual language inference, many recent works explore the interpretability of neural networks through providing interpretation to support their inference results (Ribeiro et al., 2016; Chen et al., 2018; Liu et al., 2019; Thorne et al., 2019; Kumar and Talukdar, 2020). Three forms of interpretation are provided by these works: (1) feature-based interpretation (Chen et al., 2016a, 2018; Ribeiro et al., 2016, 2018; Li et al., 2016; Nguyen, 2018; Feng et al., 2018; Gururan-gan et al., 2018) such as attention distribution (Xu et al., 2015), heatmap (Samek et al., 2017), alignment rationale (Jiang et al., 2021), gradients (Li et al., 2016), magnitude of hidden states (Linzen et al., 2016), etc.; (2) token-level interpretation that relatively easy to comprehend but prone to ambiguity (Ribeiro et al., 2016; Liu et al., 2019; Thorne et al., 2019), and (3) sentence-level interpretation which has the best human-readability Cam-buru et al. (2018); Talmor et al. (2019); Kumar and Talukdar (2020). Different from the previous work which only include one-side promotion, we proposed the mutual promotion mechanism that can improve the performance of both inference and sentence-level interpretation. In this work, we propose to mutually promote model inference ability and interpretability from multi-levels. From the model-level, we propose Stepwise Integration Mechanism to enable the model to refine the prediction conclusion as the explaining proceeds and also to guide the generation of better explanation with the inference procedure of reaching prediction conclusion. From the optimization-level, we propose an Adversarial Fidelity Regularization, which leverages the Adversarial Mutual Information method to improve the fidelity between the inference and interpretation, which further guarantees faithfulness. Experiment results show the effectiveness of our proposed method on both NLI and CQA tasks. Future work will involve extending our approaches into other tasks of NLP. We hope that our work can encourage further research in this direction. We would like to acknowledge Chuhan Wu for the helpful discussion. We also want to thank Jiale Xu for his kindness and help.
abstain abstain objective objective objective abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result objective objective abstain abstain abstain result result method objective objective result abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain other other abstain other abstain abstain abstain abstain other abstain other abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result result abstain abstain result abstain result abstain result result abstain result abstain other abstain result result result result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain result abstain abstain abstain abstain result abstain abstain result other other other other objective objective objective objective objective objective method other other