Recently, the Transformer model (Vaswani et al., 2017) that is based solely on attention mechanisms, has advanced the state-of-the-art on various machine translation tasks. However, recent studies reveal that the lack of recurrence hinders its further improvement of translation capacity (Chen et al., 2018; De-hghani et al., 2019). In response to this problem, we propose to directly model recurrence for Transformer with an additional recurrence encoder. In addition to the standard recurrent neural network, we introduce a novel attentive recurrent network to leverage the strengths of both attention and recurrent networks. Experimental results on the widely-used WMT14 English  German and WMT17 Chinese  English translation tasks demonstrate the effectiveness of the proposed approach. Our studies also reveal that the proposed model benefits from a short-cut that bridges the source and target sequences with a single recurrent layer, which outperforms its deep counterpart. Recently, Transformer (Vaswani et al., 2017)  a new network architecture based solely on attention mechanisms, has advanced the state-of-the-art on various translation tasks across language pairs. Compared with the conventional recurrent neural network (RNN) (Schuster and Paliwal, 1997) based model that leverages recurrence as the ba-sic building module (Sutskever et al., 2014; Bah-danau et al., 2015; Chen et al., 2018), Transformer replaces RNN with self-attention network (SAN) to model the dependencies among input elements. One appealing strength of SAN is that it breaks  Zhaopeng Tu is the corresponding author of the paper. This work was conducted when Jie Hao and Baosong Yang were interning at Tencent AI Lab. down the sequential assumption to obtain the ability of highly parallel computation: input elements interact with each other simultaneously without regard to their distance. However, prior studies empirically show that the lack of recurrence modeling hinders Transformer from further improvement of translation quality (Dehghani et al., 2019). Modeling recurrence is crucial for capturing several essential properties of input sequence, such as structural representations (Tran et al., 2016) and positional encoding (Shaw et al., 2018), which are exactly the weaknesses of SAN (Tran et al., 2018). Recently, Chen et al. (2018) show that the representations learned by SAN-based and RNN-based encoders are complementary to each other, and merging them can improve translation performance for RNN-based NMT models. Starting from these findings, we propose to directly model recurrence for Transformer with an additional recurrence encoder. The recurrence encoder recurrently reads word embeddings of input sequence and outputs a sequence of hidden states, which serves as an additional information source to the Transformer decoder. In addition to the standard RNN, we propose to implement recurrence modeling with a novel attentive recurrent network (ARN), which combines advantages of both SAN and RNN. Instead of recurring over the individual symbols of sequences like RNN, the ARN recurrently revises its representations over a set of feature vectors, which are extracted by an attention model from the input sequence. Accordingly, ARN combines the strong global modeling capacity of SAN with the recurrent bias of RNN. We evaluate the proposed approach on widely-used WMT14 English  German and WMT17 Chinese  English translation tasks. Experimental results show that the additional recurrence encoder, implemented with either RNN or ARN, Source Multi-Head Attention Feed Forward Add & Norm Add & Norm   Positional Encoding N  Softmax Output Probabilities Recurrence Modeling Feed Forward Norm Add & Norm Source Embedding Multi-Head Attention Add & Norm N t h D ec od e r L a ye r S e l fA tt e n ti o n E n c od e r R ec u rr e n ce E n c od e r Source Embedding Source Multi-Head Attention Feed Forward Add & Norm Add & Norm   Positional Encoding N  Target Embedding Target Multi-Head Attention Feed Forward Add & Norm Add & Norm   Positional Encoding N  Multi-Head Attention Add & Norm Softmax Output Probabilities [AAAI2019] Recurrence Encoder Multi-Head Attention Add & Norm Feed Forward Add & Norm Multi-Head Attention Add & Norm Figure 1: The architecture of Transformer. In transformer, SELF-ATT (  ) computes attention over the input H n  1 e as follows: SELF-ATT ( H n  1 e ) = softmax ( QK (cid:62)  d k ) V (3) where { Q , K , V } are query, key and value vectors that are transformed from the input representations H n  1 e . d k is the scaling factor where the d k is the dimension size of the query and key vectors. Formally, the output of the first sub-layer C ne and the second sub-layer H ne are sequentially calculated as: C ne = LN (cid:0) SELF-ATT ( H n  1 e ) + H n  1 e (cid:1) , (1) H ne = LN (cid:0) FFN ( C ne ) + C ne (cid:1) , (2) where SELF-ATT (  ) , LN (  ) , and FFN (  ) are respectively self-attention mechanism, layer normalization, and feed-forward network with ReLU activation in between. consistently improves translation performance, demonstrating the necessity of modeling recurrence for Transformer. Specifically, the ARN implementation outperforms its RNN counterpart, which confirms the strength of ARN. Further analyses reveal that our approach benefits from a short-cut that bridges the source and target sequences with shorter path. Among all the model variants, the implementation with shortest path performs best, in which the recurrence encoder is single layer and its output is only fed to the top decoder layer. It consistently outperforms its multiple deep counterparts, such as multiple-layer recurrence encoder and feeding the output of recurrence encoder to all the decoder layers. In addition, our approach indeed generates more informative encoder representations, especially representative on syntactic structure features, through conducting linguistic analyses on probing tasks (Conneau et al., 2018). Figure 1 shows the model architecture of Transformer. The encoder is composed of a stack of N identical layers, each of which has two sub-layers. The first sub-layer is a self-attention network, and the second one is a position-wise fully connected feed-forward network. A residual connection (He et al., 2016) is employed around each of two sublayers, followed by layer normalization (Ba et al., 2016). The decoder is also composed of a stack of N identical layers. In addition to two sub-layers in each decoder layer, the decoder inserts a third sublayer D nd to perform attention over the output of the encoder H Ne : C nd = LN (cid:0) SELF-ATT ( H n  1 d ) + H n  1 d (cid:1) , (4) D nd = LN (cid:0) ATT ( C nd , H Ne ) + C nd (cid:1) , (5) H nd = LN (cid:0) FFN ( D nd ) + D nd (cid:1) , (6) where ATT ( C nd , H Ne ) denotes attending the top encoder layer H Ne with C nd as query. The top layer of the decoder H Nd is used to generate the final output sequence. In this section, we first describe the architecture of the introduced recurrence encoder and elaborate two types of neural network that are used as recurrence encoder in this work. Then we introduce the integration of recurrence encoder into the Transformer. Specifically, two strategies are presented to fuse the representations produced by the recurrence encoder and the conventional encoder. Finally we present the short-cut connection between the recurrence encoder and the decoder that we found very effective to use the learned representation to improve the translation performance under the proposed architecture. (a) standard RNN, and (b) the proposed ARN. Figure 2 shows the architecture of the introduced recurrence encoder which reads word embeddings of source words and outputs a sequence of hidden states that embeds recurrent information. Similar to the Transformer encoder, it has a stack of N identical layers, each of which has two sub-layers. The first one is a recurrence modeling network and the second is a fully connected feed-forward network: C nr = LN ( REC ( H n  1 r ) + H n  1 r ) , (7) H nr = LN ( FFN ( C nr ) + C nr ) , (8) where REC (  ) is the function of recurrence modeling. Note that at the bottom layer of the recurrence encoder ( N =1), we do not employ a residual connection on the recurrence sub-layer (i.e. Equation 7), which releases the constraint that C 1 r should share the same length with input embeddings sequence E in 1 . This offers a more flexible choice of the recurrence functions. There are many possible ways to implement the general idea of recurrence modeling REC (  ) . The aim of this paper is not to explore this whole space but simply to show that some fairly straightforward implementations work well. In this work, we investigate two representative implementations, namely RNN and its variation attentive current network that combines advantages of both RNN and attention models, as shown in Figure 3. 1 The input of the lowest layer in the recurrence encoder is the word embeddings of input sequence E in . Recurrent Neural Network (RNN) An intuitive choice of recurrence modeling is RNN, which is a standard network to model sequence orders. In this work, we use a bidirectional RNN (BiRNN), which is widely applied in RNN-based NMT models (Bahdanau et al., 2015; Chen et al., 2018). Each hidden state in the output representations H n RNN = { h n 1 , . . . , h nJ } is calculated as h nj = (cid:2)  h j ;  h j (cid:3) , (9)  h j =  f (  h j  1 , h n  1 j ) , (10)  h j =  f (  h j +1 , h n  1 j ) , (11) where  f (  ) and  f (  ) are the activation functions of forward and backward RNN respectively, which can be implemented as LSTM (Hochreiter and Schmidhuber, 1997) or GRU (Cho et al., 2014). h n 0 is the initial state of RNN, which is the mean of H n  1 RNN . H 0 RNN represents the word embeddings of the input sequence. Attentive Recurrent Network (ARN) We can also extend RNN by recurring over a set of feature vectors extracted with an attention model, which allows the model to learn a compact, abstractive feature vectors over the input sequence. Specifi-cally, the ARN performs T recurrent steps on the attentive output of the input representation H n  1 r : h nt = f ( h nt  1 , c nt ) , (12) c nt = ATT ( h nt  1 , H n  1 r ) . The output representations H n ARN = { h n 1 , . . . , h nT } are fed to the subsequent modules. Analogous to Equations 9-11, ARN can be extended to the bidirectional variant, i.e. BiARN, Softmax Output Probabilities Multi-Head Attention Add & Norm [AAAI2019] Integration Multi-Head Attention Add & Norm Feed Forward Add & Norm Multi-Head Attention Add & Norm Multi-Head Attention Add & Norm Multi-Head Attention Add & Norm  Multi-Head Attention Add & Norm Gated Sum Softmax Output Probabilities Feed Forward Add & Norm Target Embedding Target   PosEnc Target Embedding Target   Output of Transformer Encoder Output of Recurrence Encoder PosEnc Output of Transformer Encoder Output of Recurrence Encoder N  N except that the input is the attentive context vector c nt rather than the individual representation vector of the input sequence. Note that, the number of recurrence step T is allowed to be unequal to the length of input sequence J . In contrast to RNN which recurs over the individual symbols of the input sequences, ARN recurrently revises its representations of all symbols in the sequence with an attention model. Since the output of recurrence encoder unnecessarily shares the same length with that of Transformer encoder (e.g. when ARN is used as recurrence function), combination strategy on the encoder side, such as concatenating the outputs of both encoders (Chen et al., 2018), is not an universal solution in this scenario. Accordingly, we feed the information of the additional recurrence encoder into the decoder of Transformer. Specifi-cally, we serve an additional attention layer R nd as the fourth sub-layer in each decoder block to perform attention over the output of the recurrence encoder H Nr . As shown in Figure 4, we present two strategies to integrate R nd , namely gated sum and stack , which differ at how R nd interacts with the output of attention over the Transformer encoder, i.e., D nd in Equation 5. Gated Sum The first strategy combines the outputs of the two attention sub-layers in a gating fusion (Figure 4(a)), in which the outputs of both encoders are attended simultaneously: R nd = LN (cid:0) ATT ( C nd , H Nr ) + C nd (cid:1) , (14) (cid:98) D nd =  n D nd + (1   n ) R nd , (15) H nd = LN (cid:0) FFN ( (cid:98) D nd ) + (cid:98) D nd (cid:1) , (16) where  n is an interpolation weight calculated by a logistic sigmoid function:  n = sigmoid ( D nd , R nd ) (17) As seen, the output of self-attention layer C nd serves as a query to attend the outputs of both encoders (Equations 5 and 14), and the outputs of both attention models { D nd , R nd } are combined via a gated sum (Equation 15), which is subsequently fed to the feed-forward layer (Equation 16). Stack We can also arrange the sub-layers in a stack (Figure 4(b)), in which the outputs of both encoders are attended sequentially: R nd = LN (cid:0) ATT ( D nd , H Nr ) + D nd (cid:1) , (18) H nd = LN (cid:0) FFN ( R nd ) + R nd (cid:1) , (19) The decoder first attends the output of Transformer encoder, and the attention output D nd serves as the query to attend the output of recurrence encoder (Equation 18). The introduced recurrence encoder provides an additional computation path ranging from the input sequence to the output sequence. Chung et al. (2017) and Shen et al. (2019) have shown that a shortcut for gradient back-propagation benefits language modeling. Inspired from them, we use a shorter path to transform the learned recurrence. We call this the  short-cut effect . Among all the model variants, we implement shortest path as: the recurrence encoder is single layer and its output is only fed to the top decoder layer while the first N  1 decoder layers perform the same as the standard Transformer (e.g. Equations 4-6). Accordingly, the computation path is E in  H r  R Nd  H Nd , then the decoder uses H Nd to make a target word prediction. It is much simpler than that of the conventional Transformer, which transfers information learned from input sequences across multiple stacking encoder and decoder layers. We expect it outperforms its multiple deep counterparts, such as multiple-layer recurrence encoder and feeding the output of recurrence encoder to all the decoder layers. Improving Transformer Encoder From the perspective of representation learning, there has been an increasing amount of work on improving the representation power of SAN encoder. Baw-den et al. (2018) and Voita et al. (2018) exploit external context for SAN encoder, while Yang et al. (2019) leverage the intermediate representations to contextualize the transformations in SAN. A number of recent efforts have explored ways to improve multi-head SAN by encouraging individual attention heads to extract distinct information (Strubell et al., 2018; Li et al., 2018). Concerning multi-layer SAN encoder, Dou et al. (2018, 2019) and Wang et al. (2018) propose to aggregate the multi-layer representations, and De-hghani et al. (2019) recurrently refine these representations. Our approach is complementary to theirs, since they focus on improving the representation power of SAN encoder, while we aim to complement SAN encoder with an additional recurrence encoder. Along the direction of modeling recurrence for SAN, Vaswani et al. (2017) and Shaw et al. (2018) inject absolute position encoding and relative positional encoding to consider the position information respectively. Shen et al. (2018) introduce a directional self-attention network (DiSAN), which allows each token to attend to previous (or following) tokens only. Both studies verify the necessity of modeling recurrence for SAN. We reimplemented these approaches on top of Transformer, and experimental results show that our approach outperforms them by explicitly augmenting Transformer with an additional recurrence encoder. It should be emphasized that our approach is complementary to theirs, and combining them together is expected to further improve performance, which we leave for future work. Closely related to our work, Chen et al. (2018) propose to combine SAN encoder with an additional RNN encoder. The main differences between our work and theirs are: 1) we enhance the state-of-the-art Transformer with recurrence information, while Chen et al. (2018) augment RNN-based models with SAN encoder. To this end, we propose a novel attentive recurrent network to implement the additional recurrence encoder in Transformer. We re-implemented the approach proposed by Chen et al. (2018) on top of Transformer. Experimental results indicate the superiority of our approach, which confirms our claim. In addition, we elaborately design the integration strategy to effectively feed the recurrence information to the decoder, and empirically show that the proposed model benefits from the short-cut effect. Comparison to Reviewer Network Attentive recurrent network are inspired by the reviewer network, which is proposed by Yang et al. (2016) for the image caption generation task. There are two key differences which reflect how we have generalized from the original model. First, we perform attention steps over the source embeddings instead of the encoder representations. The main reason is that the Transformer encoder is implemented as multiple layers, and higher layers generally encode global information, as indicated by Peters et al. (2018). Second, we feed the feature vectors together with the original encoder representations to the decoder. In image caption generation, the source side (i.e. image) contains much more information than the target side (i.e. caption) (Tu et al., 2017). Therefore, they aim at learning a compact and abstractive representation from the source information, which serves as the only input to the decoder. In this work, we focus on leveraging the attention model to better learn the recurrence, which we expect to complement the Transformer model. In our preliminary experiments, attending over the encoder representations does not improve performance, while feeding the feature vectors only to the decoder seriously harms performance. We conducted experiments on the widely-used WMT14 English-to-German (4.6M sentence pairs, En  De) and WMT17 Chinese-to-English (20.6M sentence pairs, Zh  En) translation tasks. All the data had been tokenized and segmented into subword symbols using byte-pair encoding (Sennrich et al., 2016) with 32K merge operations 2 . We used case-sensitive NIST BLEU score (Papineni et al., 2002) as the evaluation metric, and bootstrap resampling (Koehn et al., 2003) for statistical significance test. We implemented the proposed approaches on top of the Transformer model (Vaswani et al., 2017). Both in our model and related model of Subsection 5.3, the RNN is implemented with GRU (Cho et al., 2014) for fair comparison. We followed the configurations in Vaswani et al. (2017), and reproduced their reported results on the En  De task. We initialized parameters of the proposed models by the pre-trained baseline model. We have tested both Base and Big models, which differ at hidden size (512 vs. 1024), filter size (2048 vs. 4096), and number of attention heads (8 vs. 16). In consideration of computation cost, we studied model variations with Base model on En  De task, and evaluated overall performances with both Base and Big models on both En  De and Zh  En translation tasks. In this subsection, we conducted ablation studies to evaluate the different implementations of the proposed model, e.g., recurrence encoder and integration strategy, under the proposed architecture. Effect of Recurrence Modeling We first investigated the effect of recurrence encoder implementations, as listed in Table 1. We observed that introducing an additional recurrence encoder improves translation performance in all cases. 2 https://github.com/rsennrich/subword-nmt Model Rec. Among all model variations, BIARN outperforms its BIRNN counterpart. Concerning BIARN models, reducing the layers consistently improves performance. Specifi-cally, the 1-Layer BIARN achieves the best performances in both translation quality and training speed. This confirms the claim that the proposed approach benefits from a short-cut on gradient back-propagation. Accordingly, we adopted 1-Layer BIARN as the default setting in the following experiments. Effect of Integration Strategies We then tested the effect of different integration strategies, as showed in Table 2. We have two observations. First, feeding only to the top decoder layer consistently outperforms feeding to all decoder layers with different integration strategies. This empirically reconfirms the short-cut effect. Second, the stack strategy marginally outperforms its gated sum counterpart. Therefore, in the following experiments, we adopted the Stack + Top model in Table 2 as defaulting setting. Performances across Languages Finally, we evaluated the proposed approach on the widely used WMT17 Zh  En and WMT14 En  De data, as listed in Table 3. To make the evaluation convincing, we reviewed the prior reported systems, and built strong baselines which outperform the reported results on the same data. As seen in Table 3, modeling recurrence consistently improves translation performance across model variations (BASE and BIG models) and language pairs (Zh  En and En  De), demonstrating the effectiveness and universality of our approach. Comparison with Previous Work In order to directly compare our approach with the previous work on modeling recurrence, we re-implemented their approaches on top of the TRANSFORMERBASE in WMT14 En  De translation task. For relative position encoding, we used unique edge representations per layer and head with clipping distance k = 16 . For the DiSAN strategy, we applied a mask to the TRANSFORMER encoder, which constrains the SAN to focus on forward or backward elements. For the multi-column encoder, we re-implemented the additional encoder with six RNN layers. Table 4 lists the results. As seen, all the recurrence enhanced approaches achieve improvements over the baseline model TRANSFORMERBASE , which demonstrates the necessity of modeling recurrence for TRANSFORMER . Among these approaches, our approach ( i.e., 1-Layer BIARN Encoder) achieves the best performance. Effect of Recurrent Steps To verify the recurrence effect on the proposed model, we conducted experiments with different recurrent steps on single-layer BIARN model. As shown in Figure 5, the BLEU score typically goes up with the increase of the recurrent steps, while the trend does not hold when T > 8 . This finding is consistent with Yang et al. (2016), which indicates that conducting too many recurrent steps fails to generate a compact representation. This is exactly one of the ARN's strengths. Linguistic Analyses In this section, we conducted 10 probing tasks 3 to study what linguistic properties are captured by the encoders (Conneau et al., 2018). A probing task is a classification problem that focuses on simple linguistic properties of sentences. SeLen' predicts the length of sentences in terms of number of words. WC' tests whether it is possible to recover information about the original words given its sentence embedding. TrDep' checks whether an encoder infers the hierarchical structure of sentences. In ToCo' task, sentences should be classified in terms of the sequence of top constituents immediately below the sentence node. BShif' tests whether two consecutive tokens within the sentence have been inverted. Tense' asks for the tense of the main-clause verb. SubN' focuses on the number of the main clause's subject. ObjN' tests for the number of the direct object of the main clause. In SoMo', some sentences are modified by replacing a random noun or verb with another one and the classi-fier should tell whether a sentence has been modified. CoIn' contains sentences made of two coordinate clauses. Half of sentences are inverted the order of the clauses and the task is to tell whether a sentence is intact or modified. We used the pre-trained encoders of model variations in Table 1 to generate the sentence representations of input, which are used to carry out probing tasks. For the TRANSFORMER-BASE model, the mean of the encoder top layer representations is used as the sentence representation. For the proposed models, which have two encoders, two sentence representations are generated from the same way in base model. To make full use of the learned representations, we combined these two sentence representations via a gate as the final sentence representation to conduct the experiments. Table 5 lists the results. Clearly, the proposed models significantly improve the classification accuracies, although there is still considerable difference among different variants. More specifically,  Concerning surface properties, among the ARN variants, multi-layer ARN inversely decreases the accuracies, while 1-layer ARN consistently improves the accuracies. Considering the related results presented in Table 1 (Row 3-5), we believe that ARN benefits from the shallow structure. ARN tends to capture deeper linguistic properties, both syntactic and semantic. Especially, among these probing tasks, TrDep' and Toco' tasks are related to syntactic structure modeling. As expected, TRANSFORMER augmented with an additional encoders outperforms the baseline model, which demonstrates that the proposed models successfully model the syntactic structure. In this work, we propose to directly model recurrence for Transformer with an additional recurrence encoder. We implement the recurrence encoder with a novel attentive recurrent network as well as RNN. The recurrence encoder is used to generate recurrence representations for the input sequence. To effectively feed the recurrence representations to the decoder to guide the output sequence generation, we study two strategies to integrate the recurrence encoder into the Transformer. To evaluate the effectiveness of the proposed model, we conduct experiments on large-scale WMT14 EN  DE and WMT17 ZH  EN datasets. Experimental results on two language pairs show that the proposed model achieves significant improvements over the baseline TRANSFORMER . Linguistic analyses on probing tasks further show that our model indeed generates more informative representations, especially representative on syntactic structure features. Ziyi Dou, Zhaopeng Tu, Xing Wang, Shuming Shi, and Tong Zhang. 2018. Exploiting deep representations for neural machine translation. In EMNLP . Ziyi Dou, Zhaopeng Tu, Xing Wang, Longyue Wang, Shuming Shi, and Tong Zhang. 2019. Dynamic layer aggregation for neural machine translation. In AAAI . Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Feder-mann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, et al. 2018. Achieving human parity on automatic chinese to english news translation. arXiv preprint arXiv:1803.05567 . Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR . Sepp Hochreiter and Jurgen Schmidhuber. 1997. Long short-term memory. Neural computation , 9(8):17351780. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In ACL . Jian Li, Zhaopeng Tu, Baosong Yang, Michael R. Lyu, and Tong Zhang. 2018. Multi-Head Attention with Disagreement Regularization. In EMNLP . Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL . Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In NAACL . Mike Schuster and Kuldip K Paliwal. 1997. Bidirectional recurrent neural networks. IEEE Transactions on Signal Processing , 45(11):26732681. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In ACL . Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. 2018. Self-Attention with Relative Position Representations. In NAACL . Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. 2018. DiSAN: directional self-attention network for RNN/CNN-free language understanding. In AAAI . Yikang Shen, Shawn Tan, Alessandro Sordoni, and Aaron Courville. 2019. Ordered neurons: Integrating tree structures into recurrent neural networks. In ICLR . Emma Strubell, Patrick Verga, Daniel Andor, David Weiss, and Andrew McCallum. 2018. Linguistically-Informed Self-Attention for Semantic Role Labeling. In EMNLP . Future work includes validating the proposed model in other tasks, such as reading comprehension, language inference, and sentence classification. Another promising direction is to directly augment Transformer encoder on recurrence modeling without the additional encoder. J.Z. was supported by the National Institute of General Medical Sciences of the National Institute of Health under award number R01GM126558. We thank the anonymous reviewers for their insightful comments.
abstain abstain objective objective abstain objective abstain abstain abstain method abstain abstain abstain abstain objective abstain objective abstain abstain objective other other other other abstain abstain result abstain abstain method other other other other other other other method method abstain method abstain abstain abstain abstain abstain method abstain abstain method method abstain method other abstain abstain method abstain abstain other abstain abstain abstain abstain method method method abstain other method other abstain abstain method method method abstain abstain method other other other other objective other other other objective objective abstain abstain objective abstain abstain objective other abstain objective other abstain other other method abstain method abstain method method method method method method method abstain abstain other abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain objective objective abstain method objective abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other other