The IMPRESSIONS section of a radiology report about an imaging study is a summary of the radiologist's reasoning and conclusions, and it also aids the referring physician in con-firming or excluding certain diagnoses. A cascade of tasks are required to automatically generate an abstractive summary of the typical information-rich radiology report. These tasks include acquisition of salient content from the report and generation of a concise, easily consumable IMPRESSIONS section. Prior research on radiology report summarization has focused on single-step end-to-end models  which subsume the task of salient content acquisition. To fully explore the cascade structure and explainability of radiology report summarization, we introduce two innovations. First, we design a two-step approach: extractive summarization followed by abstractive summarization. Second, we additionally break down the extractive part into two independent tasks: extraction of salient (1) sentences and (2) keywords. Experiments on English radiology reports from two clinical sites show our novel approach leads to a more precise summary compared to single-step and to two-step-with-single-extractive-process baselines with an overall improvement in F1 score of 3-4%. A diagnostic radiology report about an examination includes FINDINGS in which the radiologist describes normal and abnormal imaging results of their analysis (Dunnick and Langlotz, 2008). It also includes IMPRESSIONS or a summary that communicates conclusions about the findings and suggestions for the referring physician; a sample report is shown in Table 1. FINDINGS are often lengthy and information-rich. According to a survey of referring physicians, IMPRESSIONS may be the only part of the report that is read (Wallis and McCoubrie, 2011). Overall, referring physicians seem to appreciate the explainability (or self-explanitoriness) of FINDINGS  there is no evidence of midline shift or mass effect . IMPRESSIONS as it helps them evaluate differential diagnoses while avoiding additional conversations with the radiologist or the need for repeat procedures. A well known end-to-end method for text summarization is two-step : extractive summarization followed by abstractive summarization. For instance, Chen and Bansal (2018) initially train extractive and abstractive systems separately and then use the extractive system as an agent in a single-agent reinforcement learning (RL) setup with the abstractive system as part of the environment. Their extractive system extracts salient sentences and the abstractive system paraphrases these sentences to produce a summary. This summary is in turn used to compute the reward for RL training. However, this single-agent setup often fails to extract some 1542 salient sentences or it extracts irrelevant ones, leading to the generation of incomplete/incorrect IMPRESSIONS . We hypothesize that granular categories of core concepts (e.g., abnormalities, procedures) can be leveraged for generating more comprehensive summaries. Thus, a separate RL agent is dedicated to the task of extracting salient keywords (core concepts) in the two-step system. The novelty in this approach is that the new, second agent can now collaborate with the first one and the two can influence each other in their extraction decisions. Multiagent reinforcement learning (MARL) requires that an agent coordinate with the other agents to achieve the desired goal. MARL often has centralized training and decentralized execution (Foerster et al., 2016; Kraemer and Banerjee, 2016). There are several protocols for MARL training, such as sharing parameters between agents and explicit (Foerster et al., 2016, 2018; Sukhbaatar et al., 2016; Mordatch and Abbeel, 2018) or implicit (Tian et al., 2020) communication between agents by using an actor-critic policy gradient with a centralized critic for all agents (Foerster et al., 2018). The aim of these protocols is to correctly assign credits so that an agent can deduce its contribution to the team's success. To train our cooperative agents that extract salient sentences and keywords, we propose a novel Differentiable Multiagent Actor-Critic (DiMAC) RL learning method. We learn independent agents in an actor-critic setup and use a communication channel to allow agents to coordinate by passing real-valued messages. As gradients are pushed through the communication channel, DiMAC is end-to-end trainable across agents. The novelties in the paper are threefold:  a summarization system that leverages core concepts via keywords, refines them and makes them the basis for more fine-grained explainability  a multi-agent RL (MARL) based extractive component for a two-step summarization framework,  a Differentiable Multi-agent Actor-Critic (Di-MAC) with independent actors leveraging a communication channel for cooperation The remaining paper is structured as follows. two-step framework. In Section 3, we introduce the DiMAC training algorithm. In Section 4, we describe training data and experiments. In Section 5, we discuss the results. In Section 6, we discuss related work. In Section 7, we present our conclusions. Problem statement. We design a two-step summarization framework that takes the FINDINGS ( F ) section of a radiology report (consisting of a sequence of sentences) and a set of keywords ( K ) as input and produces an IMPRESSIONS ( I ) section (consisting of a sequence of sentences). In the first step of the framework, the two extractors independently select words and sentences from FINDINGSF but also coordinate such that the selection of salient words is followed by the selection of the sentence comprising these words. In the next step, a seq2seq abstractor paraphrases the selected sentences to generate IMPRESSIONSI . Figure 1 illustrates the proposed framework. We refer to Table 2 for basic notations used in this paper. We often combine notations to indicate a framework component concisely. Two-step summarization framework. The proposed system includes encoder networks to encode words and sentences into vector representations. It also includes two pointer extractor networks (Vinyals et al., 2015) to determine salient words and sentences by selecting their indices. Both extractor networks run for the same number of steps; however, at each step, the output index of one extractor network is chosen while the other is set as empty (  ). When the input is  , an extractor pauses its activity and guides the other extractor in an optimal direction. encoder, E w 2 w , is run on m word embeddings of FINDINGS sentences to obtain word representations, { h E w 2 w 1 ,    , h E w 2 w m } . A convolutional network ( Conv ) is run on concatenated word ( v w ) and position ( v p ) embeddings in a sentence to obtain an intermediate sentence representation ( h s ). Then, a bi-directional LSTM sentence encoder, E s 2 s , leverages the intermediate representations to obtain the final sentence representations, { h E s 2 s 1 ,    , h E s 2 s n } . Extractors. Two LSTM based pointer extractors, i.e., word, D w 2 w , and sentence, D s 2 s , select a source word and sentence index at each step of decoding respectively. At any step j of decoding, each extractor independently uses its hidden state h D w 2 w j and h D s 2 s j to compute an attention score over its source item w i and s k as:  wi,j = softmax ( v T  ( WD h D w 2 w j + WE h E w 2 w i ))  sk,j = softmax ( v T  (  WD h D s 2 s j +  WE h E s 2 s k )) where WD , WE , v ,  WD ,  WE and  v are trainable parameters, T and  are transpose and tanh functions respectively, and softmax normalizes the scores. Word and sentence context vectors are computed using attention scores and encoder representations as c wj = (cid:80) mi =1  wi,j h E w 2 w i and c sj = (cid:80) nk =1  sk,j h E s 2 s k respectively. word or sentence extractor output is set to  is based on a switch probability q j =  ( switch ( h D w 2 w j , c wj , h D s 2 s j , c sj )) , where switch is a feed-forward network (omitted in Figure 1). The switch value of 0 or 1 indicates whether to set the output of sentence or word extractor to  . Based on its current cell state h D s 2 s j , D s 2 s computes the next cell state, both the context vectors c wj and c sj and the selected source item encoder representation, h E s 2 s  . Sharing context vectors between extractors is similar to the cross attention mechanism as described by Jadhav and Rajan (2018). In case D s 2 s is at pause (i.e., q j =0), the E s 2 s end representation is taken as the selected item representation. D w 2 w follows the same approach to compute its next state. As we lack gold standard FINDINGS keywords and sentence-wise one-to-one match between IMPRESSIONS and FINDINGS to train networks to perform selection, we heuristically obtain such labels. See Section 4.2 for details. We perform a maximum-likelihood (ML) end-to-end training of the encoder-extractor networks to minimize the following loss; (cid:80) tj =1  (1  y qj )( y wj log  wj )  y qj ( y sj log  sk )  y qj log q j , where t is the step when D s 2 s selects a dummy END , which indicates end of the extraction, and y qj , y sj and y wj are heuristi-1544 cally obtained switch, word and sentence selection labels at step j respectively. Abstractor. The abstractor condenses each selected sentence to a concise summary. We employ a pointer generator network (See et al., 2017) for this purpose. It uses a copy mechanism to solve the out-of-vocabulary (OOV) problem and a coverage mechanism to solve the repetition problem. See (See et al., 2017) for details. We independently train the abstractor using heuristically obtained one-to-one matches between FINDINGS and IMPRESSIONS sentences. As extractor and abstractor are separately trained in a two-step framework, Chen and Bansal (2018) proposed using RL training with the extractor assuming the agent role and the abstractor as part of the environment to address the separation. Furthermore, as RL loss is computed out of final summary and ground-truth IMPRESSIONS , RL training addresses the error due to heuristic labels in the pretrained networks. Unlike Chen and Bansal (2018), our setup involves multiple extractors, so we use MARL for the coordination. In other words, the word and sentence extractors D w 2 w and D s 2 s operate as RL agents a w and a s (Figure 1, right sidie). In (Foerster et al., 2018), an actor-critic MARL has a centralized critic and parameter-sharing actors. In contrast, our extractors have different characteristics, e.g., amount of selection (salient words greater than sentences) and size of source representations; therefore, we exclude parameter sharing between actors. Additionally, to not have actors influence each other's policies, we have a critic that estimates the value function by not conditioning on the actions of other agents, thereby ensuring actor independence. Furthermore, we introduce a communicator ( m ) that coordinates actors through message passing. The dedicated channel m addresses the issue of the environment appearing nonstationary due to independent agents; see (Foerster et al., 2016; Sukhbaatar et al., 2016; Mordatch and Abbeel, 2018). The channel allows gradients to flow between actors, transforming the setup into an end-to-end Differentiable Multi-agent Actor Critic (DiMAC). The actors and the communicator are initialized with the maximum-likelihood (ML) trained extractors and switch network, respectively. { s 1 ,    , s n } , respectively. At any decoding step j , actors choose actions (i.e., source selection) u a w j and u a s j by using policy networks  a w and  a s and hidden states h a w j and h a s j . Due to the communication between actors in DiMAC training, we intuitively expect some correlation in the actions. Reward. For any decoding step j , if the communicator indicates sentence selection ( m = 0 ), a sentence reward r a s j is computed using R 1 (ROUGE unigram recall) between the abstract summary  s a s j of selected sentence s a s j (out of action u a s j ) from the abstractor and a ground-truth IMPRESSIONS sentence. We sequentially match summary and IMPRESSIONS sentences such that a s learns to select relevant sentences sequentially. Similarly, word reward r a w j for selected word w a w j out of action u a w j is 1 if the word is in the subset of keywords in FINDINGS , KF , else it is 0. Again, we match selected and FINDINGS keywords sequentially. When an agent selects extra items, the reward for those selections is 0, and thus, the agent learns to select only relevant sentences and keywords. In addition, joint actions of actors eventually generate a global reward in a multi-agent cooperative setting as: r g = R 1 ( {  s a s 1 ,    ,  ,    ,  s a s t } , I )+ R 1 ( { w a w 1 ,    ,  ,    , w a w t } , KF ) , where t is the step when a s selects END and  is a hyperparameter to adjust the global word reward contribution. As KF keywords are not gold-standard, we set  = 0 . 1 ; this means that generated summary sentences drive most of the global learning. r g is included as the reward at the last step t for both actors. Action value functions Q a w j and Q a s j for actions u a w j and u a s j are estimated as E u a wj : t ,h a wj : t [ G a w j | h a w j , u a w j ] and E u a sj : t ,h a sj : t [ G a s j | h a s j , u a s j ] , respectively, where G a w j and G a s j are discounted rewards computed as (cid:80) t  j l =0  l r a w j + l and (cid:80) t  j l =0  l r a s j + l and  = 0 . 99 is a hyperparameter. Critic. Like the actors, the critic c is an LSTM based network. It runs for the same number of steps as the actors and estimates gradients to train them. As the critic is used only in training, at each step j , the critic conditions on the actors' ground-truth selection indices, y sj and y wj , as the actions and uses these indices to obtain word and sentence encoder representations. In addition to source representations, it uses its state, h c j , and attends to all encoder states, { h E w 2 w 1 ,    } and { h E s 2 s 1 ,    } ) to estimate a value function V j . V j is then used to compute advantage functions A a w j 1545 Algorithm 1 Differentiable Multi-Agent Actor Critic 1: procedure TRAIN-DIMAC 2: Initialize parameters of actors ( a w and a s ), critic ( c ) & communicator ( m ) as  a s := D s 2 s ,  a w := D w 2 w ,  c := D w 2 w &  m := switch 3: for each training episode i do 4: step j  1 5: while action u a s j (cid:54) = END do 6: compute actors & critic states 7: sample actions u a s j & u a w j 8: compute rewards r a w j & r a s j for u a s j & u a w j 9: compute message m j & value function V j 10: j  j + 1 11: compute global reward r g 12: for j = t to 1 do 13: compute discounted reward G a s j and G a w j 14: estimate action-value functions Q a w j & Q a s j 15: compute advantages A a s j & A a w j 16: accumulate critic gradient   c 17: accumulate actor gradients   a s &  a w 18: update critic  c i +1 =  c i     c 19: update actors as  a s i +1 =  a s i +    a s &  a w i +1 =  a w i +    a w 20: return  a s ,  a w &  m and A a s j for actors as Q a w j  V j and Q a s j  V j . At any step, one of the two ground-truth actions y sj / y wj is empty. Therefore, the computed value and action-value functions V j and Q j at that step intuitively become agent-specific, resulting in independent agent learning. Finally, agent specific advantage functions are used to compute actor gradients as   a w log  a w j A a w j and   a s log  a s j A a s j . Importantly, value, action-value and advantage can be calculated in a single forward pass of the actor and critic for each agent. See appendix for details and proofs. Communication. The communicator m (Fig-ure 1, red circles) passes messages between the actors. Actor previous hidden states and contexts, h a s j , h a w j , c s j and c w j , are fed to m and a sigmoidal m j is obtained. Value m j is fed to a s while 1  m j is fed to a w . The gradient of m j flows between actors during backpropagation and provides rich training signal that minimizes the learning effort. See Algorithm 1 for DiMAC training algorithm details. We preprocessed and filtered radiology reports from two medical centers in the USA (Courtesy of Princeton Radiology, Princeton and University of Colorado Health). 1 The resulting dataset comprises 37,408 radiology reports, which we randomly split into training (31,808), validation (3,740) and test sets (1,860). Table 3 gives dataset statistics. Training labels. Given an IMPRESSIONS sentence, we find a unique FINDINGS sentence with the highest sentence similarity score. We follow Chen and Bansal (2018) and Liu and Lapata (2019) and use ROUGE-L as the sentence similarity scorer. Furthermore, they use a greedy matching algorithm that takes similarity scores of all IMPRESSIONS and FINDINGS sentence combinations and yields a sequence of unique FINDINGS indices { y s 1 ,    } of size equivalent to the length of IMPRESSIONS . There is a 1-to-1 correspondence between FINDINGS sentences at indices and IMPRESSIONS sentences. We refer to the papers for more details. These 1-to-1 correspondence are used for abstractor pretraining. We use AutoPhrase (Shang et al., 2018) to extract keywords from training reports automatically. We select only high-quality keywords, K , and avoid too frequent ones as these can bias the system to only perform keyword selection. We implement an empirical threshold determined by hyperparameter search experiments. 2 We then find a subset of keywords, KF , in FINDINGSF and compile their indices { y w 1 ,    } . As the two extractors run for the same number of steps, we interleave the above sentence and word indices { y s ,    } and { y w ,    } into one sequence. In more detail, given a sentence index, all keywords indices within that sentence are placed in the sequence, followed by its index. A binary switch variable y q (with values 0 and 1) distinguishes the 1 Sentences split using Stanford CoreNLP (Manning et al., 2014). The following reports are excluded: (a) no FINDINGS and/or IMPRESSIONS ; (b) FINDINGS has fewer than 3 words; (c) FINDINGS has fewer words or fewer sentences than IMPRESSIONS . We replace special tokens like numbers, dates and abbreviations and used scispacy lemmatization. 2 AutoPhrase ranks keywords using a quality score based on frequency. index type in the sequence, i.e., index refers to sentence vs. keyword. Both extractors require, during a decoding step j , training labels y sj and y wj ; we set the value of non-available type as indicated by y qj to  . For example, when y qj is 0, y wj is  . Overall, an element in the final sequence is a tuple of y q , y s and y w and provides training labels for the switch, word and sentence extractor networks. See Appendix A for details on the interleaving of indices. Hyperparameters. Included in Appendix C. Evaluation measure. We follow standard practice and evaluate the quality of generated IMPRESSIONS by comparing against ground-truth IMPRESSIONS using ROUGE (Lin, 2004). In this section we describe the baselines we compare our model against: a wide variety of extractive and abstractive systems. Extractive systems LexRank (Erkan and Radev, 2011) is a graph-based method for computing relative importance in extractive summarization. PTGEN (See et al., 2017) introduces an encoder-decoder model that can copy words from the source text via pointing, while retaining the ability to produce novel words through the generator. PTGEN+Coverage (See et al., 2017) introduces a coverage mechanism to the original PTGEN model to avoid repetition. Zhang et al. (2018) provides an automatic generation system for radiology IMPRESSIONS using neural seq2seq learning. The model encodes background information of the radiology study and uses this information to guide the decoding process. Self supervised learning has recently gained popularity as parameters of large models can be trained with little to no labeled data. Pre-trained language models in which a transformer encoder is trained to reconstruct the original text from masked text, e.g., BERT (Devlin et al., 2018), have become an important component in recent summarization models (Liu and Lapata, 2019; Zhang et al., 2020; Za-heer et al., 2020). We also present results from experiments using these summarization models . Additionally, we experimented with a pre-trained seq2seq model which is learned using different self supervised techniques to reconstruct the original text, e.g., BART (Lewis et al., 2019). BertSumExtAbs (Liu and Lapata, 2019) is an encoder-decoder summarization framework that adopts BERT as its encoder. BERT is replaced by ClinicalBERT (Alsentzer et al., 2019) in all our experiments as it is adapted for the medical domain. At the first stage, a model with the BERT encoder accomplishes an extraction task. Then, the trained BERT encoder and a 6-layered transformer (Vaswani et al., 2017) are combined to form an abstractive system. As the encoder in the abstractive system is pre-trained multiple times in comparison to the decoder, two separate Adam optimizers (each with different warm-up steps and learning rates) are used during training. As the training is performed in two stages, BertSumExtAbs serves as the two-stage abstractive summarization system baseline for our experiments. 3 We also include results from BERTSUMAbs , a single-stage version in which encoder and decoder are trained only on the abstractive task. BART (Lewis et al., 2019) is a state of the art transformer-based seq2seq model similar to BERTSUMAbs. However, unlike BERTSUMAbs's fine-tuning of the encoder and denovo training of the decoder, for BART, both encoder and decoder are only fine-tuned. Sentence Rewrite (Chen and Bansal, 2018) is a two-step summarization model that initially extracts and then rewrites the sentences. This model serves as a two-step single agent baseline system for our experiments. In this section, we compare results from our model and various baselines using both automatic and human evaluation. Automatic Evaluation. Table 4 shows report summarization results of various models trained and tested on the same data. Our DiMAC model surpasses extractive-only and abstractive-only baselines, including LexRank and PTGEN+Coverage. It also outperforms the two-step single agent baseline model (Sentence Rewrite (Chen and Bansal, 2018)) and the two-stage BERTSUMExtAbs (Liu and Lapata, 2019). Besides the pre-trained encoder 3 We require hyperparameters somewhat different from the standard setup due to the small radiology report data size. Hyperparameter tuning yielded the following values. Batch size and initial learning rate of BERTSumExt are set to 16 and 5e-4, batch size in BERTSumExtAbs is 8 and initial learning rates of BERT and transformer decoder in BERTSumExtAbs are 0.0005 and 0.005. of BertSumExtAbs, which is an advantage compared to other baselines, a denovo training of a large size decoder with a relatively small number of radiology reports may have led to overfitting. This might explain the scores compared to the two-step systems. Furthermore, a highly sophisticated semi-supervised training of the encoder and decoder of BART-base resulted in lower performance compared to our model, despite the relatively larger size (100x) of BART. We hypothesize that pre-training mostly on a different domain text (e.g., Wikipedia, Books Corpus and News) and fine-tuning on small data could have adversely affected BART's performance in our setting. The domain difference may also contribute to the relatively lower performance of BART-base versus BERTSUMExtAbs, thereby signifying the importance of pre-training with relevant domain text. Moreover, DiMAC offers approximately 18 to 28% performance gains over (Zhang et al., 2018), a single-step single-agent summarization system designed specifically for the radiology domain. In our opinion, the performance improvements observed with DiMAC are likely driven by the extract-then-abstract mechanism combined with auxiliary (and salient) information from keywords, which mimics the actual reasoning process of radiologists. It is important to note that our model supports user-level validation by linking the predicted IMPRESSIONS sentences to sentences in FINDINGS , making the results explainable to radiologists and referring physicians. Human Evaluation. To assess the overall quality and factual correctness (Zhang et al., 2019) of the IMPRESSIONS generated by DiMAC, we obtained evaluations from two board-certified radiol-Gwet Win Tie Lose AC1 DiMAC vs. Base model Overall quality 25.00 59.37 15.63 .305 Factual correctness 12.50 84.37 03.13 .711 DiMAC vs. Ground Truth Overall quality 25.00 46.87 28.13 .082 Factual correctness 21.87 53.13 25.00 -.080 ogists. We randomly selected 16 radiology reports from the test set. For each radiology report, we presented to the evaluators its FINDINGS and three (blinded) versions of the summary, i.e., IMPRESSIONS : (1) the ground truth, (2) Sentence Rewrite (Chen and Bansal, 2018) and (3) DiMAC. As Sentence Rewrite has a similar two-step approach, i.e., extract-then-abstract, we evaluate the qualitative performance of DiMAC with Sentence Rewrite as the base model (instead of BERTSUMExtAbs as it is a two-stage single-step system and also had lower Rouge scores compared to Sentence Rewrite). We shuffled the three summaries such that the order cannot be guessed. Each radiologist rated the summaries on two measures in relation to the FINDINGS : (1) overall quality and (2) factual correctness and completeness. For example, the phrase pleu-ral effusions is a fact (or imaging finding); but the phrase small bilateral pleural effusions is a more precise description and should therefore have a better overall quality score. For each measure, 1548 we asked the radiologists to score the summary as 1, 2 or 3 for bad, borderline or good. Then we combined the assigned scores under two comparisons: (1) our model versus the base model and (2) our model versus ground truth. We have 32 evaluations in total: 2 radiologists  16 reports. We compared the scores provided by the radiologists to determine if they were the same (tie), higher (win) or lower (lose) for our model vs. ground truth and our model vs. base model. Table 5 shows that DiMAC has clearly better factual correctness than the base model: 12.5% of cases are better, 3.13% are worse; gwet AC1 (Gwet, 2008) inter-rater agreement for this result is strong. DiMAC exceeds the base model in 25% (vs. 15.6% lose) of evaluations for overall quality with moderate inter-rater agreement. DiMAC is only slightly worse than ground truth in overall quality (win: 25%, lose: 28.13%) and factual correctness (win: 21.87%, lose: 25%)  although inter-rater agreement is low in this case. Table 6 shows a radiology report from our dataset ( FINDINGS and IMPRESSIONS ) and IMPRESSIONS generated by DiMAC and the base model. Due to the hierarchical connections between words and sentences, there is significant overlap between the extracted sentences and words. This phenomenon eventually contributes to the RL sentence extraction reward and helps to extract sentences with more keywords. The keywords include disease or clinical diagnoses (e.g., nodule, lymphadenopathy, effusion), anatomical concepts (e.g., hepatic) and qualifiers (e.g., recent, multiple, bilateral). The baseline model (Chen and Bansal, 2018) erroneously states right greater than left pleural effusions, i.e., it hallucinates. In the sentence There is no axillary or hilar lymphadenopathy, the sentence reward is low and eventually it is not extracted despite having the keyword lymphadenopa-thy. Abstractive Summarization. An abstractive summary is a text consisting of novel phrases describing the content of the original text. Abstractive summarization involves a cascade of topic fusion and text generation (Hovy et al., 1999). Each task in this cascade typically requires expert-derived annotations, which is labor-intensive and time-FINDINGS from the report from a medical site There are multiple bilateral lung nodules , most consistent with metastatic disease . consuming. Thus, many recent abstractive summarization approaches focus on supervised/semi-supervised single-step end-to-end trainable models that implicitly address the sub-tasks of content acquisition and paraphrasing. As part of two-stage but single step abstractive summarization, a pretrained encoder first learns the extraction task independently. Then the pretrained encoder is embedded into an encoder-decoder abstractive summarization model to assist in better referencing the source content, e.g., Liu and Lapata (2019); Hsu et al. (2018). On the other hand, in two-step abstractive summarization, extractive summarization is followed by abstractive summarization and is trained end-to-end, e.g., Chen and Bansal (2018). Contrary to the two-stage singlestep approach, both extractive and abstractive summarization are pretrained (and function) separately in a two-step approach; however, an RL-based end-1549 to-end training enables alignment between them to generate better summaries. DiMAC is a two-step abstractive system. Multi-agent Reinforcement Learning (MARL). In a single-agent actor-critic (Sutton et al., 1999; Konda and Tsitsiklis, 2000) policy gradient method, an agent policy   is optimized by following a gradient computed using a value function estimated by a critic. The simplest MARL setup applies policy gradients independently (each agent with its own actor and critic) and thereby restricts each agent to learn only from its own action history (Tan, 1993). From each agent's point of view in this setting, the environment is not stationary and therefore, the RL stationary environment assumption is violated. MARL with communication or collaboration protocols. Foerster et al. (2018) proposed counterfactual policy gradients, which is an actor-critic policy gradient that leverages a centralized counterfactual critic that estimates value function for each actor by using actions performed by the other agents. However, unlike our setting, actors in (Fo-erster et al., 2018) are similar and share parameters. Additionally, the parameter sharing scheme has the limitation that the agents lack tighter coordination. Foerster et al. (2016), Sukhbaatar et al. (2016) and Mordatch and Abbeel (2018) proposed to tightly coordinate independent agents rather than use a dedicated channel. As incorporating an explicit communication channel mimics human (bidirec-tional) interactions, we design a similar Differentiable Multi-agent Actor-Critic (DiMAC) RL for our setup. In DiMAC, each agent selects one of its actions and communicates with the others at every point in time. Thus, the resulting joint action (influ-enced by the agents' communication) would aim to reach the desired (optimal) goal. In the future, we will experiment with more variations of MARL (such as counter-factual critic) and transformer-based networks. In this work, we introduce a novel extractive approach into a two-step RL-based summarization task (extractive-then-abstractive). This approach is a MARL (rather than the traditional single-agent RL) which includes a new agent that extracts salient keywords from the source text and collaborates with an agent that extracts salient sentences. We also present a Differentiable Multi-agent Actor-Critic (DiMAC) learning method, a novel yet simple MARL training for independent agents communicating via a dedicated channel. We apply the proposed two-step summarization model with DiMAC MARL training to English radiology reports. Results from our experiments indicate, based on automatic and human expert evaluations, that the DiMAC summarization model can outperform existing baseline models for text summarization. Our summarization model generates the IMPRESSIONS to reflect human-level inference and actionable information (e.g., salient sentences and keywords) towards supporting improved workflow efficiency and better-informed clinical diagnosis based on medical imaging findings. We thank Dr. Asik Ali Mohamed Ali and Dr. Abishek Balachandran for qualifying radiology reports and anonymized summaries for human evaluation. We also thank Jashwanth N B and Siemens Healthineers supercomputing team for training infrastructure. Furthermore, we thank the anonymous reviewers for their valuable feedback. Disclaimer . The concepts and information presented in this paper are based on research results that are not commercially available. Future commercial availability cannot be guaranteed.
abstain abstain abstain abstain objective objective abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain abstain abstain abstain abstain objective method abstain other abstain abstain abstain result abstain method abstain method abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain other abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain method method abstain abstain method abstain method method method method method abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain result result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain other other other other other other other other other other other other other other other other other abstain other other method other other method objective abstain objective objective result method other other other other other other