Podcasts have shown a recent rise in popularity. Summarization of podcasts is of practical benefit to both content providers and consumers. It helps people quickly decide whether they will listen to a podcast and/or reduces the cognitive load of content providers to write summaries. Nevertheless, podcast summarization faces significant challenges including factual inconsistencies of summaries with respect to the inputs. The problem is exacerbated by speech disfluen-cies and recognition errors in transcripts of spoken language. In this paper, we explore a novel abstractive summarization method to alleviate these issues. Our approach learns to produce an abstractive summary while grounding summary segments in specific regions of the transcript to allow for full inspection of summary details. We conduct a series of analyses of the proposed approach on a large podcast dataset and show that the approach can achieve promising results. Grounded summaries bring clear benefits in locating the summary and transcript segments that contain inconsistent information, and hence improve summarization quality in terms of automatic and human evaluation. Podcasts are one of the most popular forms of new media. As of today, over 155 million people listen to a podcast every week (Christian, 2021). With the growing interest, there is an increased demand for textual summaries that foretell the content of podcasts. Those summaries help people decide, in a few seconds, if they will listen to a podcast or subscribe to the channel. They are helpful for users who want to find podcasts previously listened to. Furthermore, they can be re-purposed for social media posts or email marketing campaigns, enabling content creators to make their podcasts accessible to a larger audience. It is desirable to generate grounded summaries from podcast transcripts, where spans of summary text are closely tethered to the original audio. Figure 1 provides an example of a grounded abstractive summary. When a user clicks on a summary segment, she will be directed to an audio clip that gives further detail of the conversational context. Grounded summaries give us a preview of notable podcast clips (Shalom, 2019) and they may further release summarization service providers from potential legal claims by directing users to the original audio. This is because, speech recognizers induce transcription errors and abstractive summarization models may hallucinate facts that are not entailed by the original (Kryscinski et al., 2020), both can cause podcast summaries to contain misleading or inaccurate information. With grounded summaries, users are able to frame, interpret, and place into context any system-generated summaries, thus reducing the barriers to deploy podcast summarization technology. One may attempt to align summary text and podcast transcripts in a post-processing step to generate grounded summaries. Unfortunately, hallucinations do not allow for proper alignments as they are not found in the transcripts (Maynez et al., 2020). Hierarchical attention models may seem promising for this task (Liu and Lapata, 2019). However, the excessive length of the transcripts makes it difficult to produce attention distributions over the entire transcripts. Recent evidence suggests that attention weights are not reliable indicators of the relative importance of inputs (Jain and Wallace, 2019), thus it remains an open question whether attention can be used to find alignments between transcripts and summary segments. In this paper, we seek to generate grounded summaries from podcast transcripts by exploring an on-demand abstractive summarizer. It mimics how a human might approach a lengthy transcript  the expert would identify a portion of the transcript that is deemed most important and relevant to the existing summary, use it as a ground to produce a new 4407 Welcome to my podcast series a normal kind of all the Poetry from a normal Kind of Love is from my chapbook presentation. piece of the summary, and that process is repeated until the summary is finished. Our summarizer employs a novel regularization technique that enables it to visit portions of the transcript in chronological order, while allowing zigzags in order to produce a coherent summary. This has another implication. It implies that we may estimate what percentage of a podcast transcript is covered by the summary and thus adjust that when necessary. Distinguishing our work from earlier research on extract-then-abstract methods (Hsu et al., 2018; Chen and Bansal, 2018; Gehrmann et al., 2018; Lebanoff et al., 2019; Jin et al., 2020; Pilault et al., 2020), we require selected transcript chunks to have high salience, but also those salient content must appear at the beginning of the selected chunks, so that the corresponding audio clips can provide good jump-in points for users to start listening. Our experiments are performed on a large podcast summarization dataset containing over 100,000 English podcasts (Clifton et al., 2020). We show that our proposed grounded summarizer can perform competitively or better than the state-of-the-art methods, including the recent methods that leverage large, pretrained models (Lewis et al., 2020; Beltagy et al., 2020) as judged by automatic metrics and human evaluation. Our contributions in this paper are as follows. We address the problem of podcast summarization by investigating an on-demand summarizer that produces grounded abstracts. The abstracts help users quickly decide if they will listen to the podcasts and offer a sampler of salient podcast clips. The on-demand summarizer does not need to encode the entire transcript, hence substantially reduces the GPU memory footprint. We conduct a series of analyses to gain insights into the impact of specific design decisions. They include how a transcript chunk should be defined, whether those transcript chunks overlap, to what extent the summary content is taken verbatim from selected chunks, and how the summary may be extended to cover more information. Through extensive experiments on a benchmark podcast dataset, we demonstrate the effectiveness of our proposed approach and show results that are comparable to human writer performance. The approach opens an avenue towards generating a new kind of abstractive summaries that allow users to verify the information consistency of summary parts against the original audio clips. 1 2 Related Work With the rapid rise of podcasts comes the need for automatic summarization of podcast transcriptions. While comparatively understudied, recent work has shown great progress. Clifton et al. (2020) present the Spotify dataset that was adopted in TREC 2020 for the podcast summarization task. 2 Our participating system in TREC 2020 focuses on identifying salient segments from transcripts and using them as input to an abstractive summarizer (Song et al., 2020). Reddy et al. (2021) develop classifiers to detect and eliminate extraneous marketing materials in podcasts to aid summarization. In this paper, we explore techniques that generate grounded podcast summaries where pieces of summary text are tied to short podcast clips. One of the most serious problems of neural abstractive summarization is that the summaries can contain factually incorrect information and hallucinations (Falke et al., 2019; Kryscinski et al., 2020; Maynez et al., 2020; Lebanoff et al., 2020). Without grounded summarization, users have to listen to the full episodes to find connections between details of the summaries and the original podcasts. If successful, grounded summaries will benefit a number of summarization tasks where the input involves lengthy transcripts, including meetings (Li et al., 2019; Koay et al., 2020, 2021; Zhong et al., 2021), medical conversations (Liu and Chen, 2019), interviews (Zhu et al., 2021), livestreams (Cho et al., 2021) and more. An extract-then-abstract strategy could be used to produce grounded abstractive summaries (Chen and Bansal, 2018; Gehrmann et al., 2018; Hsu et al., 2018; Jin et al., 2020; Pilault et al., 2020). Most of these approaches are tailored to written documents, e.g., news, Wikipedia, and scholarly articles. They extract sentences from the documents and use them as input to an abstractive summarization model to produce a summary. Nevertheless, transcripts of spoken language lack essential document structure such as sentence, paragraph and section boundaries, making it unclear how these approaches will perform on podcasts. Attention provides another mechanism for aligning the summary and transcript segments. The use of sparse attention allows a summarization model to potentially scale to longer documents (Beltagy et al., 2020; Kitaev et al., 2020; Huang et al., 2021). Hierarchical Transformer encodes multiple paragraphs in a hierarchical manner to allow them to exchange information (Liu and Lapata, 2019; Fab-bri et al., 2019; Chen and Yang, 2020). However, it is shown that attention weights are not reliable indicators of the relative importance of inputs, as alternative attention distributions would have yielded similar results (Jain and Wallace, 2019). Our approach in this paper is to better align summary segments with chunks of the transcripts to allow easy tracing of inconsistent information. It features a generator that writes a summary from beginning to end, and a savvy selector that knows when to switch to a new transcript chunk and where to switch to. Differing from PG networks (See et al., 2017) and retrieval-augmented generation (Guu et al., 2020; Lewis et al., 2021), our selector places heavy emphasis on modeling and selection of transcript chunks. A desirable chunk is expected to be about 2 minutes long and places important information at the beginning to enable easy user verification. In the following section, we present details of the model implementation. A major challenge facing podcast summarization is the dramatic length difference between source and target sequences. At a speaking rate of 122 words per minute for spontaneous speech (Polifroni et al., 1991), the full transcript of a 1-hour long episode contains roughly 7,000 words and that of a 1.5-hour long episode could reach 10,000 words. In contrast, a podcast summary is short, containing on average 61 words according to Manakul and Gales (2020). The ratio of their lengths could reach as high as 100-to-1, and this motivates our study of abstractive grounded summarization where summary segments are grounded to selected chunks of transcripts as a way of combating the inevitable errors that occur in podcast summarization. Let x be the sequence of tokens in the source transcript and y be the sequence of tokens in the summary. These tokens share the same vocabulary V . We use x C to denote a chunk of the transcript, and C gives the indices of tokens that belong to the chunk. The full transcript can be decomposed into a sequence of chunks, denoted by {C 1 ,    , CM } . The chunks may have varying sizes and overlap with each other; they are the grounds for generating a podcast summary. Our assumption is twofold. Firstly, we assume a summary segment is produced by conditioning on the previously generated tokens ( y <j ) and a specific chunk of the transcript. Secondly, there exists a function G ( x , y <j ) (Eq. (1)) that determines the most appropriate grounding chunk for generating all tokens of the segment. Particularly, when the entire transcript is treated as a single chunk, it reduces to the standard conditional generation model p  ( y j | y <j , x ) . Thus, the crucial point is a coarse segmentation of the source transcript and an alignment between the transcript chunks and summary segments. In this work we use a sliding window to produce transcript chunks, with window size W and stride size 4409 S . 3 The sizes can be measured in terms of tokens. E.g., W =256 and S =128 tokens will produce a series of fixed-length chunks that overlap with each other. The rationale for using overlapping chunks is to find those that serve both as grounds for summary generation and good jump-in points for user verification. The sizes can also be measured by the number of sentences. E.g., W =20 and S =20 sentences produce a set of varying-length, non-overlapping chunks. In spoken language, a series of consecutive short sentences often indicates the content is relatively unimportant (Marge et al., 2010). Given a summary segment (cid:101) y , we designate x C as a grounding chunk if it attains the highest score S ( x C , (cid:101) y ) (Eq. (2)). This position-biased coverage score favors the transcript chunk that covers summary bigrams and puts summary content at the beginning to aid humans in performing content verification. It measures the percentage of unique summary bigrams B ( (cid:101) y ) covered by a chunk x C . Particularly, I [ b k  x C ] is an indicator that returns 1 if the bigram b k appears in x C and 0 otherwise. Each bigram b k has an associated weight w k (Eq. (3)). If it appears in the first position of x C ( pos k = 0 ), it receives a weight of one. Otherwise, the weight is decayed according to the relative position of the bigram's first occurrence in the chunk ( pos k ) and  is a coefficient for the decay. 4 S ( x C , (cid:101) y ) = 1 |B ( (cid:101) y ) | (cid:88) b k B ( (cid:101) y ) w k I [ b k  x C ] (2) w k = 1   pos k |C| ;   [0 , 1] (3) We proceed by training a neural encoder-decoder model to generate an abstractive summary from the grounding transcript chunks. Each segment of the summary (= sentence) 5 is generated conditioned on its grounding chunk x C and all the previously generated tokens y <j . The process starts from the first 3 Discourse segmentation is beyond the scope of this work. There is little to no data available to build a discourse segmentation tool and little existing work on discourse analysis of podcasts. We refer the reader to Joty et al. (2019) for recent advances in discourse processing research. 4 If a summary segment cannot be mapped to a chunk using Eqs. (2-3), we perform the following: (cid:101) y is assigned to the first chunk C 1 if it is the first segment of the summary. Otherwise, (cid:101) y is assigned to the same chunk as the previous summary segment to improve coherence. We require x C and (cid:101) y to have a minimum of four shared bigrams (stopwords-only bigrams are excluded). Future work may consider aligning transcripts and summaries based on propositions (Ernst et al., 2020). 5 We use sentences as summary segments; other sentencelike segments are possible in future work. chunk of the transcript x C 1 . The encoder converts this grounding chunk into a sequence of hidden vectors [ h C 1 , . . . , h C m ] (Eq. (4)). The decoder predicts the next summary token y j (Eq. (5)) and continues to do so until a  switch point  is detected. At this point the current summary segment is finished and the decoder is poised to select the next transcript chunk x C new and generate a new summary segment from it. The decoding process finishes when a special symbol ( [sep] ) is predicted that indicates the end of the summary. [ h C 1 , . . . , h C m ] = Encode ( x C ) (4) y j = Decode ( y <j , [ h C 1 , . . . , h C m ]) (5) G ( x , y <j ) =   x C 1 , j = 1 x C new , j > 1 & switch G ( y <j  1 ) , j > 1 & no-switch There is a notable difference between our approach and most extract-then-abstract approaches that select important sentences from the document and provide them to the abstractor all-at-once. As illustrated in Figure 2, strong position bias causes the abstractor to use only content at the beginning of the input to generate a summary. By exposing the chunks progressively, our approach naturally makes use of this characteristic to consolidate information from multiple source chunks. It reduces the amount of computation necessary to train the encoder-decoder model, as only selected transcript chunks are encoded which is equal to the number of summary segments. Moreover, it is possible to encourage the summary to have a good coverage of the source content by specifying a minimal set of grounding chunks to be used for generation. Regularizing Chunk Selection. Learning function G ( x , y <j ) that predicts a transcript chunk x C to switch to is crucial for success at inference time. Let there be M transcript chunks and N summary segments in a training instance. We define p cj to be the model probability that the c -th chunk is predicted as the ground for generating the j -th summary segment; c  is the gold chunk obtained using Eq. (2-3). Our learning objective is a cross-entropy loss against the gold labels with a novel regularizing term R to enable chunks to be selected as per their original order in the transcript, while allowing zigzags to produce a coherent summary (Eq. (6-7)). L (  ) =  (cid:80) Nj =1 log p c  j +  R (6) R = 1 N (cid:80) Nj =1 (cid:80) Mc =1 max(0 , s cj +1  s cj ) (7) Particularly, s cj = (cid:80) cc  =1 p c  j denotes the sum of the probability assigned to all chunks up to the c -th position, in order to generate the j -th summary segment. We encourage (cid:80) Mc =1 max(0 , s cj +1  s cj ) to be a small value so that if a chunk (up to the c -th position) is assigned to the j -th summary segment, it is unlikely to be assigned to the ( j +1 )-th segment. R is designed to regularize the loss and penalize violations;  is its coefficient which will be tuned on the validation set. Given a partial summary y <j , selecting the next transcript chunk depends on two factors. Firstly, it should be a chunk that contains salient content at its beginning. We use I ( x C ) to denote the importance of the chunk. It is obtained by encoding the chunk into a vector h x C using RoBERTa (Liu et al., 2019), then apply a feedforward network to it to estimate the importance (Eq. (9)). 6 p cj  exp( I ( x C ) + R ( x C , y <j )) (8) I ( x C ) = FFN 1 ( h x C ) (9) R ( x C , y <j ) = FFN 2 ([ h x C || h y <j ]) (10) + LowRank ( h  x CW h y <j ) Secondly, the chunk may be relevant to the partial summary y <j . We define the relevance score R ( x C , y <j ) to capture two levels of interaction between the candidate chunk, represented by h x C and 6 The parameters of FFN 1 are pretrained on an extraction task that favors chunks that contain summary content at the beginning. For each chunk, we compute its position-biased coverage score (Eq. (2)) against the entire summary. 1/4 of the chunks that yield the highest coverage scores are designated as positive instances, the remaining are negative instances. FFN 1 is thus pretrained as a binary classifier. the last hidden state of the partial summary, represented by h y <j . Their linear interaction is captured by a feedforward network ( FFN 2 ) and bilinear interaction is modelled by h  x CW h y <j where a lowrank approximation is used: LowRank ( p  W q ) = ( p  U )( V  q ) . The score p cj is the likelihood that the c -th chunk is assigned to the j -th summary segment considering saliency and content relevancy. Switch Point. A skilled writer pauses after writing down a sentence. We borrow that intuition to inform the construction of a switch-point predictor. The model combines the last hidden state of the summary sequence h y <j and the embedding of the anticipated token E ( y j ) , and use a feedforward network FFN 3 to predict if the j -th decoding step corresponds to a  switch point  (Eq. (11)). During training, the last token of each summary sentence is a ground-truth switch point. At inference time, the model predicts a switch point if p ( switch ) exceeds a threshold, at which point we compute p cj to decide the next chunk. Note that the model may choose use the same transcript chunk after switching. With over 100,000 podcast episodes, the Spotify dataset (Clifton et al., 2020) is one of the largest corpora available for podcast search and summarization. It encompasses a wide range of topics: travel, business, sports, book reviews, mysteries, guided meditations, nutrition and weight loss, among others. Each episode is accompanied by an audio file, an automatic transcript generated by Google's Speech-to-Text API, 7 and metadata provided by the podcast creator. We do not use the audio data in this paper. Our summarizer takes as input a transcript and uses the creator-provided episode description as the reference summary. Data Filtering. Episode descriptions provided by podcast creators show wide variations in quality. When noisy descriptions are used as reference summaries, they can cause a summarizer to hallucinate content. We conduct aggressive filtering of the training data to remove low-quality creator descriptions so as to maintain a balance between the amount of training examples available and quality of those examples. We clean up reference summaries on the token-, sentenceand summary-level. and specific products we love! hk_uu_podcast1 In this episode, Jessica and Natalie go head-to-head in the Great Exfoliation Debate! They each advocate their own type of exfoliator and try each other's products to see if they're worth the price difference. They also do a wine pairing and talk about the pros and cons of each of the products they tried. UCF_NLP2 In this weeks episode, Jessica and Natalie go head-to-head in the great exfoliation debate. They each advocate for their own type of exfoliator, and then try each other's products for 10 minutes to see what they think. We also talk about the pros and cons of each type of product and recommend a wine to pair with this episode. Santa Julia Winemakers Reserve Mountain Blend . cued_speechUniv2 In this episode of the Great Exfoliation Debate, Jessica and Natalie talk about their favorite types of exfoliators and the pros and cons of each of their favorite products. We also do a wine pairing and talk about the benefits and drawbacks of different types of chemical and physical exfoliation products. GrndAbs-to This week we are talking about what we like to call the Great Exfoliation Debate. Because we've got two different points of view and we are going to Duke it out mano a mano this week. We will also of course do our wine pairing because we are your Somali A's and this week we're going with something a little bit more aggressive...a little bit bold. GrndAbs-sn In this episode, Natalie and Jessica debate the pros and cons of exfoliation. Exfoliation is this step in your skincare routine that is taking off all the dead skin cells on your face. And the point of Exfoliating is to reveal brighter, healthier skin while reducing the size of your pores. In this week's episode, we'll be discussing the pros, cons, and what we think is the best way to exfoliate your skin. GrndAbs-so Natalie and Jessica are back to debate the merits of exfoliation. This week, they are going mano a mano and will be debating the pros and cons of using exfoliating on your face. We will also do our wine pairing because we are your Somali A's and this week we're going with something a little bit more aggressive. We would like to recommend Santa Julia Winemakers' Reserve Mountain Blend. That is a Malbec and Cab Franc blend from 2016. It's just a bit of a middle of the road wine but super super tasty Table 1: Grounded abstractive summaries ( GrndAbs-* ) demonstrate a high level of specificity compared to summaries without grounding. Tokens that correspond to URLs, email addresses, @mentions, #hashtags, and those excessively long tokens (>25 characters) are directly removed from the summaries. Each sentence in the summary is given a salience score that is the sum of IDF scores of its words. A low score (<10) indicates the sentence contains few informative words and it is thus removed from the summary. Finally, if, after sentence removal, the reference summary is too short or cannot be properly aligned to transcript chunks (3), the instance is removed from the dataset. 8 This process filters out a substantial amount of low-quality reference summaries, yielding 40,302 episodes in the training set. The Spotify dataset has a standard test set of 1,027 episodes and 179 of them are set for human evaluation. Baselines. Our baselines consist of three of the best performing systems in the TREC 2020 competition on podcast summarization. These systems were judged the best performing by both automatic metrics and human evaluation performed by NIST assessors. All systems make use of the BART-large model (Lewis et al., 2020). The model is tuned first on a news summarization dataset, i.e., CNN/DM or XSum, then fine-tuned on the podcast dataset. Due to the long length of the transcripts, Karlbom and Clifton (2020) describe a combined Longformer-BART model that replaces the BART attention lay-8 A summary is required to contain a minimum of 10 BPE tokens and have >2 shared bigrams with all of its grounding chunks. ers with attentions of Longformer (Beltagy et al., 2020); their system is named hk_uu_podcast1 . Song et al. (2020) develop an extractive module to select segments from transcripts, then integrate the extractor with BART abstractor to generate summaries ( UCF_NLP2 ). Their baseline ( UCF_NLP1 ) directly truncates the transcript to the first 1,024 tokens. Manakul and Gales (2020) develop a similar baseline ( cued_speechUniv3 ) using the first 1,024 tokens. Further, they perform sentence filtering using a hierarchical attention model ( cued_speechUniv1/2/4 ) and ensembles of models from different data shuffles and checkpoints ( cued_speechUniv1/2 ). In this paper, our system is called GrndAbs for generating grounded abstracts. It has 4 options: -to, -tn, -so, -sn , indicating the sliding window is defined in terms of tokens ( -t ) or sentences ( -s ), overlapping ( -o ) or non-overlapping ( -n ). We obtain outputs from these competitive baselines and our system to examine both the successes and failures of these attempts. Experimental Settings. Our encoder-decoder model uses BART-large as the base model before fine-tuning it on the podcast dataset. We use the AdamW (Loshchilov and Hutter, 2017) optimizer, where the momentum parameters are set to 0.9 and 0.999. The regularizing coefficient  is tuned on the validation set in the range of { 0 , 0 . 01 , 0 . 1 , 1 } . For summary decoding, we use beam search with a beam size K =4 and a length penalty p =2. Our 4412 Run ID R-1(%) R-2(%) R-L(%) BertS(%) BLEURT SummL cued_speechUniv1 30.54 11.25 21.05 84.17 -0.7434 58.16 cued_speechUniv2 30.52 11.36 21.16 84.20 -0.7491 56.93 cued_speechUniv3 28.44 9.55 19.52 83.77 -0.7897 55.58 cued_speechUniv4 29.00 10.42 19.95 83.99 -0.7781 51.75 UCF_NLP1 30.09 12.07 21.75 84.16 -0.7508 57.35 UCF_NLP2 30.44 11.99 21.67 84.14 -0.7382 57.85 hk_uu_podcast1 29.02 10.70 20.66 84.21 -0.7992 44.63 GrndAbs-so 25.42 7.95 16.93 82.62 -0.8164 80.44 GrndAbs-sn 25.58 8.27 16.99 82.64 -0.8220 78.80 GrndAbs-to 25.79 8.38 17.15 82.67 -0.8028 82.98 GrndAbs-tn 25.79 8.25 17.20 82.71 -0.8130 79.90 Table 2: Results on the standard test set containing 1,027 episodes. sliding window, measured in terms of tokens or sentences, only contain whole sentences. We use the Byte-Pair Encoding (BPE) tokenizer with a vocabulary size V =50,265. For transcripts and reference summaries, we use the SpaCy tool to segment them into sentences (model en_core_web_lg 2.2.5 ). Example Summaries. In Table 1, we provide a direct comparison of system summaries. This podcast is hosted by Natalie and Jessica who call themselves  Skincare Sommeliers.  The episode is named  The Great Exfoliation Debate . We find that grounded abstractive summaries ( GrndAbs-* ) have a higher level of specificity compared to summaries without grounding. Segments of grounded summaries are tied to specific transcripts chunks. If a listener finds a summary segment interesting, they can tap to hear the selected summary segment in context. Our baselines are highly competitive. Their summaries tend to contain more generic content. The description provided by podcast creators is relatively short and at times it does not directly summarize the episode. There are clear benefits in automatic summarization of podcasts, which can reduce the cognitive load and the time it takes for podcast creators to write the summary. Automatic Metrics. In Table 2, we report results on the standard test set containing 1,027 podcast episodes. The metrics include ROUGE (Lin, 2004) variants that compare system summaries with creator descriptions based on n-gram overlap. Further, we experiment with recently developed metrics: BertScore (Zhang et al., 2020) and BLEURT (Sel-lam et al., 2020) that draw on deep neural representations to evaluate generated text. Our approach does not outperform the baselines in ROUGE evaluation against creator descriptions. However, the gap has been substantially reduced when more advanced metrics (BertScore and BLEURT) are considered. There are two possible explanations. First, grounded summaries are about 50% longer than plain abstractive summaries. Their average length is about 80 words per summary, yielding low precision scores. Second, the quality of creator descriptions can be poor. Jones et al. (2020) report only 40% of such descriptions are of Good or Excellent quality, indicating future work may consider creating high-quality ground-truth summaries. Among the four variants of our approach, we observe that their difference is not prominent. The token-based, non-overlapping windows (-tn) variant outperforms others in terms of R-1 and R-L. This system is used in subsequent experiments and analyses. Human Evaluation. It is imperative to perform human evaluation given that creator-provided descriptions are of poor quality and ground-truth summaries are nonexistent. We follow the TREC guidelines to ask human evaluators to assign each summary to one of the four grades: Excellent , Good , Fair and Poor . The excellent summary will accurately conveys the most important content of the episode (topical content, genre, and participants). It should contain almost no redundant material, be coherent, comprehensible, and has no grammatical errors (Jones et al., 2020). We also asked the human evaluators to answer 8 yes/no questions re-4413 Q1: People Q2: People Q3: Main Q4: Podcast Q5: Title Q6: Summ Q7: Good Q8: Start/End System Names Add Info Topics Format Context Redund English Points creator_description 60.08 50.19 80.81 59.61 57.00 16.28 88.76 60.16 hk_uu_podcast1 64.15 47.29 85.63 57.62 58.95 10.85 94.76 70.35 UCF_NLP2 67.38 51.55 87.02 63.57 62.52 14.40 95.15 71.71 cued_speechUniv2 69.12 50.67 87.98 64.73 63.62 12.87 94.93 77.00 GrndAbs-tn 75.15 64.47 89.73 69.51 66.15 17.09 94.55 73.35 Table 4: Average scores per human judgment of 179 testing summaries on 8 Yes/No questions. Q1 Does the summary include names of the main people (hosts, guests, characters) involved or mentioned in the podcast? Q2 Does the summary give any additional information about the people mentioned (such as their job titles, biographies, personal background, etc)? Q3 Does the summary include the main topic(s) of the podcast? Q4 Does the summary tell you anything about the format of the podcast ; e.g. whether it's an interview, whether it's a chat between friends, a monologue, etc? Q5 Does the summary give you more context on the title of the podcast? Q6 Does the summary contain redundant information ? Q7 Is the summary written in good English ? Q8 Are the start and end of the summary good sentence and paragraph start and end points? garding the quality of the summary as (Jones et al., 2020) suggested, those questions are shown in Table 5. We conduct these experiments on the test set containing 179 podcast episodes as (Jones et al., 2020) did, where each summary is evaluated by five Master workers recruited on the mechanical turk. As shown in Table 3, we find that humans prefer the lengthier grounded abstractive summaries, which substantially outperform all baselines. 25% of grounded abstractive summaries are rated as Excellent and 76% of them receive a rating of either Excellent or Good . Table 4 shows the results of the 8 questions. Comparing to previous best systems, our grounded abstractive summaries have a significant performance gain on retrieving important information including People Names(+6.03%), People Additional Information(+12.92%), Main Topics(+1.75%), Podcast Format(+4.78%) and Title related context(+2.47%) with slight redundancy. We are curious to know how well our system performs on predicting grounding chunks: G ( x , y <j ) . In this study, we assume switch points are known and report results on the validation set. Our decoder starts from the first transcript chunk and predicts the next chunk at each switch point. We find that it achieves an accuracy of 86.02% on identifying ground-truth chunks. Next, we examine the performance of switch point prediction. On the validation set, we observe that the predictor achieves 98.75%, 84.95% and 91.33%, respectively, for precision, recall and F-score. Moreover, each summary has an average of 3.67 switch points. A majority of the time (92.42%) the model decides to use the current chunk to continue to decode the next summary segment. At a small percentage (7.58%) the model decides to find to a new grounding chunk. We find 1.24 unique grounding chunks per summary. The statistics suggest that identifying grounding chunks is crucial for summary generation. Grounded Summaries. In Table 7, we measure the percentage of summary n-grams that appear in the transcripts (for all baselines) or grounding chunks (for our approach). While the distributions of unigrams are largely similar, we observe that grounded abstractive summaries tend to reuse more bigrams and trigrams of their grounding chunks. Moreover, for trigrams that are found in the grounding chunks, we find 70% of them tend to appear at the beginning  the front half of the chunks. These results suggest that the grounding chunks identified by our approach can provide effective support for summary generation. We manually analyze a large amount of transcripts and their creator descriptions to identify the challenging points of podcast summarization in Table 8:  Substantial lexical mismatch exists between the spoken and written form of descriptions. Speech recognition errors are abundant. E.g.,  by Hans Christian Andersen  has been misrecognized into 4414 Excellent The summary accurately conveys all the most important attributes of the episode, which could include topical content, genre, and participants. buy homes Christian Andersen .  The creator descriptions are sometimes highly abstractive, do not always summarize the episode and contain teasers. E.g.,  A male perspective podcast to start a conversation...  and  Ever wondered how Ed Sheeran became famous .  The transcripts contain advertising inserts, e.g.,  I need to tell you about our sponsor...  and the same description is used for different episodes that causes confusion to the model, e.g.,  The goal of Daily Fortnite is to build a community...  6 Conclusion In this paper, we investigate podcast summarization to produce textual summaries for podcast episodes that help listeners to understand why they might want to play those podcasts. We present a new kind of podcast summary where spans of summary text are tethered to the original audio to allow users to interpret system-generated abstracts in context. The authors would like to thank all anonymous reviewers for their insightful comments which helped improve this paper. This research was supported in part by the National Science Foundation (NSF) Grant #2143792.
abstain abstain abstain abstain abstain objective method objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain abstain objective abstain abstain method method abstain objective objective objective objective method objective objective objective objective abstain abstain method abstain objective abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain method abstain abstain method method abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain method abstain method abstain method abstain method abstain method abstain abstain abstain abstain abstain abstain abstain method abstain method abstain abstain abstain abstain abstain method abstain method abstain other method abstain abstain abstain method abstain abstain other method method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain method abstain abstain abstain other abstain other other