title: Sentence-wise Speech Summarization: Task, Datasets, and End-to-End
    Modeling with LM Knowledge Distillation
summary: This paper introduces a novel approach called sentence-wise speech
  summarization (Sen-SSum), which generates text summaries from a spoken document
  in a sentence-by-sentence manner. Sen-SSum combines the real-time processing of
  automatic speech recognition (ASR) with the conciseness of speech
  summarization. To explore this approach, we present two datasets for Sen-SSum:
  Mega-SSum and CSJ-SSum. Using these datasets, our study evaluates two types of
  Transformer-based models: 1) cascade models that combine ASR and strong text
  summarization models, and 2) end-to-end (E2E) models that directly convert
  speech into a text summary. While E2E models are appealing to develop
  compute-efficient models, they perform worse than cascade models. Therefore, we
  propose knowledge distillation for E2E models using pseudo-summaries generated
  by the cascade models. Our experiments show that this proposed knowledge
  distillation effectively improves the performance of the E2E model on both
  datasets.
paper_authors._id: 66ac3bdbae2981107ddb3890
paper_authors.name: Kohei Matsuura
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:24:25.010Z
paper_authors._id: 645092abfc5e5ff919b4aa0a
paper_authors.avatarUrl: /avatars/72e5d4db1e489924d03ab30041923a41.svg
paper_authors.fullname: Kohei Matsuura
paper_authors.type: user
paper_authors.user: komats
paper_authors._id: 66ac3bdbae2981107ddb3891
paper_authors.name: Takanori Ashihara
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:24:30.725Z
paper_authors._id: 631807e61d6018cb851c47bb
paper_authors.avatarUrl: /avatars/5e7e92b4dda08ad385bd858c35677577.svg
paper_authors.fullname: Takanori Ashihara
paper_authors.type: user
paper_authors.user: ashi-ta
paper_authors._id: 66ac3bdbae2981107ddb3892
paper_authors.name: Takafumi Moriya
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:24:36.514Z
paper_authors._id: 664ef5d055bbfe79416102b2
paper_authors.avatarUrl: /avatars/8eca89f2d5f24420b2f99d4be01c2d2c.svg
paper_authors.fullname: Takafumi Moriya
paper_authors.type: user
paper_authors.user: tmoriya777
paper_authors._id: 66ac3bdbae2981107ddb3893
paper_authors.name: Masato Mimura
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:24:41.551Z
paper_authors._id: 65d56a4187c889ddc40769c9
paper_authors.avatarUrl: /avatars/3f0df20f3b98b980bfbe0219b0a3a63c.svg
paper_authors.fullname: Masato Mimura
paper_authors.type: user
paper_authors.user: mimura
paper_authors._id: 66ac3bdbae2981107ddb3894
paper_authors.name: Takatomo Kano
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:24:46.869Z
paper_authors._id: 64f19c83963e4bc2b7cc9dd7
paper_authors.avatarUrl: /avatars/e0b9854dad00ca3cf604ead1401d3719.svg
paper_authors.fullname: takatomo kano
paper_authors.type: user
paper_authors.user: tkano
paper_authors._id: 66ac3bdbae2981107ddb3895
paper_authors.name: Atsunori Ogawa
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:24:53.393Z
paper_authors._id: 651d1582a4e688e310ce7c81
paper_authors.avatarUrl: /avatars/8d75a4e4b97046f24b01f9158f8adea4.svg
paper_authors.fullname: Atsunori Ogawa
paper_authors.type: user
paper_authors.user: atsunori0108
paper_authors._id: 66ac3bdbae2981107ddb3896
paper_authors.name: Marc Delcroix
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-02T09:24:58.750Z
paper_authors._id: 6459a69c5a263a4c87b72d0e
paper_authors.avatarUrl: /avatars/efff233120a8e727e90a74779d520cbb.svg
paper_authors.fullname: Marc Delcroix
paper_authors.type: user
paper_authors.user: mdelcroix
paper_title: Sentence-wise Speech Summarization: Task, Datasets, and End-to-End
    Modeling with LM Knowledge Distillation
paper_summary: This paper introduces a novel approach called sentence-wise speech
  summarization (Sen-SSum), which generates text summaries from a spoken document
  in a sentence-by-sentence manner. Sen-SSum combines the real-time processing of
  automatic speech recognition (ASR) with the conciseness of speech
  summarization. To explore this approach, we present two datasets for Sen-SSum:
  Mega-SSum and CSJ-SSum. Using these datasets, our study evaluates two types of
  Transformer-based models: 1) cascade models that combine ASR and strong text
  summarization models, and 2) end-to-end (E2E) models that directly convert
  speech into a text summary. While E2E models are appealing to develop
  compute-efficient models, they perform worse than cascade models. Therefore, we
  propose knowledge distillation for E2E models using pseudo-summaries generated
  by the cascade models. Our experiments show that this proposed knowledge
  distillation effectively improves the performance of the E2E model on both
  datasets.
paper_ai_summary: A sentence-wise speech summarization approach combines ASR with text summarization using cascade and end-to-end Transformer models, with knowledge distillation improving E2E model performance.
paper_ai_keywords: automatic speech recognition speech summarization Transformer-based models cascade models end-to-end models knowledge distillation