  Simultaneous translation systems start producing the output while processing
the partial source sentence in the incoming input stream. These systems need to
decide when to read more input and when to write the output. These decisions
depend on the structure of source/target language and the information contained
in the partial input sequence. Hence, read/write decision policy remains the
same across different input modalities, i.e., speech and text. This motivates
us to leverage the text transcripts corresponding to the speech input for
improving simultaneous speech-to-text translation (SimulST). We propose
Decision Attentive Regularization (DAR) to improve the decision policy of
SimulST systems by using the simultaneous text-to-text translation (SimulMT)
task. We also extend several techniques from the offline speech translation
domain to explore the role of SimulMT task in improving SimulST performance.
Overall, we achieve 34.66% / 4.5 BLEU improvement over the baseline model
across different latency regimes for the MuST-C English-German (EnDe) SimulST
task.

Sound;Computation and Language;Audio and Speech Processing