Spontaneous Informal Speech Dataset for Punctuation Restoration
<|reference_start|>Spontaneous Informal Speech Dataset for Punctuation Restoration: Presently, punctuation restoration models are evaluated almost solely on well-structured, scripted corpora. On the other hand, real-world ASR systems and post-processing pipelines typically apply towards spontaneous speech with significant irregularities, stutters, and deviations from perfect grammar. To address this discrepancy, we introduce SponSpeech, a punctuation restoration dataset derived from informal speech sources, which includes punctuation and casing information. In addition to publicly releasing the dataset, we contribute a filtering pipeline that can be used to generate more data. Our filtering pipeline examines the quality of both speech audio and transcription text. We also carefully construct a ``challenging" test set, aimed at evaluating models' ability to leverage audio information to predict otherwise grammatically ambiguous punctuation. SponSpeech is available at https://github.com/GitHubAccountAnonymous/PR, along with all code for dataset building and model runs.<|reference_end|>
@article{liu2024spontaneous,
    title={Spontaneous Informal Speech Dataset for Punctuation Restoration},
    author={Xing Yi Liu and Homayoon Beigi},
    journal={Recognition Technologies, Inc. Technical Report, 2024},
    year={2024},
    doi={10.13140/RG.2.2.33136.88329},
    number={RTI-20240917-01},
    archivePrefix={arXiv},
    eprint={2409.11241},
    primaryClass={cs.CL cs.HC cs.LG cs.SD eess.AS}
}