Multi-task Training Format Framework Motivated by Whisper (Radford et al., 2023), to incorporate
different kinds of audio, we propose a multitask training format framework as follows:

• Transcription Tag: The initiation of prediction is denoted using a transcription tag. The <|startof-
transcripts|> is employed to indicate the tasks involve accurately transcribing the spoken words and

7

capturing the linguistic content of a speech recording, such as speech recognition and speech translation
tasks. For other tasks, the <|startofanalysis|> tag is utilized.

• Audio Language Tag: Then, we incorporate a language tag that indicates the spoken language in
the audio. This tag uses a unique token assigned to each language present in our training set, eight
languages in totally. In the case where an audio segment does not contain any speech, such as natural
sounds and music, the model is trained to predict a <|unknown|> token.
Qwen-Audio