How was the dataset collected?
data was collected using crowdsourcing where speakers were recorded saying random ordered phrases for each intent twice
Whereas conventional spoken language understanding (SLU) systems map speech to text, and then text to intent, end-to-end SLU systems map speech directly to intent through a single trainable model. Achieving high accuracy with these end-to-end models without a large amount of training data is difficult. We propose a method to reduce the data requirements of end-to-end SLU in which the model is first pre-trained to predict words and phonemes, thus learning good features for SLU. We introduce a new SLU dataset, Fluent Speech Commands, and show that our method improves performance both when the full dataset is used for training and when only a small subset is used. We also describe preliminary experiments to gauge the model's ability to generalize to new phrases not heard during training.
Spoken language understanding (SLU) systems infer the meaning or intent of a spoken utterance BIBREF0 . This is crucial for voice user interfaces, in which the speaker's utterance needs to be converted into an action or query. For example, for a voice-controlled coffee machine, an utterance like “make me a large coffee with two milks and a sugar, please” might have an intent representation like {drink: "coffee", size: "large", additions: [{type: "milk", count: 2}, {type: "sugar", count: 1}]}. The conventional SLU pipeline is composed of two modules: an automatic speech recognition (ASR) module that maps the speech to a text transcript, and a natural language understanding (NLU) module that maps the text transcript to the speaker's intent BIBREF1 , BIBREF2 , BIBREF3 . An alternative approach that is beginning to gain popularity is end-to-end SLU BIBREF4 , BIBREF5 , BIBREF6 , BIBREF7 , BIBREF8 . In end-to-end SLU, a single trainable model maps the speech audio directly to the speaker's intent without explicitly producing a text transcript (Fig. FIGREF4 ). Unlike the conventional SLU pipeline, end-to-end SLU: End-to-end models have been made possible by deep learning, which automatically learns hierarchical representations of the input signal BIBREF9 , BIBREF10 , BIBREF11 , BIBREF12 , BIBREF13 . Speech is natural to represent in a hierarchical way: waveform INLINEFORM0 phonemes INLINEFORM1 morphemes INLINEFORM2 words INLINEFORM3 concepts INLINEFORM4 meaning. However, because speech signals are high-dimensional and highly variable even for a single speaker, training deep models and learning these hierarchical representations without a large amount of training data is difficult. The computer vision BIBREF14 , BIBREF15 , natural language processing BIBREF16 , BIBREF17 , BIBREF18 , BIBREF19 , BIBREF20 , and ASR BIBREF21 , BIBREF22 communities have attacked the problem of limited supervised training data with great success by pre-training deep models on related tasks for which there is more training data. Following their lead, we propose an efficient ASR-based pre-training methodology in this paper and show that it may be used to improve the performance of end-to-end SLU models, especially when the amount of training data is very small. Our contributions are as follows: