Recurrent Neural Network Structured Output Prediction for Spoken Language Understanding
Recurrent Neural Networks (RNNs) have been widely used for sequence modeling due to their strong capabilities in modeling temporal dependencies. In this work, we study and evaluate the effectiveness of using RNNs for slot filling, a key task in Spoken Language Understanding (SLU) with special focus on modeling label sequence dependencies. Recent work on slot filling using RNNs did not model label sequence dependencies explicitly. We propose to introduce label dependencies in model training by feeding previous output label, using a sampling approach on true and predicted labels, to the current sequence state. The revised RNN model is evaluated on slot filling task using standard ATIS dataset. Our results show that the proposed methods consistently outperform the baseline RNN system. Error analysis illustrates clear advantages of learning output sequence dependencies in RNN sequence modeling.