  This paper proposes a deep neural network model for joint modeling Natural
Language Understanding (NLU) and Dialogue Management (DM) in goal-driven
dialogue systems. There are three parts in this model. A Long Short-Term Memory
(LSTM) at the bottom of the network encodes utterances in each dialogue turn
into a turn embedding. Dialogue embeddings are learned by a LSTM at the middle
of the network, and updated by the feeding of all turn embeddings. The top part
is a forward Deep Neural Network which converts dialogue embeddings into the
Q-values of different dialogue actions. The cascaded LSTMs based reinforcement
learning network is jointly optimized by making use of the rewards received at
each dialogue turn as the only supervision information. There is no explicit
NLU and dialogue states in the network. Experimental results show that our
model outperforms both traditional Markov Decision Process (MDP) model and
single LSTM with Deep Q-Network on meeting room booking tasks. Visualization of
dialogue embeddings illustrates that the model can learn the representation of
dialogue states.

Computation and Language;Artificial Intelligence