  User queries for a real-world dialog system may sometimes fall outside the
scope of the system's capabilities, but appropriate system responses will
enable smooth processing throughout the human-computer interaction. This paper
is concerned with the user's intent, and focuses on out-of-scope intent
classification in dialog systems. Although user intents are highly correlated
with the application domain, few studies have exploited such correlations for
intent classification. Rather than developing a two-stage approach that first
classifies the domain and then the intent, we propose a hierarchical multi-task
learning approach based on a joint model to classify domain and intent
simultaneously. Novelties in the proposed approach include: (1) sharing
supervised out-of-scope signals in joint modeling of domain and intent
classification to replace a two-stage pipeline; and (2) introducing a
hierarchical model that learns the intent and domain representations in the
higher and lower layers respectively. Experiments show that the model
outperforms existing methods in terms of accuracy, out-of-scope recall and F1.
Additionally, threshold-based post-processing further improves performance by
balancing precision and recall in intent classification.

Computation and Language