  User Satisfaction Estimation (USE) is an important yet challenging task in
goal-oriented conversational systems. Whether the user is satisfied with the
system largely depends on the fulfillment of the user's needs, which can be
implicitly reflected by users' dialogue acts. However, existing studies often
neglect the sequential transitions of dialogue act or rely heavily on annotated
dialogue act labels when utilizing dialogue acts to facilitate USE. In this
paper, we propose a novel framework, namely USDA, to incorporate the sequential
dynamics of dialogue acts for predicting user satisfaction, by jointly learning
User Satisfaction Estimation and Dialogue Act Recognition tasks. In specific,
we first employ a Hierarchical Transformer to encode the whole dialogue
context, with two task-adaptive pre-training strategies to be a second-phase
in-domain pre-training for enhancing the dialogue modeling ability. In terms of
the availability of dialogue act labels, we further develop two variants of
USDA to capture the dialogue act information in either supervised or
unsupervised manners. Finally, USDA leverages the sequential transitions of
both content and act features in the dialogue to predict the user satisfaction.
Experimental results on four benchmark goal-oriented dialogue datasets across
different applications show that the proposed method substantially and
consistently outperforms existing methods on USE, and validate the important
role of dialogue act sequences in USE.

Computation and Language