Recent studies in dialogue state tracking (DST) leverage historical information to determine states which are generally represented as slot-value pairs. However, most of them have limitations to efficiently exploit relevant context due to the lack of a powerful mechanism for modeling interactions between the slot and the dialogue history. Besides, existing methods usually ignore the slot imbalance problem and treat all slots indiscriminately, which limits the learning of hard slots and eventually hurts overall performance. In this paper, we propose to enhance the DST through employing a contextual hierarchical attention network to not only discern relevant information at both word level and turn level but also learn contextual representations. We further propose an adaptive objective to alleviate the slot imbalance problem by dynamically adjust weights of different slots during training. RETRACTED This paper was retracted. Experimental results show that our approach reaches 52.68% and 58.55% joint accuracy on MultiWOZ 2.0 and MultiWOZ 2.1 datasets respectively and achieves new state-of-the-art performance with considerable improvements (+1.24% and +5.98%). 1 1 Introduction Recently, task-oriented dialogue systems have attracted increasing attention in both industry and academia due to their broad application for helping users accomplish tasks through spoken interactions (Young, 2002; Young et al., 2013; Gao et al., 2019a). Dialogue state tracking (DST) is an essential part of dialogue management in task-oriented dialogue systems. Given current utterances and dialogue history, DST aims to determine the set of  Joint work with Pattern Recognition Center, WeChat AI, Tencent Inc. User: Hello, I'm looking for a resraurant with fair prices. State: price range=moderate Sys: OK. There are Golden Wok Chinese restaurant and Nirala which serves Indian food, which one do you like? User: Are they both have a reasonable price ? State: price range=moderate Sys: Of course. User: Please tell me the address of Golden Wok. State: price range=moderate; food=chinese Table 1: An example dialogue. At the last turn, it is necessary to capture relevant information in dialogue history to correctly predict the value of slot  food , which is underlined. User and Sys represent user utterance and system response respectively, and the italic text means dialogue states. As Table 1 shows, the dialogue state is usually dependent on relevant context in the dialogue history, which is proven in previous studies (Sharma et al., 2019; Wu et al., 2019). However, traditional DST models usually determine dialogue states by considering only utterances at current turn (Hen-derson et al., 2014b; Mrksic et al., 2017; Zhong et al., 2018; Chao and Lane, 2019) which neglects the use of dialogue history. Recent researches attempt to address this problem through introducing historical dialogue information into the prediction of slot-value pairs. Most of them leverage a naive attention between slots and concatenated historical utterances (Wu et al., 2019; Zhou and Small, 2019; Gao et al., 2019b; Zhang et al., 2019; Le et al., 2020a,b) or only utilize partial history (Ren et al., 2019; Kim et al., 2019; Sharma et al., 2019) or lack direct interactions between slots and history (Ren et al., 2018; Lee et al., 2019; Goel et al., 2019). Briefly, these methods are deficient in exploiting relevant context from dialogue history. Furthermore, there are differences in the frequency of different slots and different slot-value pairs. For example, in MultiWOZ 2.0 train set, there are 15384 samples related to the slot  train-day  while 5843 for the slot  attraction-name ; the slot-value pair ( attraction-area , center ) occurs 5432 times and ( taxi-departure , royal spice ) occurs only 9 times; etc. We refer to this problem as slot imbalance, which makes the learning difficulties of different slots varies (Refer to Appendix for de-tails). However, existing approaches usually ignore the slot imbalance problem and treat all slots indiscriminately, which limits the learning of those hard slots and eventually damages the overall performance. To address the two aforementioned problems, we propose an effective model equipped with a c ontextual h ierarchical a ttention n etwork (CHAN) to fully exploit relevant context from dialogue history, and an adaptive objective to alleviate the slot imbalance problem. In CHAN, the slot firstly retrieves word-level relevant information from utterances at each turn. Then, these word-level relevant information will be encoded into contextual representations by rich interactions. Finally, the slot aggregates all contextual representations into turn-level relevant information and then we combine it with word-level relevant information to obtain the outputs. To further enhance the ability to exploit relevant context, we employ a state transition prediction task to assist DST learning. For the slot imbalance problem, our adaptive objective can dynamically evaluate the difficulties in an accuracy-sensitive manner and then adaptively adjust the learning weights for different slots. Thus, it can balance the learning of all slots as far as possible. RETRACTED This paper was retracted. We evaluate the effectiveness of our model on MultiWOZ 2.0 and MultiWOZ 2.1 datasets. Experimental results show that our model reaches 52.68% and 58.55% joint accuracy, outperforming previous state-of-the-art by +1.24% and +5.98% , respectively. The ablation study also demonstrates each module's effectiveness in our model. Our contributions are as follows:  We propose an effective contextual hierarchical attention network to fully exploit relevant context from dialogue history and employ a state transition prediction task to further enhance it. We design an adaptive objective to address the slot imbalance problem by dynamically adjusting the weight of each slot. To the best of our knowledge, our method is the first to address the slot imbalance problem in DST. Experimental results show that our model achieves state-of-the-art performance with significant improvements over all previous models. As shown in Figure 1, the proposed model consists of three components: 1) the c ontextual h ierarchical a ttention n etwork (CHAN); 2) the state transition prediction module; 3) the adaptive objective. We share all the model parameters for each slot to keep our model universal for all slots. 2.1 Problem Statement Given a dialogue X = { ( U 1 , R 1 ) , ..., ( UT , RT ) } of T turns where U t represents user utterance and R t represents system response of turn t , we define the dialogue state at each turn t as B t = { ( s, v t ) , s  S} where S is a set of slots and v t is the corresponding value of the slot s . Following Lee et al. (2019), we use the term slot to refer to the concatenation of a domain name and a slot name in order to represent both domain and slot information. For example,  restaurant-food . Similar to (Ren et al., 2018; Lee et al., 2019), we decompose the dialogue state tracking to a multi-label classification problem where we score each value with slot-related features in a non-parametric way and then choose the best candidate. We also add a literally  none  into the value set of each slot to represent that no corresponding value is tracked. Recently the pre-trained BERT language model (Devlin et al., 2019) shows powerful ability in universal contextual semantics representation, thus we employ BERT to encode utterances, slots and values. To better retrieve relevant context from dialogue history, we devise Slot-Word Attention and Slot-Turn Attention to query both relevant keywords and turns. Specifically, we exploit a Context Encoder between word-level and turn-level attention to capture contextual representations of relevant information from dialogue history. Furthermore, we devise a Global-Local Fusion Gate to balance the information from global context and local utterances. Sentence Encoder. BERT leverages a special token [CLS] to aggregate the whole representation of a sentence and a special token [SEP] to indicate the end of a sentence. where BERT finetune means that it will be fine-tuned during training. Therefore, BERT finetune will learn a corresponding generalization of sentence representations and adapt to dialogue state tracking task. For slot s and value v t , we adopt another pre-trained BERT fixed to encode them into contextual semantics vectors h s and h vt respectively. Different from utterances, we use the output vector of the special token [CLS] to obtain the whole sentence representation: h s = BERT fixed ( s ) (2) h vt = BERT fixed ( v t ) where the weights of BERT fixed are fixed during training thus our model can be scalable to any unseen slots and values with sharing the original BERT representation. Slot-Word Attention . The slot-word attention is a multi-head attention (MultiHead( Q , K , V )), which takes a query matrix Q , a key matrix K and a value matrix V as inputs. Refer to (Vaswani et al., 2017) for more details. For each slot s , the slot-word attention summarizes word-level slot-related information from each turn t into a d -dimensional vector c words,t , which can be determined as follows: c words,t = MultiHead( h s , h t , h t ) (3) Context Encoder . The context encoder is a unidirectional transformer encoder, which is devised to model the contextual relevance of the extracted word-level slot-related information among { 1 , ..., t } turns. The context encoder contains a stack of N identical layers. Each layer has two sub-layers. The first sub-layer is a masked multi-head self-attention ( MultiHead ), in which Q = K = V . The second sub-layer is a position-wise fully connected feed-forward network ( FFN ), which consists of two linear transformations with a ReLU activation (Vaswani et al., 2017). where m n is the output of the n -th layer of context encoder and PE(  ) denotes positional encoding function. Note that residual connection and layer normalization are omitted in the formula. Slot-Turn Attention . To retrieve turn-level relevant information from contextual representation, we devise a slot-turn attention which is the multihead attention as follows: c turns,t = MultiHead( h s , c ctxs,  t , c ctxs,  t ) (6) Therefore, the model can access word-level and turn-level relevant information from the historical dialogues. Global-Local Fusion Gate . To balance the information of global context and local utterances, we propose to dynamically control each proportion of contextual information and current turn information so that the model can not only benefit from relevant context but also keep a balance between global and local representations. Similar to Hochreiter and Schmidhuber (1997), we leverage a fusion gate mechanism, which computes a weight to decide how much global and local information should be combined according to c words,t and c turns,t . It can be defined as follows: g s,t =  ( W g (cid:12) [ c words,t ; c turns,t ]) (7) c gates,t = g s,t  c words,t + (1  g s,t )  c turns,t where W g  R 2 d  d are parameters,  means sigmoid activation function, (cid:12) and  mean the pointwise and element-wise multiplication respectively. P TP where V s is the candidate value set of slot s and  v t  V s is the ground-truth value of slot s . To better capture relevant context, we further introduce an auxiliary binary classification task to jointly train with DST: State Transition Prediction (STP), which is to predict if the value for a slot is updated compared to previous turn. This module reads c gates,t  1 and c gates,t as inputs and the transition probability p stps,t can be calculated as follows: c stps,t = tanh( W c (cid:12) c gates,t ) (10) p stps,t =  ( W p (cid:12) [ c stps,t ; c stps,t  1 ]) where W c  R d  d , W p  R 2 d are parameters. Note that when t = 1 , we simply concatenate c stps,t with zero vectors. For this task, we calculate the binary cross entropy loss between ground-truth transition labels y stps,t and the transition probability p stps,t , which is defined as follows: L stp = X s S TX t =1  y stps,t  log( p stps,t ) (11) 2.4 Adaptive Objective Essentially, the slot imbalance problem can be considered as a kind of class imbalance because there is an imbalance among both different slots and different samples. Instead of treating all slots indiscriminately, it is important to balance the learning of different slots. Recently, Lin et al. (2017) propose a soft-sampling method, Focal Loss, to re-weight the losses of different classes. Inspired by their work, we design a novel adaptive objective for DST which evaluates the difficulty from each slot's accuracy on the validation set and adaptively adjusts the weight of each slot during optimization. We define the accuracy of slot s on validation set as acc vals . Our adaptive objective is based on the following intuitions: (1) If acc vals  acc vals 0 ; then slot s is more difficult than slot s 0 . Suppose this slot-level difficulty is defined as  ; then  s = 1  acc vals P s 0 S 1  acc vals 0  |S| (12) (2) Suppose there are two samples { ( U t , R t ) , ( s, v t ) } and { ( U t 0 , R t 0 ) , ( s 0 , v t 0 ) } . If the former confidence is lower than the latter, then sample { ( U t , R t ) , ( s, v t ) } is more difficult than { ( U t 0 , R t 0 ) , ( s 0 , v t 0 ) } . Suppose this sample-level difficulty is defined as  ; then  ( s, v t ) = (1  p ( s, v t ))  (13) where p ( s, v t ) is the confidence of sample { ( U t , R t ) , ( s, v t ) } and  is a hyper-parameter. L adapt ( s, v t ) =   s  ( s, v t ) log( p ( s, v t )) (14) Focal Loss assigns static learning weights on slots and doesn't change them anymore during the whole training. Compared to Focal Loss, our adaptive objective can fit data better by dynamically evaluate the difficulties in an accuracy-sensitive manner and then adaptively control the learning weights for different slots, which is proved in our experiments. If the difficulty of slot s is greater than the average difficulty of all slots,  s would increase and enlarge the loss of s . Similarly, the optimization of sample { ( U t , R t ) , ( s, v t ) } with a low confidence p ( s, v t ) would be encouraged by a larger loss. When an epoch ends, the adaptive objective re-evaluates the difficulty of each slot and updates  s . Therefore, it can not only encourage the optimization of those hard slots and samples but also balance the learning of all slots. In our model, we firstly jointly train the DST and STP tasks to convergence and then fine-tune DST task with the adaptive objective. During joint training, we optimize the sum of these two loss functions as following: L joint = L dst + L stp (15) At the fine-tuning phase, we adopt the adaptive objective to fine-tune DST task as following: L finetune = X s S TX t =1 L adapt ( s,  v t ) (16) 3 Experiments Setup 3.1 Datasets & Metrics Hotel Train Attraction Restaurant Taxi Slots price, type,parking,stay,day,people,area,stars,internet,name destination,departure,day,arriveby,leaveat,people area,name,type food,price,area,name,time,day,people destination,departure,arriveby,leaveby Train 3381 3103 2717 3813 1654 Valid 416 484 401 438 207 Test 394 494 395 437 195 Table 2: The dataset statistics of MultiWOZ 2.0 & 2.1. public task-oriented dialogue datasets, including about 10,000 dialogues with 7 domains and 35 domain-slot pairs. MultiWOZ 2.1 shares the same dialogues with MultiWOZ 2.0 but it fixed previous annotation errors. The statistics are shown in Table 2. Following (Wu et al., 2019), we use only 5 domains { restaurant , hotel , train , attraction , taxi } excluding hospital and police since these two domains never occur in the test set. We preprocess the datasets following (Lee et al., 2019) 2 . We use joint accuracy and slot accuracy as our evaluation metrics. Joint accuracy is the accuracy of the dialogue state of each turn and a dialogue state is evaluated correctly only if all the values of slots are correctly predicted. Slot accuracy only considers individual slot-level accuracy. 3.2 Baseline Models We compare our results with the following competitive baselines: DSTreader proposes to model DST as a machine reading comprehension task and extract spans from dialogue history (Gao et al., 2019b). GLAD-RCFS uses a heuristic rule to extract relevant turns and lets slot-value pairs to query relevant context from them (Sharma et al., 2019). HyST employs a hierarchical encoder and takes a hybrid way combining both predefined-ontology and open-vocabulary settings (Goel et al., 2019). TRADE encodes the whole dialogue context and decodes the value for every slot using a copy-augmented decoder (Wu et al., 2019). DST-QA proposes to model DST as a question answering problem and uses a dynamically-evolving knowledge graph to learn relationships between slot pairs (Zhou and Small, 2019). SOM-DST considers the dialogue state as an explicit fixed-size memory and proposes a selectively overwriting mechanism (Kim et al., 2019). SUMBT exploits BERT as the encoder of the utterances, slots and values. It scores every candidate slot-value pair in a non-parametric manner using a distance measurement (Lee et al., 2019). candidate values and slot-context encoding considering all slots as picklist-based slots (Zhang et al., 2019). GLAD-RCFS, HyST, SUMBT, DST-picklist are predefined-ontology models as well as our model and DSTreader, TRADE, DST-QA, SOM-DST are open-vocabulary models. We employ the pre-trained BERT model that has 12 layers of 784 hidden units and 12 self-attention heads 3 . For the multi-head attention, we set heads count and hidden size to 4 and 784, respectively. For the context encoder, we set the transformer layers to 6. We set the max sequence length of all inputs to 64 and the batch size to 32. In all training, we use Adam optimizer (Kingma and Ba, 2015) and set the warmup proportion to 0.1. Specifically, in the joint training phase, we set the peak learning rate to 1e-4. At the fine-tuning phase, we set  to 2, peak learning rate to 1e-5. The training stopped early when the validation loss was not improved for 15 consecutive epochs. For all experiments, we report the mean joint accuracy over multiple different random seeds to reduce statistical errors. To prove the effectiveness of each module of the proposed CHAN, we conduct ablation experiments RETRACTED This paper was retracted. 4 Experiment Results 4.1 Main Results Table 3 shows the joint accuracy of our model and other baselines on the test sets of MultiWOZ 2.0 and 2.1. Our model beats all baselines whether they are based on predefined ontology or open vocabulary, and achieves 52.68% and 58.55% joint accuracy with considerable improvements (1.24% and 5.98%) over previous best results on MultiWOZ 2.0 and 2.1, respectively. Also, our model achieves 97.69% and 98.14% slot accuracy with 0.36% and 0.58% improvements over the previous best results on MultiWOZ 2.0 and 2.1, respectively. Similar to (Kim et al., 2019), we find that our model achieves much higher improvements on MultiWOZ 2.1 than 3 It is published as bert-base-uncased model in https://github.com/huggingface/pytorch-transformers Model MultiWOZ 2.1 Our Model 58.55 state transition prediction 57.86 (-0.69) adaptive objective fine-tuning 57.45 (-1.10) above two (only CHAN)  57.00 (-1.55) Our Model (FL (  =1,  =2))  58.10 (-0.45) Table 4: The ablation study of the state transition prediction and the adaptive objective on the MultiWOZ 2.1 test set with joint accuracy (%). means fine-tuning with focal loss instead. that on MultiWOZ 2.0. This is probably because MultiWOZ 2.1 fixes lots of notation errors in MultiWOZ 2.0 and our model can benefit more from more accurate relevant context. As shown in Table 4, we estimate the effectiveness of the proposed state transition prediction and adaptive objective on the MultiWOZ 2.1 test set. The results show that both state transition prediction task and adaptive objective can boost the performance. Removing the state transition prediction task reduces joint accuracy by 0.69%, and the joint accuracy decreases by 1.10% without the adaptive objective fine-tuning. Moreover, when we remove the state transition prediction task and don't fine-tune our model with adaptive objective (only CHAN remains), the joint accuracy decreases by 1.55%. Also, to explore the importance of adjusting the  s adaptively, we replace the adaptive objective with original focal loss (  = 1 ,  = 2 ), which leads to 0.45% drop. on the MultiWOZ 2.1 test set as shown in Table 5. We observe that a slight joint accuracy drop of 0.24% after removing the global-local fusion gate, which proves the effectiveness of fusing global context and local utterances. Moreover, removing the slot-turn attention and context encoder leads to a decrease by 0.15% and 1.72% respectively, which demonstrates that the turn-level relevant information and the contextual representations of word-level relevant information are effective to improve the performance. Moreover, after we remove the aforementioned three modules and sum the word-level relevant information of { 1 ,    , t } turns as output, the joint accuracy reduces by 6.72%, which is much higher than the sum of above three reductions. It demonstrates that effectively modeling interactions with word-level relevant information of dialogue history is crucial for DST. Figure 2 shows the visualization of turn-level and word-level attention of the  restaurant-name  slot on a prediction example of our model at turn 5. while almost pays no attention to turns { 1,2 } . And from the word-level attention visualization, we can easily find that the  restaurant-name  slot attends to the  dojo noodle bar  with the highest weight in both turn 3 and turn 4. Although there is no slot-related information at turn 5, our model still makes the correct decision by exploiting relevant context from the historical dialogue. 4.4 Effects of Adaptive Obj. on Acc. per Slot As Figure 3 shows, we draw the accuracy changes of each slot on MultiWOZ 2.1 test set after fine-tuning our model with adaptive objective. quency (The detailed accuracy results are in the Appendix). Thus, slots on the left side are relatively more difficult than slots on the right side. After fine-tuning with the adaptive objective, most slots on the left side achieve significant improvements, which proves the adaptive objective can encourage the learning of the hard slots. Although adaptive objective tends to decrease the weight of slots on the right side, they also benefit from the fine-tuning. We think that this is because encouraging the optimizing of hard slots enhances our model by tracking more complicated dialogue states. It proves that our adaptive objective can not only improve the performance of relatively hard slots but also boost the performance of relatively easy slots. 4.5 Qualitative Analysis To explore the advantages of our model compared to baseline models, we conduct a human evaluation on a subset of the MultiWOZ 2.1 test set where our model makes correct predictions while SUMBT (a previous strong baseline) fails. We predefine three types of improvements: historical information inference improvement which means inferring historical information is necessary for correct decisions, current information inference improvement which means inferring current information is enough for correct decisions, and other improvements. As shown in Table 6, 64.49% improvements come from historical information inference, which demonstrates that our model can better exploit relevant context from the dialogue history. The general backbone of our model is a hierarchical attention network that can effectively aggregate query-related information at multiple levels (Yang RETRACTED This paper was retracted. For more information, see https://aclanthology.org/2020.acl-main.563 . 6330 et al., 2016; Ying et al., 2018; Wang et al., 2018; Xing et al., 2018; Aujogue and Aussem, 2019; Naik et al., 2018; Liu and Chen, 2019). Traditional statistical dialogue state tracking models combine semantics extracted by spoken lan-Improvement guage understanding modules to predict the current dialogue state (Williams and Young, 2007; Thomson and Young, 2010; Wang and Lemon, 2013; Williams, 2014) or to jointly learn speech understanding (Henderson et al., 2014b; Zilka and Jur-cicek, 2015; Wen et al., 2017). One drawback is that they rely on hand-crafted features and complex domain-specific lexicons besides the ontology, and they are hard to extend and scale to new domains. Recent neural network models are proposed for further improvements (Mrksic et al., 2015; Hori et al., 2016; Mrksic et al., 2017; Lei et al., 2018; Xu and Hu, 2018; Zhong et al., 2018; Nouri and Hosseini-Asl, 2018; Wu et al., 2019; Ren et al., 2019; Balaraman and Magnini, 2019). Ren et al. (2018) and Lee et al. (2019) use an RNN to encode the slot-related information of each turn, where slots can not attend to relevant information of past turns directly. Sharma et al. (2019) employ a heuristic rule to extract partial dialogue history and then integrate the historical information into prediction in a coarse manner. Goel et al. (2019) encode the dialogue history into a hidden state and then simply combine it with the slot to make decisions. These models are deficient in fully exploiting the relevant context in dialogue history. Gao et al. (2019b) introduce a slot carryover model to decide whether the values from the previous turn should be used or not and Kim et al. (2019) introduce a state operation predictor to decide the operation with the previous state. Different from them, we consider the state transition prediction as an additional enhancement while they integrate it into their DST pipelines. Besides, Zhong et al. (2018) only employ local modules to model the slot-specific representations, which neglects the slot imbalance problem. We introduce an effective model that consists of a contextual hierarchical attention network to fully exploit relevant context from dialogue history and an adaptive objective to alleviate the slot imbalance problem in dialogue state tracking. Experimental results show that our model achieves state-of-the-art performance of 52.68% and 58.55% joint accuracy with considerable improvements (+1.24% and +5.98%) over previous best results on MultiWOZ 2.0 and MultiWOZ2.1 datasets, respectively. Although our model is based on predefined ontology, it is universal and scalable to unseen domains, slots and values. The main contributions of our model, CHAN and adaptive objective, can also be applied to open-vocabulary models. We will explore it in the future. We thank the anonymous reviewers for their insightful comments. This work was supported by National Key R&D Program of China (NO. 2017YFE0192900). RETRACTED This paper was retracted.
abstain abstain abstain objective objective abstain objective abstain abstain abstain objective objective objective objective objective objective objective objective objective abstain abstain abstain other abstain abstain abstain abstain abstain objective abstain abstain result method objective abstain method method result objective objective objective objective result abstain abstain abstain abstain method method method abstain method method method method method method abstain abstain abstain abstain method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain method method other abstain method abstain other method method abstain abstain method method method abstain abstain abstain abstain method abstain abstain abstain abstain method abstain method abstain abstain abstain method method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method abstain abstain abstain method abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain other other other other other other other other other other method other objective result abstain objective objective other other other