Recent efforts in Dialogue State Tracking (DST) for task-oriented dialogues have progressed toward open-vocabulary or generation-based approaches where the models can generate slot value candidates from the dialogue history itself. These approaches have shown good performance gain, especially in complicated dialogue domains with dynamic slot values. However, they fall short in two aspects: (1) they do not allow models to explicitly learn signals across domains and slots to detect potential dependencies among (domain, slot) pairs; and (2) existing models follow auto-regressive approaches which incur high time cost when the dialogue evolves over multiple domains and multiple turns. In this paper, we propose a novel framework of Non-Autoregressive Dialog State Tracking (NADST) which can factor in potential dependencies among domains and slots to optimize the models towards better prediction of dialogue states as a complete set rather than separate slots. In particular, the non-autoregressive nature of our method not only enables decoding in parallel to significantly reduce the latency of DST for realtime dialogue response generation, but also detect dependencies among slots at token level in addition to slot and domain level. Our empirical results show that our model achieves the state-of-the-art joint accuracy across all domains on the MultiWOZ 2.1 corpus, and the latency of our model is an order of magnitude lower than the previous state of the art as the dialogue history extends over time.
The authors build on recent work for non-autoregressive encoder-decoder models in the context of machine translation (most significantly [Gu, et al., ICLR18]) and adapt this to dialogue state tracking. Specifically, as in [Gu, et al, ICLR18], they use a fertility decoder modified for DST to be on a per-slot basis which is input to a second decoder to generate the (open-vocabulary) tokens representing dialogue state. An interesting aspect of the resulting model as formulated is that the latent space also takes into account interdependencies between generated slot values, which leads to a direct structured prediction-like loss of joint accuracy. Additionally, as slot values have a smaller (and likely peakier) combinatorial space, NAT models actually are more applicable to DST than MT. The resulting model achieves state-of-the-art empirical results on the MultiWOZ dataset while incurring decoding times that are an order of magnitude faster. 
Once one understands this conceptual modification of modifying the NAT string encoder-decoder to a more structured NAT encoder-decoder (which in DST is more of a normalized string), they apply all of the state-of-the-art techniques to build a DST system: gating of all potential (domain, slot) pairs as an auxiliary prediction (e.g., [Wu et al., ACL19]), de-lexicalizing defined value types [Lei, et al., ACL18], content encoder, and domain-slot encoder (with pretty standard hyper-parameters, etc.). The significant addition is the fertility decoder and the associated NAT state decoder. Thus, from a conceptual level, this isn’t a huge leap and something many researchers *could* have done (i.e., I think many people, including myself, to have expected this paper to come out) — thus, it is more of a ‘done first’ paper than a ‘done unexpectedly’ paper. However, it is done well and the results are convincing and interesting. Given the impressive performance, I expect others to continue building on this work and potentially even influencing people to combine encoder-decoder models with a more structured prediction approach to DST.  Thus, I would prefer to see it accepted if possible.
That being said, I do have a few questions regarding this work — but these are more questions that might be able to be addressed than actual criticisms per se. First, in Table 5, why without the delexicalized dialogue history does the performance drop from 49.04% to 39.45%? This does not make sense to me as the model is much more complex than TRADE; however, TRADE does not do delexicalization yet achieves 45.6% joint accuracy. Meanwhile, with such complex model, I would expect the model can learn from raw data without delexicalization. Moreover, the proposed method use both previous predictions and *previous system actions* to do delexicalization. Also, the NATMT models don’t do delexicalization (although they have significantly more data). I think the authors should do an ablation study that do not use previous system actions, because this is extra information compared with TRADE — even if delexicalizing. Secondly, another worthy baseline would be an autoregressive decoder (with other blocks of the model remain the same). I’d assume that the performance is slight higher. It is interesting to see the gap, because this gap is the cost to speed up decoder using fertility — even if it is a bit counter-intuitive. If there is no improvement in this setting, then structured prediction in general may make more sense. Honestly, I think more would be interested in the second point than the first.
In any case, nice paper — well-written, well-motivated, interesting empirical results. The only reason I am recommending ‘weak accept’ is that the novelty is fairly straightforward and the strength is in the execution.