Detecting out-of-domain (OOD) intents is crucial for the deployed task-oriented dialogue system. Previous unsupervised OOD detection methods only extract discriminative features of different in-domain intents while supervised counterparts can directly distinguish OOD and in-domain intents but require extensive labeled OOD data. To combine the benefits of both types, we propose a self-supervised contrastive learning framework to model discriminative semantic features of both in-domain intents and OOD intents from unlabeled data. Besides, we introduce an adversarial augmentation neural module to improve the efficiency and robustness of contrastive learning. Experiments on two public benchmark datasets show that our method can consistently outperform the baselines with a statistically significant margin. 1 1 Introduction Task-oriented dialog systems (Sarikaya, 2017; Akasaki and Kaji, 2017; Gnewuch et al., 2017; Shum et al., 2018; Tulshan and Dhage, 2018) such as Google's DialogFlow or Amazon's Lex have become ubiquitous to make people interact with machines using natural language. In the architecture of a dialogue system, detecting unknown or OOD (Out-of-Domain) intents from user queries is an essential component that aims to know when a user query falls outside their range of predefined supported intents. Different from traditional intent detection tasks, we do not know the exact number of unknown intents in practical scenarios and can barely annotate extensive OOD samples. Lack of real OOD examples always leads to poor prior knowledge about these unknown intents, making it  Weiran Xu is the corresponding author. Previous methods of detecting OOD intents can be generally classified into two types: unsupervised and supervised OOD detection. Unsupervised OOD detection (Breunig et al., 2000; Bendale and Boult, 2016; Hendrycks and Gimpel, 2017; Shu et al., 2017; Lee et al., 2018; Ren et al., 2019a; Lin and Xu, 2019; Snell et al., 2017; Finn et al., 2017; Xu et al., 2020) means no labeled OOD samples except for labeled in-domain data. By contrast, supervised OOD detection (Scheirer et al., 2013; Fei and Liu, 2016; Kim and Kim, 2018; Larson et al., 2019; He et al., 2020b; Zheng et al., 2020) represents that there are extensive labeled OOD samples in the training data. Most of unsupervised OOD detection methods follow a two-stage framework: training and detecting. They first train an in-domain intent classifier to extract intent representations, then detect whether the test query belongs to OOD by estimating its probability density. For example, Hendrycks and Gimpel (2017); Shu et al. (2017) simply use a threshold on the in-domain classifier's probability estimate. Lin and Xu (2019) employs an unsupervised density-based novelty detection algorithm, local outlier factor (LOF) to detect unseen intents. However, such neural models can only extract discriminative features of different in-domain intents since they are trained on the in-domain data without access to OOD data. Therefore, these methods are known to produce highly overconfident posterior distributions even for such abnormal OOD samples (Guo et al., 2017; Liang et al., 2017, 2018). For supervised OOD detection, classical methods such as (Fei and Liu, 2016; Larson et al., 2019), form a ( N + 1) -class classification problem where the ( N + 1) -th class represents the unseen intents. Further, Zheng et al. (2020) uses labeled OOD data to generate an entropy regularization term to enforce the predicted distribution of OOD inputs closer Embedding User Utterance BiLSTM Encoder Loss Function Training LOF/GDA OOD Detection Unlabeled Text f  Back Translate Augmented f  f  f  Contrastive Loss Adversarial Contrastive Loss FGV Attack Text f  Cross-EntropyLoss to the uniform distribution. However, collecting large-scale labeled OOD data is usually difficult and expensive. These drawbacks limit the broad application of supervised OOD detection. In this paper, we aim to capitalize on the benefits of both self-supervised and supervised OOD detection: (1) simultaneously modeling semantic features of both in-domain and OOD data; (2) inducing no labor-intensive OOD annotation. In this paper, we propose a self-supervised contrastive learning framework to model discriminative semantic features of both in-domain intents and OOD intents from unlabeled data. Without access to labeled OOD data, our method aims to learn representations that discriminate between all unlabeled intents in the instance level. When combined with supervised in-domain training, our method learns features that are both rich and semantically discriminative. Besides, to replace the stochastic data augmentation mechanisms like random cropping, random color distortions in the image processing field (Chen et al., 2020a), we propose an adversarial augmentation neural module to improve the diversity and complexity of pre-defined transformation functions. Specifically, we compute model-agnostic adversarial worst-case perturbations to the inputs in the direction that significantly increases the original contrastive loss. Intuitively, adversarial learning can generate pseudo hard positive pairs thus improve the efficiency and robustness of contrastive learning. Our contributions are three-fold: (1) We propose a self-supervised learning framework to simultaneously modeling semantic features of both in-domain and OOD data. (2) We apply an adversarial augmentation mechanism to improve the efficiency and robustness of self-supervised learning. (3) Experiments conducted on two benchmark OOD datasets show the effectiveness of our proposed method. Overall Architecture Fig 1(a) shows the overall architecture of our proposed two-stage framework. We first train an in-domain intent classifier to extract intent representations using two objectives then use the detection algorithms MSP (Hendrycks and Gimpel, 2017), LOF (Lin and Xu, 2019) or GDA (Xu et al., 2020) to detect OOD. In the training stage, we first train a BiLSTM in-domain intent classifier similar to Lin and Xu (2019) using labeled in-domain data. Then we apply an adversarial contrastive objective to continue training on the unlabeled data. Self-Supervised Contrastive Learning To simultaneously model semantic features of both in-domain and OOD data, we propose a self-supervised contrastive learning framework to utilize unlabeled data. Following (Chen et al., 2020a; He et al., 2020a; Chen et al., 2020b; Winkens et al., 2020; Jiang et al., 2020), we formulate the contrastive loss for a positive pair of examples ( i, j ) as: (cid:96) i,j =  log exp (sim ( z i , z j ) / ) (cid:80) 2 Nk =1 1 [ k (cid:54) = i ] exp (sim ( z i , z k ) / ) (1) where z i represents the feature vector of i -th sentence sample extracted by concatenating the first and final hidden states of BiLSTM, and 1 [ k (cid:54) = i ]  { 0 , 1 } is an indicator function evaluating to 1 if k (cid:54) = i . denotes a temperature parameter. The final loss is computed across all positive pairs, both ( i, j ) and ( j, i ) in a mini-batch of N examples. Here we use back-translation as data augmentation to generate positive pairs. Previous work (Chen et al., 2020a) has shown the necessity of more data augmentations, thus we propose an adversarial neural augmentation as follows. Adversarial Neural Augmentation To improve the diversity of data augmentation and avoid handcrafted engineering, we apply adversarial attack (Goodfellow et al., 2015; Kurakin et al., 2016; Miyato et al., 2016; Jia and Liang, 2017; Zhang et al., 2019; Ren et al., 2019b) to generate pseudo positive samples. It should be noted that samples obtained by adversarial attack is in the form of embedding to ensure end-to-end training. Specifically, we need to compute the worst-case perturbation  that maximizes the original contrastive loss L :  = arg max (cid:107)  (cid:48) (cid:107) (cid:15) L (cid:0)  , x +  (cid:48) (cid:1) , where  represents the parameters of a model and x denotes a given sample. (cid:15) is the norm bound of the perturbation  . In practical implementation, we apply Fast Gradient Value (FGV) (Rozsa et al., 2016) to approximate the perturbation  :  = (cid:15) g || g || ; where g =  ( x i , x j ) L ( f ( x i , x j ;  )) (2) where ( x i , x j ) represents the original positive pair generated by back-translation. We perform normalization to g and then use a small (cid:15) to ensure the approximate is reasonable. Finally, we can obtain the pseudo adversarial sample x advi = x i +  as well as x advj . Therefore, we get ( x i , x j , x advi , x advj ) from the original positive pair ( x i , x j ) . We implement four different contrastive settings: (1) Standard-to-Standard (S2S): the original contrastive loss using ( x i , x j ) ; (2) Adversarial-to-Adversarial (A2A): the adversarial contrastive loss using ( x advi , x advj ) ; (3) Standard-to-Adversarial (S2A): the mixed contrastive loss using ( x i , x advi ) or ( x j , x advj ) ; (4) Dual Stream (DS): combining S2S and A2A as Fig 1(c) shows. Experiment 3.4 shows that the last setting works best. We argue that DS capture better feature alignment in the latent space. 2 Besides, we find only applying the contrastive loss leads to the worse in-domain intent detection metrics, therefore 2 We leave the comprehensive theoretical analysis to future work. we mix up the two kinds of objectives during training to avoid catastrophic forgetting (Kirkpatrick et al., 2017). We present an Algorithm section in the appendix. Datasets We perform experiments on two variants of the OOD benchmark dataset CLINC 3 (Larson et al., 2019), namely CLINC-OOS+ and CLINC-Small. Table 1 shows the detailed statistics of two datasets. They both contain 150 in-domain intents across 10 domains where CLINC-OOS+ contains 100 samples for each intent and CLINC-Small has 50 training samples for each intent. Besides, CLINC-OOS+ has 250 OOD examples in training set, while CLINC-Small contains 100. To construct the unlabeled data, we mix up 10% of in-domain data and all of the OOD data in the training set. The total amount of unlabeled data is equal to 1500 in CLINC-OOS+ and 750 in CLINC-Small, where the number of OOD data is 250 and 100, respectively. Note that during the self-supervised learning phase, we don't utilize label information of the unlabeled data and only perform contrastive learning at the instance-level. During the supervised learning phase, we use the other in-domain training data for cross-entropy loss. Metrics We report both in-domain metrics: Ac-curacy(ACC) and F1-score(F1), and OOD metrics: Recall and F1-score(F1). OOD Recall and F1-score are the main metrics in this paper. We compare our proposed self-supervised methods to two types of OOD detection methods, which are supervised and fully unsupervised. The former applies a supervised OOD entropy regularization. We use this setting as the reference upper bound for OOD detection results. The latter represents that we train the sentence feature extractor using only in-domain data. We treat this setting as the 3 https://github.com/clinc/oos-eval CLINC-OOS+ CLINC-Small Model in-domain OOD in-domain OOD ACC F1 Recall F1 ACC F1 Recall F1 N+1 88.6 91.46 19.12 32.00 85.23 88.58 17.46 29.99 Supervised Entropy+MSP(oracle) 87.38 85.71 44.82 57.48 84.52 84.07 27.23 36.81 OOD Entropy+LOF(oracle) 84.08 85.12 60.44 61.89 82.16 82.83 60.72 61.39 Entropy+GDA(oracle) 86.53 87.57 70.20 71.22 84.56 84.68 66.98 67.07 MSP 83.61 84.05 24.28 36.57 81.84 82.20 19.12 29.79 MSP+S2S(w/o adv) 84.11 84.93 37.36 45.52 83.98 83.65 22.40 33.06 MSP+DS(ours) 84.85 84.91 41.76 * 47.62 * 83.93 83.21 25.62 * 34.82 * Self-Supervised LOF 84.20 85.08 57.40 58.78 82.22 82.73 57.20 58.10 OOD LOF+S2S(w/o adv) 85.62 85.99 59.12 59.41 82.84 83.67 57.92 59.04 LOF+DS(ours) 85.87 86.06 59.96 * 61.20 * 82.89 83.85 59.68 * 60.77 * GDA 86.34 87.73 63.70 65.23 84.24 84.30 60.40 61.07 GDA+S2S(w/o adv) 88.56 88.10 64.92 67.22 85.76 86.20 62.80 64.20 GDA+DS(ours) 88.71 88.98 67.24 * 69.17 * 85.78 86.69 64.52 * 65.55 * Table 2: Performance comparison between our method and baselines on CLINIC-OSS+ and CLINIC-Small datasets. reference lower bound. For each training method, we use different OOD detection models to verify its performance. Therefore, the model proposed in this paper can be divided into two stages. Firstly, the feature extractor training is completed in the training stage, and then the OOD detection is conducted by using different models in detection stage. Training Stage On the basis of fully unsupervised setting, our proposed four types of adversarial self-supervised learning settings are added, respectively. Standard-to-Standard (S2S): Original setting. The contrastive loss is computed between origin and augmented data. The adversarial attack is not involved. Adversarial-to-Adversarial (A2A): The setting injecting two adversarial attacks to origin data and augmented data first, then compute contrastive loss between them. Standard-to-Adversarial (S2A): This setting divide contrastive loss into two parts. One uses origin data with adversarial attack and augmented data, the other uses augmented data with adversarial attack and origin data. Dual Stream (DS): The setting combining S2S and A2A. The contrastive loss contains two parts. One uses origin data and augmented data, the other uses corresponding data with adversarial attacks. Detection Stage As mentioned above, we compare three OOD detection models: MSP (Maxi-mum Softmax Probability)(Hendrycks and Gimpel, 2017) applies a threshold on the maximum softmax probability where the threshold is set as 0.5. LOF (Local Outlier Factor)(Lin and Xu, 2019) uses the local outlier factor to detect unknown intents. GDA (Gaussian Discriminant Analysis)(Xu et al., 2020) is a generative distance-based classifier for out-of-domain detection with Euclidean and Mahalanobis distances. In this paper, the experiments and analysis are mainly conducted around the training stage. Different detection models are used to verify the generalization of our proposed method. Table 2 displays the experiment results. Our method consistently outperforms all the unsupervised baselines in all settings, even close to the supervised oracles. Under the GDA setting, our proposed method outperforms the unsupervised method by 3.94%(OOD F1), 3.54%(OOD Recall) in CLINC-OOS+ and 4.48%(OOD F1), 4.12%(OOD Recall) in CLINC-Small. We also observe similar improvements on the MSP and LOF settings. The results confirm the effectiveness of our self-supervised learning method. Considering the effect of adversarial augmentation, our GDA+DS outperforms the standard contrastive learning (GDA+S2S(w/o adv)) by 1.95%(OOD F1), 2.32%(OOD Recall) in CLINC-OOS+ and 1.35%(OOD F1), 1.72%(OOD Recall) in CLINC-Small. The results demonstrate that adversarial attack can improve the efficiency and robustness of contrastive learning. For in-domain ACC and F1, our method also achieves slightly better performance, even close to N+1 which suffers from a severe drop in OOD metrics for unbalanced data. Effect of Unlabeled Data Size. Fig 2 shows the effect of different sizes of unlabeled data for contrastive learning. We extract each subsets of the total CLINC-OOS+ unlabeled dataset through random sampling, so that the expectation of OOD 0 300 600 900 1200 1500 Number of sampled unlabeled data 60 62 64 66 68 70 OODF 1 s c o r e ( m a c r o ) Upper Bound of GDA Lower Bound of GDA Upper Bound of LOF Lower Bound of LOFGDA LOF Figure 2: Relation between unlabeled data size and OOD detection F1-score. proportion in every subset is close to the full set (16.67%). We choose LOF and GDA for comparison. The lower bound and upper bound respectively represent unsupervised and supervised OOD. Our method achieves superior performance along with the increase of unlabeled data under two settings. It confirms that our proposed method can learn rich and semantically discriminative features via unlabeled data to facilitate OOD detection. Fig 3 shows the relative increment of the F1-score during the uniform increase of unlabeled data. Specifically, the difference between the current F1-score and the previous state F1-score is recorded for every 300 samples added. As the amount of data increases uniformly, the extent of increment of OOD F1-score decrease. It confirms that our proposed method can optimize the performance of OOD detection by taking full advantage of unlabeled data and achieve impressive performance with only a small amount of data. Generally, our proposed methods have strong robustness and generalization capability. Analysis of Norm of Adversarial Perturbation. Fig 4 displays the effect of norm (cid:15) of adversarial noise. (cid:15) controls the range of adversarial perturbation  . In both LOF and GDA, (cid:15)  (1 . 0 , 1 . 5) achieves better performances. A smaller or larger value both impair the capability of contrastive learning. We argue that small noise can not improve the complexity of augmentation and large noise may hurt the alignment of positive example pairs. In this paper, we focus on combining the benefits of both unsupervised and supervised OOD detection: simultaneously modeling semantic features of both in-domain and OOD data without requiring labor-intensive OOD annotation. We propose a self-supervised contrastive learning framework to learn rich and semantically discriminative representations from unlabeled data. Besides, we propose an adaptive end-to-end adversarial augmentation neural module to improve the diversity and complexity of pre-defined transformation functions. Experiments show that our method achieves better performance than unsupervised OOD baselines, even close to supervised OOD oracles. Task-oriented dialog systems have demonstrated remarkable performance across a wide range of applications, with the promise of a significant positive impact on human production mode and lifeway. However, in scenarios where information is complex and rapidly changing, models usually face input that is meaningfully different from typical examples encountered during training. Current models are prone to make unfounded but overconfident predictions on these inputs, which may affect human judgment and thus impair the safety of models in practical applications. In domains with the greatest potential for societal impacts, such as navigation or medical diagnosis, models should be able to detect potentially agnostic OOD and be robust to high-entropy inputs to avoid catastrophic errors. This work proposes a new adversarial self-supervised learning method for OOD detection. The overall robustness of the model is significantly improved by making full use of unlabeled data with potential threats through contrastive learning and adversarial attacks, which takes a step towards the ultimate goal of enabling the safe real-world deployment of task-oriented dialog systems in safety-critical domains. The experimental results have been reported on standard benchmark datasets for considerations of reproducible research. This work was partially supported by National Key RD Program of China No. 2019YFF0303300 and Subject II No. 2019YFF0303302, DOCOMO Beijing Communications Laboratories Co., Ltd, MoE-CMCC "Artifical Intelligence" Project No. MCM20190701.
abstain abstain objective result result abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective objective objective method abstain method abstain objective result objective abstain method method method method method method abstain abstain method method method abstain method abstain method method method method method abstain abstain method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain other abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result result result abstain result abstain abstain other abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result method objective objective result abstain abstain abstain abstain abstain abstain abstain other other