summaries: The paper proposes a technique for assessing the uncertainty of a Transformer-based NMT model on a given input $x$. The technique relies on computing a variance-like estimate over a collection of translation candidates for $x$, where these candidates are obtained by perturbing the decoding mechanism through the use of dropout at test time. Experiments compare this technique with other ways of measuring the "epistemic uncertainty" of the NMT model. In limited training data conditions, the proposed measure is better aligned with the actual performance of the model than competing measures, and in particular is better able to detect Out-of-Domain translation requests. The paper proposed a Baysian method for detecting out of distribution (OOD) in machine translation. To this end, the paper introduces BLEU variance (BLEUVar) that is computed based on a number of samples from Transformer with MC Dropout. The advantage of BLEUVar is that it doesn’t require reference, instead it’s computed based on pairwise comparison of the decoded sentences. This paper describes a method for estimating a neural machine translation (NMT) system's uncertainty about its translation of a sentence that has two parts: (1) use MC Dropout as a proxy for integrating out parameters; (2) two uncertainty metrics (probability of translation summing over randomly-sampled parameters and variance in BLEU using randomly-sampled parameters). The baseline method is just to use the probability of the 1-best translation under the MLE parameters. The method is evaluated by measuring the BLEU score of a test set retaining only the most-certain fraction of the sentences.
abstractText: We detect out-of-training-distribution sentences in Neural Machine Translation using the Bayesian Deep Learning equivalent of Transformer models. For this we develop a new measure of uncertainty designed specifically for long sequences of discrete random variables—i.e. words in the output sentence. Our new measure of uncertainty solves a major intractability in the naive application of existing approaches on long sentences. We use our new measure on a Transformer model trained with dropout approximate inference. On the task of German-English translation using WMT13 and Europarl, we show that with dropout uncertainty our measure is able to identify when Dutch source sentences, sentences which use the same word types as German, are given to the model instead of German.
references.authors: Dzmitry Bahdanau Kyunghyun Cho Yoshua Bengio
references.title: Neural machine translation by jointly learning to align and translate
references.venue: In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations,
references.authors: John Blatz Erin Fitzgerald George Foster Simona Gandrabur Cyril Goutte Alex Kulesza Alberto Sanchis Nicola Ueffing
references.title: Confidence estimation for machine translation
references.venue: In Coling 2004: Proceedings of the 20th International Conference on Computational Linguistics,
references.authors: David M Blei Alp Kucukelbir Jon D McAuliffe
references.title: Variational inference: A review for statisticians
references.venue: Journal of the American Statistical Association,
references.authors: Peter F Brown Vincent J Della Pietra Stephen A Della Pietra Robert L Mercer
references.title: The mathematics of statistical machine translation: Parameter estimation
references.venue: Computational linguistics,
references.authors: Kurtland Chua Roberto Calandra Rowan McAllister Sergey Levine
references.title: Deep reinforcement learning in a handful of trials using probabilistic dynamics models
references.venue: In Advances in Neural Information Processing Systems,
references.authors: Shrey Desai Greg Durrett
references.title: Calibration of pre-trained transformers
references.authors: Angelos Filos Sebastian Farquhar Aidan N. Gomez Tim G.J. Rudner Zachary Kenton Lewis Smith Milad Alizadeh Arnoud de Kroon Yarin Gal
references.title: Benchmarking bayesian deep learning with diabetic retinopathy diagnosis
references.authors: Marina Fomicheva Shuo Sun Lisa Yankovskaya Frédéric Blain Francisco Guzmán Mark Fishel Nikolaos Aletras Vishrav Chaudhary Lucia Specia
references.title: Unsupervised quality estimation for neural machine translation
references.venue: Trans. Assoc. Comput. Linguistics,
references.authors: Yarin Gal
references.title: Uncertainty in deep learning
references.venue: PhD thesis, PhD thesis, University of Cambridge,
references.authors: Yarin Gal Phil Blunsom
references.title: A systematic Bayesian treatment of the IBM alignment models
references.venue: In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,
references.authors: Yarin Gal Rowan McAllister Carl Edward Rasmussen
references.title: Improving pilco with bayesian neural network dynamics models
references.venue: In Data-Efficient Machine Learning workshop, ICML,
references.authors: Yarin Gal Riashat Islam Zoubin Ghahramani
references.title: Deep bayesian active learning with image data
references.venue: In Proceedings of the 34th International Conference on Machine Learning-Volume
references.authors: Zoubin Ghahramani
references.title: Probabilistic machine learning and artificial intelligence
references.venue: Nature, 521(7553):
references.authors: Chuan Guo Geoff Pleiss Yu Sun Kilian Q Weinberger
references.title: On calibration of modern neural networks
references.venue: In Proceedings of the 34th International Conference on Machine Learning-Volume
references.authors: Yashar D Hezaveh Laurence Perreault Levasseur Philip J Marshall
references.title: Fast automated analysis of strong gravitational lenses with convolutional neural networks
references.authors: Marc Hon Dennis Stello Joel C Zinn
references.title: Detecting solar-like oscillations in red giants with deep learning
references.venue: The Astrophysical Journal,
references.authors: Michael I Jordan Zoubin Ghahramani Tommi S Jaakkola Lawrence K Saul
references.title: An introduction to variational methods for graphical models
references.venue: Machine learning,
references.authors: Nal Kalchbrenner Phil Blunsom
references.title: Recurrent continuous translation models
references.venue: In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing,
references.authors: Alex Kendall Yarin Gal
references.title: What uncertainties do we need in bayesian deep learning for computer vision
references.venue: In Advances in Neural Information Processing Systems,
references.authors: Hyun Kim Jong-Hyeok Lee Seung-Hoon Na
references.title: Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation
references.venue: In Proceedings of the Second Conference on Machine Translation,
references.authors: Andreas Kirsch Joost van Amersfoort Yarin Gal
references.title: Batchbald: Efficient and diverse batch acquisition for deep bayesian active learning
references.venue: In Advances in Neural Information Processing Systems,
references.authors: Julia Kreutzer Shigehiko Schamoni Stefan Riezler
references.title: Quality estimation from scratch (quetch): Deep learning for word-level translation quality estimation
references.venue: In Proceedings of the Tenth Workshop on Statistical Machine Translation,
references.authors: Solomon Kullback Richard A Leibler
references.title: On information and sufficiency
references.venue: The annals of mathematical statistics,
references.authors: Aviral Kumar Sunita Sarawagi
references.title: Calibration of encoder decoder models for neural machine translation
references.authors: Christian Leibig Vaneeda Allken Murat Seçkin Ayhan Philipp Berens Siegfried Wahl
references.title: Leveraging uncertainty information from deep neural networks for disease detection
references.venue: Scientific reports,
references.authors: Geert Litjens Thijs Kooi Babak Ehteshami Bejnordi Arnaud Arindra Adiyoso Setio Francesco Ciompi Mohsen Ghafoorian Jeroen Awm Van Der Laak Bram Van Ginneken Clara I Sánchez
references.title: A survey on deep learning in medical image analysis
references.venue: Medical image analysis,
references.authors: Minh-Thang Luong Hieu Pham Christopher D Manning
references.title: Effective approaches to attention-based neural machine translation
references.authors: André FT Martins Ramón Astudillo Chris Hokamp Fabio Kepler
references.title: Unbabel’s participation in the wmt16 word-level translation quality estimation shared task
references.venue: In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers,
references.authors: André FT Martins Marcin Junczys-Dowmunt Fabio N Kepler Ramón Astudillo Chris Hokamp Roman Grundkiewicz
references.title: Pushing the limits of translation quality estimation
references.venue: Transactions of the Association for Computational Linguistics,
references.authors: Tanya Nair Doina Precup Douglas L Arnold Tal Arbel
references.title: Exploring uncertainty measures in deep networks for multiple sclerosis lesion detection and segmentation
references.venue: Medical image analysis,
references.authors: Radford M Neal
references.title: Bayesian learning for neural networks, volume 118
references.venue: Springer Science & Business Media,
references.authors: Franz Josef Och
references.title: Minimum error rate training in statistical machine translation
references.venue: In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics-Volume
references.authors: Myle Ott Michael Auli David Grangier Marc’Aurelio Ranzato
references.title: Analyzing uncertainty in neural machine translation
references.venue: In Proceedings of the 35th International Conference on Machine Learning, ICML 2018,
references.authors: Kishore Papineni Salim Roukos Todd Ward Wei-Jing Zhu
references.title: Bleu: a method for automatic evaluation of machine translation
references.venue: In Proceedings of the 40th annual meeting on Association for Computational Linguistics,
references.authors: Rico Sennrich Barry Haddow Alexandra Birch
references.title: Edinburgh neural machine translation systems for WMT 16. In Proceedings of the First Conference on Machine Translation, WMT 2016, colocated with ACL 2016, August 11-12, Berlin, Germany, pp. 371–376
references.authors: Matthew Snover Bonnie Dorr Richard Schwartz Linnea Micciulla John Makhoul
references.title: A study of translation edit rate with targeted human annotation
references.venue: In Proceedings of association for machine translation in the Americas,
references.authors: Frank Soboczenski Michael D Himes Molly D O’Beirne Simone Zorzan Atilim Gunes Baydin Adam D Cobb Yarin Gal Daniel Angerhausen Massimo Mascaro Giada N Arney
references.title: Bayesian deep learning for exoplanet atmospheric retrieval
references.authors: Lucia Specia Dhwaj Raj Marco Turchi
references.title: Machine translation evaluation versus quality estimation
references.venue: Machine translation,
references.authors: Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Łukasz Kaiser Illia Polosukhin
references.title: Attention is all you need
references.venue: In Advances in Neural Information Processing Systems,
references.authors: Ashish Vaswani Samy Bengio Eugene Brevdo François Chollet Aidan N. Gomez Stephan Gouws Llion Jones Lukasz Kaiser Nal Kalchbrenner Niki Parmar Ryan Sepassi Noam Shazeer Jakob Uszkoreit
references.title: Tensor2tensor for neural machine translation
references.venue: Proceedings of the 13th Conference of the Association for Machine Translation in the Americas,
references.authors: Stephan Vogel Hermann Ney Christoph Tillmann
references.title: Hmm-based word alignment in statistical translation
references.venue: In Proceedings of the 16th conference on Computational linguistics-Volume
references.authors: Mike Walmsley Lewis Smith Chris Lintott Yarin Gal Steven Bamford Hugh Dickinson Lucy Fortson Sandor Kruk Karen Masters Claudia Scarlata
references.title: Galaxy zoo: Probabilistic morphology through bayesian cnns and active learning
references.venue: Monthly Notices of the Royal Astronomical Society,
references.authors: Shuo Wang Yang Liu Chao Wang Huanbo Luan Maosong Sun
references.title: Improving back-translation with uncertainty-based confidence estimation
references.venue: In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing,
references.authors: Yonghui Wu Mike Schuster Zhifeng Chen Quoc V Le Mohammad Norouzi Wolfgang Macherey Maxim Krikun Yuan Cao Qin Gao Klaus Macherey
references.title: Google’s neural machine translation system: Bridging the gap between human and machine translation
sections.heading: 1 INTRODUCTION
sections.text: Statistical Machine Translation (SMT, (Brown et al., 1993; Och, 2003)), built on top of probabilistic modelling foundations such as the IBM alignment models (Vogel et al., 1996; Brown et al., 1993; Gal & Blunsom, 2013), has largely been replaced in recent years following the emergence of Neural Machine Translation approaches (NMT, (Kalchbrenner & Blunsom, 2013; Bahdanau et al., 2015; Luong et al., 2015; Vaswani et al., 2017)). This change has brought with it huge performance gains to the field (Sennrich et al., 2016), but at the same time we have lost many desirable properties of these models. Statistical probabilistic models can inform us when they are guessing at random on inputs they never saw before (Ghahramani, 2015). This information can be used, for example, to detect out-of-training-distribution examples for selective classification by referring uncertain inputs to an expert for annotation (Leibig et al., 2017), or for a human-in-the-loop approach to reduce data labelling costs (Gal et al., 2017; Walmsley et al., 2019; Kirsch et al., 2019).
  With new tools in machine learning we can now incorporate such probabilistic foundations into deep learning NLP models without sacrificing performance. This field, known as Bayesian Deep Learning (BDL, (Neal, 2012; Gal, 2016)), is concerned with the development of scalable tools which capture epistemic uncertainty—the model’s notion of “I don’t know”, a measure of a model’s lack of knowledge e.g. due to lack of training data, or when an input is given to the model which is very dissimilar to what the model has seen before. Such BDL tools have been used extensively in the Computer Vision literature (Kendall & Gal, 2017; Litjens et al., 2017), and have been demonstrated to be of practical use for applications including medical imaging (Litjens et al., 2017; Nair et al., 2020), robotics (Gal et al., 2016; Chua et al., 2018), and astronomy (Hon et al., 2018; Soboczenski et al., 2018; Hezaveh et al., 2017).
  In this paper we extend these tools, often used for vision tasks, to the language domain. We demonstrate how these tools can be used effectively on the task of selective classification in NMT by identifying source sentences the translation model has never seen before, and referring such source sentences to an expert for translation. We demonstrate this with state-of-the-art Transformer models, and show how model performance increases when rejecting sentences the model is uncertain about—i.e. the model’s measure of epistemic uncertainty correlates with mistranslations.
  For this we develop several measures of epistemic uncertainty for applications in natural language processing, concentrating on the task of machine translation (§3). We compare these measures both with standard deterministic Transformer models, as well as with Variational Transformers, a new approach we introduce to capture epistemic uncertainty in sequence models using dropout approximate inference (Gal, 2016). We give an extensive analysis of the methodology, and compare
  the different approaches quantitatively in the out-of-training-distribution settings (§4), which shows our proposed uncertainty estimate BLEUVar works well for measuring the epistemic uncertainty for machine translation. We also analyse the performance of BLEUVar qualitatively from both the influence of sentence length and from the linguistic perspective. We finish with a discussion in potential use cases for the new methodogy proposed.
  The closest NLP task to the above problem definition is the quality estimation (QE) task in Machine Translation (Specia et al., 2010; Blatz et al., 2004), which tries to solve a similar problem by predicting the quality of a translation with a score called Human Translation Error Rate (HTER, (Snover et al., 2006)). This is done by training a surrogate QE model on source sentences and their corresponding machine-generated translations in a specific domain, with the target of the surrogate to predict the the percentage of edits needed to be fixed. While many methods have been shown to successfully solve the task of estimating the quality of translations (Kim et al., 2017; Martins et al., 2016; 2017; Kreutzer et al., 2015), by definition QE crucially relies on examples of mistranslations to train the surrogate. The assumption that such training data is available is often violated in practice though (e.g. in active learning), thus existing approaches in QE research cannot generally be used to detect out-of-training-distribution examples (see Appendix C for detail discussion about the differences between QE and our task, as well as other related work that is similar to ours but not solving the same problem).
sections.heading: 2 BACKGROUND: UNCERTAINTY IN DEEP LEARNING
sections.text: For most machine learning models, the optimisation objectives give us a point estimate of the model parameters, which maximise the likelihood of the model generating the training data (i.e. p(Y |X,ω = ω∗), ω∗ ∈ Ω s.t. Ω is the set of all possible model parameters, Y,X are the training data). Such point estimate ω∗ gives us a very good prediction when the test data follow the same distribution as the training data distribution. Given a new input x∗ at test time, the model prediction for the corresponding y∗ is
  ŷ∗ = arg max y∗ p(y∗|x∗, ω∗). (1)
  However, we cannot expect the model to perform well on out-of-distribution (OOD) data which it never saw before. Instead, we would wish for the model to indicate its uncertainty towards such inputs. We could use p(ŷ∗|x∗, ω∗) as an estimate for model uncertainty, but as we show below, it would not be a well calibrated one. It might be the case that many ω might give equally good predictions on the train set, but might widely disagree with their predictions on OOD data. In fact, ω∗ might give arbitrary predictions on OOD training data which is very dissimilar to previously observed inputs. Thus, a high score does not distinguish whether x∗ is OOD or not, and whether we should trust the model’s prediction.
sections.heading: 2.1 BAYESIAN INFERENCE
sections.text: Bayesian probabilistic models capture the notion of uncertainty explicitly. Rather than considering a single point estimate ω∗, Bayesian models aim to capture the entire distribution of ω from the training data. The resulting distribution is called posterior distribution
  p(ω|X,Y ) = p(Y |X,ω)p(ω) p(Y |X) . (2)
  At test time, we can make prediction about the corresponding y∗ by integrating out all possible ω p(y∗|x∗, X, Y ) = ∫ p(y∗|x∗, ω)p(ω|X,Y )dω. (3)
  Using the variance of the predictive distribution p(y∗|x∗, X, Y ) as the uncertainty measure would have taken into account the variance of ω. Hence, an uncertainty measure based on this quantity could be a strong indicator for x∗ being OOD.
sections.heading: 2.2 APPROXIMATE INFERENCE
sections.text: The difficulty in doing Bayesian inference comes from the intractability of calculating the evidence p(Y |X) = ∫ p(Y |X,ω)p(ω)dω. (4)
  There might be a closed form solution for a simple model. But for most interesting problems, it is too difficult to compute an exact solution. Therefore, approximations are often used for such inference problems.
  Variational inference (VI) is a pragmatic popular method for doing approximate inference (Jordan et al., 1999). The method involves defining an approximating distribution qθ(ω), and trying to find the parameter θ for that minimises the Kullback-Leibler (KL) divergence (Kullback & Leibler, 1951) between qθ(ω) and the posterior
  KL(qθ(ω)‖p(ω|X,Y )) = ∫ qθ(ω)log qθ(ω)
  p(ω|X,Y ) dω. (5)
  The resulting θ∗ gives us the closest approximation for the posterior in the family of distribution qθ(ω). However, calculating the KL divergence here is also intractable as we still have to do the integration for the evidence in posterior.
  Fortunately, a tractable and equivalent objective for minimising the KL divergence is to maximise the evidence lower bound (ELBO, (Blei et al., 2017)) of qθ(ω)
  ELBO(qθ) = ∫ qθ(ω)log p(Y |X,ω)dω −KL(qθ(ω)‖p(ω)). (6)
  This is tractable as we know all the distributions (both qθ(ω), p(Y |X,ω) and p(ω) are defined by the model or by our assumptions).
sections.heading: 2.3 BAYESIAN INFERENCE IN DEEP LEARNING
sections.text: Almost all deep models treat the units in a deep neural network as deterministic functions. To adapt Bayesian methods in deep learning, we need to first turn our model into a probabilistic model. It can be done by modelling the weights in each unit of the network as samples from probability distributions. Such networks are called Bayesian neural networks (Neal, 2012). One major challenge with Bayesian neural networks is that the integration in ELBO becomes intractable when we have more than one hidden layer (Gal, 2016).
  Many works have tried tackling this problem. One practical method proposed by Gal (2016) is MC Dropout. Gal (2016) showed that optimising any neural networks with dropout can be viewed as an approximate inference in a probabilistic model (when dropout p is tuned correctly), which implies that a trained neural network with dropout can be interpreted as a Bayesian neural network Gal (2016). Stochastic forward passes with dropout ‘turned-on’ at test time then correspond to draws from the predictive distribution. Here we extend on these ideas and propose the Variational Transformer, which is based on MC Dropout applied to the original Transformer model. We perform extensive empirical evaluation with this model on the task of NMT. Representing uncertainty in a translation model is the first step towards detecting OOD data. We next discuss how to use this uncertainty effectively, and provide the main contribution of this work.
sections.heading: 3 MEASURES OF UNCERTAINTY FOR NMT
sections.text: Principally, we care about measuring the variance of a model’s outputs around some given input point. In the context of a simple classifier model, the solution is often found by measuring the mutual information between the predicted discrete distribution and model parameters, evaluating the output’s entropy, or simply computing the variance of model outputs (Gal, 2016). In the domain of language, however, there are many semantically equivalent alternatives to the same prediction, and it is a difficult matter to measure the disagreement between the predicted discrete sequences, which in turn complicates the estimation of variance in the output space. Much worse, when attempting to naively use MI or entropy with long sequences or large sets of discrete random variables, we quickly discover that even approximate integration over the product space becomes prohibitive (Kirsch et al., 2019). In order to capture epistemic uncertainty in the task of NMT, we propose several measures of uncertainty appropriate for long sequences of discrete variables (Beam Score and Sequence Probability are measures similar to (Wang et al., 2019; Fomicheva et al., 2020)):
  1. Beam Score: we assign a confidence to output y generated (using beam search) from input x using the score assigned to y’s beam (Wu et al., 2016), where length_penalty(y;α) = ( 5+|y| 5+1 )α .
  BS = log (pω∗(y|x))
  length_penalty(y; 0.6) (7)
  2. Sequence Probability: we assign a confidence to the deterministic model output y generated from input x by taking the log predictive probability under the weight distribution.
  SP = log ( Eω∼qθ∗ (ω) pω(y|x) ) length_penalty(y; 0.6)
  (8)
  3. BLEU Variance: ideally, we would like to measure the variance of outputs y as the uncertainty at an input x, i.e.:
  Var = Eω∼qθ∗ (ω)Ey∼pω(y|x) (y − µ) 2 = Eω∼qθ∗ (ω)Ey,y′∼pω(y|x) 1 2 (y − y′)2 , (9)
  where µ = Ey∼pω(y|x)[y]. If we treat sentences as points in some high dimensional space, ||y−y′|| corresponds to a distance between these two points, which is a numerical value representing the difference between two sentences. Thus, any metric for measuring the difference between sentences will allow us to calculate the variance of output y. In our experiments we choose BLEU (Papineni et al., 2002). The BLEU score of a candidate text to the reference text is a number between 0 and 1, with the value closer to 1 indicating the two texts are more similar1. Now, we can estimate the variance at an input x by producing pairs of outputs from the model and measuring the squared complement of the BLEU between them, i.e. ||y − y′||2 := (1− BLEU(y, y′))2, and we have:
  BLEUVar = Eω∼qθ∗ (ω)Ey,y′∼pω(y|x) (1− BLEU(y, y ′)) 2 (10)
  For the beam score we use the deterministic model found by gradient descent and simply take the probabilities from under its output probabilities. This will be our baseline. For sequence probability and BLEU variance we use MC Dropout (Gal, 2016) and take a number of samples (N ) to estimate the expectations2:
  SP ≈ log (∑N i=1 pωi(y|x) )
  length_penalty(y; 0.6) (11)
  For the BLEUVar approximation, we opt for decoding outputs using beam search applied to different model samples (realised by randomising the dropout masks) and measuring the complement BLEU between pairs of these examples.
  BLEUVar ≈ N∑ i=1 N∑ j 6=i ( 1− BLEU(decωi(x),decωj (x)) )2 . (12)
  Additionally, out of the N sample sequences generated by BLEUVar, we need to choose one or generate a new sequence as the result for a specific input. In regression, the mean of the samples is normally chosen as the result, which is an approximation for the predictive mean. In our case, the ‘mean’ of N sentences is hard to derive or even not properly defined. Therefore, we use the sampled sequence that is the closest to the ideal ‘mean’ as an approximation. Since we can measure the disagreement between any two sentences using BLEU, the sequence that is the closest to the ‘mean’ of the N samples must have the smallest disagreement with rest of the N − 1 samples. Hence, the final output sequence of the method BLEU Variance is:
  µ̃ = arg min yi ( N∑ ∀j 6=i ( 1− BLEU(yi, y′j) ) + N∑ ∀j 6=i ( 1− BLEU(y′j , yi) )) . (13)
  One remark for the above three methods is that the two methods BS and SP have the same resulting translation y given x, but with different values as its uncertainty estimates. For the method BLEUVar, it uses µ̃ for the resulting translation, and a value in a different range as the uncertainty estimate. This is reflected in the plots from the experiments section. For example, in Figure 6 (b), BS and SP converged into the same value, while BLEUVar converged to a different value.
sections.heading: 3.1 EVALUATING UNCERTAINTY IN SEQUENCE MODELS
sections.text: A number of uncertainty evaluation metrics have been proposed for standard classifier networks such as the popular ECE and MSE metrics (Guo et al., 2017); however, for sequence modelling these classification-specific metrics are not applicable. Instead, we opt for the performance versus retention curve method for evaluating our uncertainty measures following Filos et al. (2019).
  1A common practice in the NMT literature is to scale it up by ×100, i.e. in the range of [0, 100]. 2The approximations below should have constant scaling factors, but these don’t impact our evaluation metric
  (performance-retention curves) so we leave them out for simplicity.
  The performance-retention curve indicates how well an uncertainty measure would perform if the k% least certain outputs were deleted from the test dataset. The x-axis ranges along the fraction of data retained, while the y-axis measures some performance metric of the model on the retained data. A performance-retention curve of a well-calibrated uncertainty measure will see a clear and sustained improvement in performance as low-confidence predictions are excluded from the test set; while a poorly-calibrated model will yield a curve that either lacks a trend or tends to lay beneath the well-calibrated metric’s curve. In addition, we also calculate the area under a curve (AUC) as a summary statistic for comparing the curves. A larger AUC corresponds to a higher curve on average.
sections.heading: 4 EXPERIMENTS
sections.text: Experiments consist of evaluations on both in-distribution (see Appendix A) and out-of-distribution test sets. The implementation of the Transformer architecture we use is taken from the Tensor2Tensor (Vaswani et al., 2018) repository.
  As discussed in the previous sections we use performance-retention curves to evaluate the different uncertainty estimates of our models. We also use scatter plots of the uncertainty versus pairwise BLEU of the predictions to offer another gauge of model uncertainty (see Appendix A.2) To visualise the quality of uncertainty estimates for Transformers, we can order the generated sequence based on their uncertainty estimates from the most confident to the most uncertain, and plot BLEU scores as a function of the fraction of data retained starting from the least uncertain output. For a good uncertainty estimate, we are expecting to see the BLEU scores decrease when the fraction of retained data increases.
  The following datasets were used in our experiments: (1) WMT EN↔ DE: The training set for translation tasks between English (EN) and German (DE) composed of news-commentary-v13 with 284k sentences pairs, wmt13-commoncrawl with 2.4m sentences pairs and europarl-v7 with 1.9m sentences pairs, in total 4.6m sentences pairs. The test set was the newstest2014 with size 3k from WMT 2014. (2) WMT NL→ EN: The test set for Dutch (NL) to English (EN) translation was a subset (size 3k sentences pairs) of news-commentary-v14.
  In the experiments below, we denote the results from methods beam score, sequence probability, and BLEU Variance as BS, SP and BLEUVar respectively. If a suffix is added such as BLEUVar-10, then the suffix 10 indicates the number of samples taken during MC dropout.
sections.heading: 4.1 OUT-OF-DISTRIBUTION EXPERIMENTS
sections.text: The in-distribution tests showed in Appendix A illustrate that when the training set is large enough, the three uncertainty estimates have similar performance if the test set is in the same domain as the training set. However, when the training set only has limited data (e.g. 50k compares to 4.6m), BLEUVar outperforms BS and SP even though the test set is in the same domain as the training set. One explanation is that with the limited amount of training data, the model was not able to learn a distribution that capture the data from the test set. Therefore, to some extend, the test data are slightly out-of-distribution.
  In this section, we will have a look of experiments under two different obviously out-of-distribution settings (which in our application is equivalent to a domain shift) for evaluating our uncertainty estimates.
  The first experiment exploits the fact that the wordings and the sentence patterns might vary across different content domains in the same language. For example, sentences from a legal document and sentences from social media are different in terms of formality, even though they can both be in the same language. If we train a model using data from one domain and test it with data from another domain, then such input would be out-of-distribution for the model.
  The second experiment explores an extreme case of out-of-distribution setting. Given a trained model for translation task from languageA to languageB, if we test it on input from language C s.t. C 6= A, it would be an out-of-distribution input.
sections.heading: 4.1.1 UNCERTAINTY CAUSED BY DIFFERENT CONTENT DOMAINS
sections.text: To evaluate our uncertainty measures on out-of-content-domain test set, we trained a model using only the news-commentary-v13 data for German to English (DE-EN) task. This training set has 284k
  samples in the domain of news commentary. During test time, we used both the in-domain test data (newstest2014) and the out-of-domain test data (a subset of Europarl). newstest2014 is in the same domain as the training set. Europarl contains samples extracted from the proceedings of the European Parliament, which has a different domain than news commentary.
  We expect a good uncertainty measure to perform well in both in-domain and out-of-domain test set. As shown in Figure 1 and Table 1, BLEUVar outperforms the other two measures by a large margin for both test sets. In particular, the unstable performance of BS on the out-of-domain test data indicating such uncertainty measure might be not be reliable on out-of-distribution data, i.e. data it never saw before.
sections.heading: 4.1.2 UNCERTAINTY CAUSED BY DIFFERENT LANGUAGE DOMAINS
sections.text: We trained a Transformer model on WMT13 and Europarl DE (German) to EN (English) sentence pairs (obtaining BLEU 33 on the WMT14 test set). We then evaluated the model on out-of-trainingdistribution input sentences in NL (Dutch), which shares a large overlapping vocabulary with German (hence input sentences look plausible to non-native speakers). One would hope that such data falling outside of the training distribution would produce model predictions with high uncertainty. A similar experiment with a different language pair (French, German to English) can be found in Appendix B.
  The BLEU scores differences between Figure 2(a) and (b) shows that feeding Dutch (NL) sentences into a German to English model does not result in meaningful translations in general. Nevertheless, BLEUVar still provide a better uncertainty estimate with Dutch input (see Figure 2(b), Table 2(b)).
  In addition, Figure 2(c) shows the performance-retention curves for the combined DE+NL test set, in which BLEUVar outperforms BS and SP by a large margin. The fact that BLEUVar is close to the DE-EN reference curve indicates most of the DE input has been assigned with high confidence correctly. As the left half of the BLEUVar curve, which corresponds to the most certain half of the test data, nearly resembles the result from Figure 2(a).
  A more interesting result is shown in Figure 3. Given the combined DE+NL test set, BLEUVar is able to nicely separate the test set into to two clusters (DE and NL) using only the uncertainty estimates without evaluating on the target translation.
  Note that we do not try to solve the problem of language detection here. Instead, we use it as an extreme example to show that our uncertainty metric is able to assign very high uncertainty scores towards out-of-distribution test data while being more confident towards in-distribution data. But our methodology was not trained specifically nor designed for language detection. However, being able to separate the two shows that our methods do reflect the uncertainty of the model very well, it also shows that Bayesian deep NLP models can be very powerful.
sections.heading: 4.2 ANALYSIS OF SENTENCE LENGTH VERSUS UNCERTAINTY
sections.text: Use sentences with different lengths and uncertainties from section §4.1.2 as examples, we can see from the Table 3 that long sentences do not necessarily have consistently low/high uncertainty (BLEUVar). For the in-distribution test data, the model is very certain for sentences of length around 21-50, this is because the training sentences length distribution is centred around these sentence lengths (with 54% of the training data in this interval). For the OOD test data, the model is very uncertain across all sentence lengths, and as the sentences get longer the model becomes more uncertain. Note that average uncertainty for the shortest OOD sentences is ∼4700, which is much greater than the average uncertainty for the longest in-distribution sentences, ∼2700. An important point is that it is not necessary that all the short sentences must have low uncertainty, even for sentences in-distribution. I.e., our uncertainty metric is not simply a measure of sentence lengths (or even correlated with it).
  We further extended the experiment design and added a naive baseline which simply looks at the sentence length to reject sentences (ordering test data using sentence_lengthlongest_sentence_length , i.e. from short sentences to long sentences). Calculating BLEU scores at different retention rates (as in Figure 2(a), see Table 4), we see that the curve is much lower than BLEUVar. In fact, the performance of sentence length referral is worse than another naive baseline: randomly referring sentences without looking at them at all.
  3With the common practice of BLEU (i.e. ×100), the BLEUVar value results in ×1002 in the plots.
sections.heading: 4.3 INTROSPECTION INTO THE MODEL UNCERTAINTY FROM THE LINGUISTIC PERSPECTIVE
sections.text: Based on experiments in section §4.1.2, below are some example sentence translations sampled from the model (and with which we estimate the model uncertainty). As a reminder, the uncertainty here (BLEUVar) is the level of disagreement between sampled sentences, as determined by pairwise-BLEU (pairwise between each model output and the other model outputs). For a very confident translation from an in-distribution input (i.e. German), refer to Appendix D.1 Table 10. We can see that all translations sampled from the model are consistent with each other, and the model has no uncertainty at all. For other less certain in-distribution translations as showed in in Appendix D.2 Table 11, we can see that each translation sampled from the model is inconsistent with the others in subtle ways, leading to a larger variability in pair-wise BLEU scores. The model has high uncertainty, but still lower than that of the average OOD sentence. In contrast, Table 5 shows 3 truncated samples from OOD input (i.e. Dutch), the complete table with 5 full samples can be found in Appendix D.3. Here each translation sampled from the model is wildly inconsistent with the others, with some translations reminiscent of nonsense translations often encountered with neural systems when these are run on inputs they never saw before. We can identify these bad translations by the large variability in pair-wise BLEU scores. The model has much higher uncertainty than that of the average in-distribution sentences.
sections.heading: 5 FUTURE DIRECTIONS
sections.text: With the new tools above we can now develop NMT systems which can be deployed in scenarios where high trust is required of the system, for example in legal applications. With the new tools proposed we can integrate expert annotation in the deployment phase of the system by referring uncertain sentences to human annotation instead of automatic one. Further, with these new tools future research could examine human-in-the-loop approaches to NLP. Such approaches will allow us to develop NLP tools in scenarios when hand-labelling of data is too costly, for example language with scarce resources.
sections.heading: B ADDITIONAL OUT-OF-DISTRIBUTION EXPERIMENTS (FR+DE TO EN)
sections.text: We have done similar experiments as section §4.1.2 on other language pair. Figure 9 here uses a similar experiment design as Figure 2 in §4.1.2. Instead of testing DE and NL on model trained with DE-EN task, here the model is trained with FR (French) to EN (English) task and tests on FR to EN (in-distribution), DE to EN (OOD) and FR+DE to EN test sets. The pair FR and EN has much less overlap in vocabulary than DE and NL.
  The BLEU on the full combined test set (see Figure 9(c)) is the best the models can do; then BLEUVar rejects German sentences until it has mostly French sentences left and it has peak performance (data retained=0.3), after which it is forced to reject French sentences as well. Note that performance goes down for small retain rate because there is a small number of DE (OOD) data erroneously being assigned with high confidence, which our metric captures well.
  This result is similar to the DE+NL experiments in §4.1.2, with BLEUVar outperforms the rest by a large margin in the mixed test set (see Figure 9(c) and Table 9(c)). Further, the BLEU scores of OOD test is roughly flat compared to the amount of data retained (see Figure 9(b)).
sections.heading: C RELATED WORK
sections.text: Our work might look similar to quality estimation (QE) task in MT (Specia et al., 2010; Blatz et al., 2004), but the problem of QE is fairly different to what we do in this paper. QE assumes the existence of a fixed translation system (e.g., an in-house encoder-decoder attention-based NMT system, as in WMT19’s shared task in Quality Estimation). The QE models then have to determine the quality of the system’s output. In contrast, we look at the problem of “introspection” where the system has to decide the the confidence (“quality”) of its own output. This confidence can then be used for selective classification where the model can reject some uncertain translation. Further, standard approaches in QE might assume access to privileged data (e.g., the NMT translations for the source sentences and their corresponding human post-edition, as in task 2 in WMT19’QE), which we do not require. In addition, most existing approaches for QE require additional model to be trained to estimate the translation quality of a MT model, while our method does not have such requirement. Therefore, our method is able to provide uncertainty estimate simply with the parallel corpus used for training the translation model without the need for additional data and training procedure.
  The closest to our paper is task 3 in WMT19’QE: a metric to score sentences is sought, which must correlate to human judgement. We would like to stress that a system’s confidence in its own prediction does not have to be correlated to human judgement. Indeed, we demonstrate this in Appendix A.2 where a model can indicate that it does not have enough training data, and requires additional data to increase its confidence (the model’s subjective view of its uncertainty does not have to correlate with empirical mistakes - Bayesian epistemology (Zalta et al., 1995)).
  In addition, QE tasks are mainly focus on estimating the in-distribution translation quality, since the test sets are in the same domain as the training sets provided by WMT QE tasks (e.g. both in the IT domain for English-German WMT18,19). In contrast, the goal for our uncertainty estimate is to identify the out-of-distribution translations, rather then estimating the quality of in-distribution translation. Therefore, our tasks are fundamentally different to QE.
  There have been some prior attempts at investigating the similar problem as ours. In particular, Kumar & Sarawagi (2019) investigated the calibration of various NMT models at the token level. Kumar & Sarawagi found that many models are ill-calibrated at the token level, leading to the resulting probability distribution over the vocabulary used during decoding is not a good reference for model uncertainty. To correct for this, Kumar & Sarawagi design a recalibration strategy that applies an adaptive temperature to the logits, determined by the token identities, attention entropies, and other relevant components. Desai & Durrett (2020) looked into the calibration of pre-trained Transformers, and discovered that pre-trained Transformers are well calibrated for in-distribution data but ill-calibrated for out-of-distribution data. Such observations on NMT calibration further motivate us to design better uncertainty measures for NMT models.
  Another study of uncertainty in NMT models comes from Ott et al. (2018); they found models tend to have overly high uncertainty in their output distribution over sequences. Note that both do not consider epistemic uncertainty, not OOD settings. There are some work consider epistemic uncertainty (Fomicheva et al., 2020; Wang et al., 2019) and propose MC Dropout-based measures similar to our Sequence Probability (SP). Our work explores this direction and offers a new uncertainty estimation technique (i.e.BLEUVar) that empirically out-performs existing methods by a significant margin.
sections.heading: D TRANSLATION SAMPLES
sections.text: D.1 IN-DISTRIBUTION CERTAIN SAMPLES
  D.2 IN-DISTRIBUTION UNCERTAIN SAMPLES
  D.3 OUT-OF-DISTRIBUTION SAMPLES
  5 The debitenland gambgambgambgambgambgambgambgambgambgambgambgambgambgam bgambgamble in their own countries; the gambgambgambgambgambgambgambgambgambga mbgambgambgambgambgambgambgambgambgambgambgambgambgambgambgambgamb gambgambgambgambgambgambgambgambgambgambgambgambgambgambgambgambga mbgambgambgambgambgambgambgambgambgambgambgambgambgambgambgambgamb gambgambgambgambgambgambgambgambgambgambgambgambgambgambgambgambga mbgambgambgambgambgambgambgambgambgambgambgambgambgambgambgambgamb gambgambgambgambgambgambgambgambgambgambgambgambgambgambgambgambga mbgambgambgambgambgambgambgambgambgambgambgambgambgambgambgambgamb gambgambgambgambgambgambgambgambgambgambgamb