summaries: This paper introduces a novel technique for debiasing pretrained contextual embedding models. Their approach trains a 2 layer fully-connected neural network which takes as input the output from the pretrained model and outputs a new, "debiased" representation. This model is trained by minimizing the InfoNCE between the representation produced of original sentence and the representation of that same sentence with some tokens replaced with differently-biased tokens (e.g. "his" -> "hers"). This paper also introduces a regularizer which minimizes the CLUB between the generated representation and a word embedding for a biased token.  This paper studied a debiasing method to remove social bias in pretrained NLP models. The authors proposed to train a neural network which takes the sentence representations of a pretrained NLP model as input and outputs the unbiased representations. The neural network is trained by maximizing the mutual information between a sentence and its “counterpart sentence”, which is automatically generated by replacing sensitive words by other values (e.g. replacing “he” with “she”). Moreover, the network can be further trained by minimizing the mutual information between the sentence representation and its sensitive word representation. The experiments show that the proposed method can effectively reduce bias while achieving better downstream task performance of the pretrained model. The paper builds on recent working attempts to debais sentence encoders by considering modified sentences. Sentences are selected that, for example, contain gender cueing words, and then swap those words with a predetermined 'opposite' (i.e. man<->woman). The core novelty in the work is to train a lightweight modification the encoding of the sentence and its swap to (a) reduce the distance between the two embeddings, using a contrastive learning objective and (b) reduce the mutual information between cueing words and the new embeddings. Evaluated on a WHEAT style task, modified for sentences, the method performs significantly better than existing recent work. 
abstractText: Pretrained text encoders, such as BERT, have been applied increasingly in various natural language processing (NLP) tasks, and have recently demonstrated significant performance gains. However, recent studies have demonstrated the existence of social bias in these pretrained NLP models. Although prior works have made progress on word-level debiasing, improved sentence-level fairness of pretrained encoders still lacks exploration. In this paper, we proposed the first neural debiasing method for a pretrained sentence encoder, which transforms the pretrained encoder outputs into debiased representations via a fair filter (FairFil) network. To learn the FairFil, we introduce a contrastive learning framework that not only minimizes the correlation between filtered embeddings and bias words but also preserves rich semantic information of the original sentences. On real-world datasets, our FairFil effectively reduces the bias degree of pretrained text encoders, while continuously showing desirable performance on downstream tasks. Moreover, our post hoc method does not require any retraining of the text encoders, further enlarging FairFil’s application space.
authors.name: Pengyu Cheng
authors.name: Weituo Hao
authors.name: Siyang Yuan
authors.name: Lawrence Carin
references.authors: Alexander A Alemi Ian Fischer Joshua V Dillon Kevin Murphy
references.title: Deep variational information bottleneck
references.venue: In International Conference on Learning Representations,
references.authors: Su Lin Blodgett Solon Barocas Hal Daumé III Hanna Wallach
references.title: Language (technology) is power: A critical survey
references.venue: of” bias” in nlp. arXiv preprint arXiv:2005.14050,
references.authors: Piotr Bojanowski Edouard Grave Armand Joulin Tomas Mikolov
references.title: Enriching word vectors with subword information
references.venue: Transactions of the Association for Computational Linguistics,
references.authors: Tolga Bolukbasi Kai-Wei Chang James Y Zou Venkatesh Saligrama Adam T Kalai
references.title: Man is to computer programmer as woman is to homemaker? debiasing word embeddings
references.venue: In Advances in neural information processing systems,
references.authors: Shikha Bordia Samuel R. Bowman
references.title: Identifying and reducing gender bias in word-level language models
references.venue: In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop,
references.authors: Marc-Etienne Brunet Colleen Alkalay-Houlihan Ashton Anderson Richard Zemel
references.title: Understanding the origins of bias in word embeddings
references.venue: In International Conference on Machine Learning,
references.authors: Aylin Caliskan Joanna J Bryson Arvind Narayanan
references.title: Semantics derived automatically from language corpora contain human-like
references.venue: biases. Science,
references.authors: Kaytlin Chaloner Alfredo Maldonado
references.title: Measuring gender bias in word embeddings across domains and discovering new gender bias word categories
references.venue: In Proceedings of the First Workshop on Gender Bias in Natural Language Processing,
references.authors: Ting Chen Simon Kornblith Mohammad Norouzi Geoffrey Hinton
references.title: A simple framework for contrastive learning of visual representations
references.venue: In Proceedings of the 37th international conference on Machine learning,
references.authors: Xi Chen Yan Duan Rein Houthooft John Schulman Ilya Sutskever Pieter Abbeel
references.title: Infogan: Interpretable representation learning by information maximizing generative adversarial nets
references.venue: In Advances in neural information processing systems,
references.authors: Pengyu Cheng Weituo Hao Shuyang Dai Jiachang Liu Zhe Gan Lawrence Carin
references.title: Club: A contrastive log-ratio upper bound of mutual information
references.venue: In Proceedings of the 37th international conference on Machine learning,
references.authors: Pengyu Cheng Martin Renqiang Min Dinghan Shen Christopher Malon Yizhe Zhang Yitong Li Lawrence Carin
references.title: Improving disentangled text representation learning with informationtheoretic guidance
references.venue: In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,
references.authors: Alexis Conneau Douwe Kiela Holger Schwenk Loı̈c Barrault Antoine Bordes
references.title: Supervised learning of universal sentence representations from natural language inference data
references.authors: Jason V Davis Brian Kulis Prateek Jain Suvrit Sra Inderjit S Dhillon
references.title: Information-theoretic metric learning
references.venue: In Proceedings of the 24th international conference on Machine learning,
references.authors: Jacob Devlin Ming-Wei Chang Kenton Lee Kristina Toutanova
references.title: Bert: Pre-training of deep bidirectional transformers for language understanding
references.venue: In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics,
references.authors: Lucas Dixon John Li Jeffrey Sorensen Nithum Thain Lucy Vasserman
references.title: Measuring and mitigating unintended bias in text classification
references.venue: In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society,
references.authors: Behnam Gholami Pritish Sahu Ognjen Rudovic Konstantinos Bousmalis Vladimir Pavlovic
references.title: Unsupervised multi-target domain adaptation: An information theoretic approach
references.venue: IEEE Transactions on Image Processing,
references.authors: Aditya Grover Jure Leskovec
references.title: node2vec: Scalable feature learning for networks
references.venue: In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining,
references.authors: Michael Gutmann Aapo Hyvärinen
references.title: Noise-contrastive estimation: A new estimation principle for unnormalized statistical models
references.venue: In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics,
references.authors: Kaiming He Haoqi Fan Yuxin Wu Saining Xie Ross Girshick
references.title: Momentum contrast for unsupervised visual representation learning
references.venue: In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,
references.authors: R Devon Hjelm Alex Fedorov Samuel Lavoie-Marchildon Karan Grewal Phil Bachman Adam Trischler Yoshua Bengio
references.title: Learning deep representations by mutual information estimation and maximization
references.venue: In International Conference on Learning Representations,
references.authors: Mandar Joshi Danqi Chen Yinhan Liu Daniel S Weld Luke Zettlemoyer Omer Levy
references.title: Spanbert: Improving pre-training by representing and predicting spans
references.venue: Transactions of the Association for Computational Linguistics,
references.authors: Masahiro Kaneko Danushka Bollegala
references.title: Gender-preserving debiasing for pre-trained word embeddings
references.venue: arXiv preprint arXiv:1906.00742,
references.authors: Svetlana Kiritchenko Saif Mohammad
references.title: Examining gender and race bias in two hundred sentiment analysis systems
references.venue: In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics,
references.authors: Ryan Kiros Yukun Zhu Russ R Salakhutdinov Richard Zemel Raquel Urtasun Antonio Torralba Sanja Fidler
references.title: Skip-thought vectors. In Advances in neural information processing
references.authors: Solomon Kullback
references.title: Information theory and statistics
references.venue: Courier Corporation,
references.authors: Keita Kurita Nidhi Vyas Ayush Pareek Alan W Black Yulia Tsvetkov
references.title: Measuring bias in contextualized word representations
references.venue: In Proceedings of the First Workshop on Gender Bias in Natural Language Processing,
references.authors: Paul Pu Liang Irene Mengze Li Emily Zheng Yao Chong Lim Ruslan Salakhutdinov LouisPhilippe Morency
references.title: Towards debiasing sentence representations
references.venue: In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,
references.authors: Zhouhan Lin Minwei Feng Cicero Nogueira dos Santos Mo Yu Bing Xiang Bowen Zhou Yoshua Bengio
references.title: A structured self-attentive sentence embedding
references.venue: arXiv preprint arXiv:1703.03130,
references.authors: Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer Veselin Stoyanov
references.title: Roberta: A robustly optimized bert pretraining approach
references.authors: Laurens van der Maaten Geoffrey Hinton
references.title: Visualizing data using t-sne
references.venue: Journal of machine learning research,
references.authors: Sean MacAvaney Andrew Yates Arman Cohan Nazli Goharian
references.title: Cedr: Contextualized embeddings for document ranking
references.venue: In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval,
references.authors: Thomas Manzini Yao Chong Lim Yulia Tsvetkov Alan W Black
references.title: Black is to criminal as caucasian is to police: Detecting and removing multiclass bias in word embeddings
references.authors: Chandler May Alex Wang Shikha Bordia Samuel R. Bowman Rachel Rudinger
references.title: On measuring social biases in sentence encoders
references.venue: In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers),
references.authors: Tomas Mikolov Kai Chen Greg Corrado Jeffrey Dean
references.title: Efficient estimation of word representations in vector space
references.venue: arXiv preprint arXiv:1301.3781,
references.authors: Aaron van den Oord Yazhe Li Oriol Vinyals
references.title: Representation learning with contrastive predictive coding
references.venue: arXiv preprint arXiv:1807.03748,
references.authors: Sunghyun Park Han Suk Shim Moitreya Chatterjee Kenji Sagae Louis-Philippe Morency
references.title: Computational analysis of persuasiveness in social multimedia: A novel dataset and multimodal prediction approach
references.venue: In Proceedings of the 16th International Conference on Multimodal Interaction,
references.authors: Matthew E Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee Luke Zettlemoyer
references.title: Deep contextualized word representations
references.venue: In Proceedings of NAACL-HLT,
references.authors: Soujanya Poria Devamanyu Hazarika Navonil Majumder Gautam Naik Erik Cambria Rada Mihalcea
references.title: MELD: A multimodal multi-party dataset for emotion recognition in conversations
references.venue: In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics,
references.authors: Rui Qian Tianjian Meng Boqing Gong Ming-Hsuan Yang Huisheng Wang Serge Belongie Yin Cui
references.title: Spatiotemporal contrastive video representation learning
references.venue: arXiv preprint arXiv:2008.03800,
references.authors: Alec Radford Karthik Narasimhan Tim Salimans Ilya Sutskever
references.title: Improving language understanding by generative pre-training, 2018
references.authors: Victor Sanh Lysandre Debut Julien Chaumond Thomas Wolf
references.title: Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter
references.authors: Dinghan Shen Pengyu Cheng Dhanasekar Sundararaman Xinyuan Zhang Qian Yang Meng Tang Asli Celikyilmaz Lawrence Carin
references.title: Learning compressed sentence representations for ondevice text processing
references.venue: In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics,
references.authors: Richard Socher Alex Perelygin Jean Wu Jason Chuang Christopher D Manning Andrew Y Ng Christopher Potts
references.title: Recursive deep models for semantic compositionality over a sentiment treebank
references.venue: In Proceedings of the 2013 conference on empirical methods in natural language processing,
references.authors: Jiaming Song Pratyusha Kalluri Aditya Grover Shengjia Zhao Stefano Ermon
references.title: Learning controllable fair representations
references.venue: In The 22nd International Conference on Artificial Intelligence and Statistics,
references.authors: Gabriel Stanovsky Noah A Smith Luke Zettlemoyer
references.title: Evaluating gender bias in machine translation
references.venue: In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics,
references.authors: Fei Sun Jun Liu Jian Wu Changhua Pei Xiao Lin Wenwu Ou Peng Jiang
references.title: Bert4rec: Sequential recommendation with bidirectional encoder representations from transformer
references.venue: In Proceedings of the 28th ACM International Conference on Information and Knowledge Management,
references.authors: Jian Tang Meng Qu Mingzhe Wang Ming Zhang Jun Yan Qiaozhu Mei
references.title: Line: Large-scale information network embedding
references.venue: In Proceedings of the 24th international conference on world wide web,
references.authors: Michael V”olske Martin Potthast Shahbaz Syed Benno Stein
references.title: TL;DR: Mining Reddit to learn automatic summarization
references.venue: In Proceedings of the Workshop on New Frontiers in Summarization,
references.authors: Alex Wang Amanpreet Singh Julian Michael Felix Hill Omer Levy Samuel Bowman
references.title: Glue: A multi-task benchmark and analysis platform for natural language understanding
references.venue: In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP,
references.authors: Alex Warstadt Amanpreet Singh Samuel R Bowman
references.title: Neural network acceptability judgments
references.venue: Transactions of the Association for Computational Linguistics,
references.authors: Kilian Q Weinberger John Blitzer Lawrence K Saul
references.title: Distance metric learning for large margin nearest neighbor classification
references.venue: In Advances in neural information processing systems,
references.authors: Svante Wold Kim Esbensen Paul Geladi
references.title: Principal component analysis
references.venue: Chemometrics and intelligent laboratory systems,
references.authors: Tianyi Zhang Varsha Kishore Felix Wu Kilian Q Weinberger Yoav Artzi
references.title: Bertscore: Evaluating text generation with bert
sections.heading: 1 INTRODUCTION
sections.text: Text encoders, which map raw-text data into low-dimensional embeddings, have become one of the fundamental tools for extensive tasks in natural language processing (Kiros et al., 2015; Lin et al., 2017; Shen et al., 2019; Cheng et al., 2020b). With the development of deep learning, largescale neural sentence encoders pretrained on massive text corpora, such as Infersent (Conneau et al., 2017), ELMo (Peters et al., 2018), BERT (Devlin et al., 2019), and GPT (Radford et al., 2018), have become the mainstream to extract the sentence-level text representations, and have shown desirable performance on many NLP downstream tasks (MacAvaney et al., 2019; Sun et al., 2019; Zhang et al., 2019). Although these pretrained models have been studied comprehensively from many perspectives, such as performance (Joshi et al., 2020), efficiency (Sanh et al., 2019), and robustness (Liu et al., 2019), the fairness of pretrained text encoders has not received significant research attention.
  The fairness issue is also broadly recognized as social bias, which denotes the unbalanced model behaviors with respect to some socially sensitive topics, such as gender, race, and religion (Liang et al., 2020). For data-driven NLP models, social bias is an intrinsic problem mainly caused by the unbalanced data of text corpora (Bolukbasi et al., 2016). To quantitatively measure the bias degree of models, prior work proposed several statistical tests (Caliskan et al., 2017; Chaloner & Maldonado, 2019; Brunet et al., 2019), mostly focusing on word-level embedding models. To evaluate the sentence-level bias in the embedding space, May et al. (2019) extended the Word Embedding Association Test (WEAT) (Caliskan et al., 2017) into a Sentence Encoder Association Test (SEAT). Based on the SEAT test, May et al. (2019) claimed the existence of social bias in the pretrained sentence encoders.
  Although related works have discussed the measurement of social bias in sentence embeddings, debiasing pretrained sentence encoders remains a challenge. Previous word embedding debiasing methods (Bolukbasi et al., 2016; Kaneko & Bollegala, 2019; Manzini et al., 2019) have limited assistance to sentence-level debiasing, because even if the social bias is eliminated at the word level,
  ∗Equal contribution.
  the sentence-level bias can still be caused by the unbalanced combination of words in the training text. Besides, retraining a state-of-the-art sentence encoder for debiasing requires a massive amount of computational resources, especially for large-scale deep models like BERT (Devlin et al., 2019) and GPT (Radford et al., 2018). To the best of our knowledge, Liang et al. (2020) proposed the only sentence-level debiasing method (Sent-Debias) for pretrained text encoders, in which the embeddings are revised by subtracting the latent biased direction vectors learned by Principal Component Analysis (PCA) (Wold et al., 1987). However, Sent-Debias makes a strong assumption on the linearity of the bias in the sentence embedding space. Further, the calculation of bias directions depends highly on the embeddings extracted from the training data and the number of principal components, preventing the method from adequate generalization.
  In this paper, we proposed the first neural debiasing method for pretrained sentence encoders. For a given pretrained encoder, our method learns a fair filter (FairFil) network, whose inputs are the original embeddings of the encoder, and outputs are the debiased embeddings. Inspired by the multi-view contrastive learning (Chen et al., 2020), for each training sentence, we first generate an augmentation that has the same semantic meaning but in a different potential bias direction. We contrastively train our FairFil by maximizing the mutual information between the debiased embeddings of the original sentences and corresponding augmentations. To further eliminate bias from sensitive words in sentences, we introduce a debiasing regularizer, which minimizes the mutual information between debiased embeddings and the sensitive words’ embeddings. In the experiments, our FairFil outperforms Sent-Debias (Liang et al., 2020) in terms of the fairness and the representativeness of debiased embeddings, indicating our FairFil not only effectively reduces the social bias in the sentence embeddings, but also successfully preserves the rich semantic meaning of input text.
sections.heading: 2 PRELIMINARIES
sections.text: Mutual Information (MI) is a measure of the “amount of information” between two variables (Kullback, 1997). The mathematical definition of MI is
  I(x;y) := Ep(x,y) [ log p(x,y)
  p(x)p(y)
  ] , (1)
  where p(x,y) is the joint distribution of two variables (x,y), and p(x), p(y) are respectively the marginal distributions of x,y. Recently, mutual information has achieved considerable success when applied as a learning criterion in diverse deep learning tasks, such as conditional generation (Chen et al., 2016), domain adaptation (Gholami et al., 2020), representation learning (Chen et al., 2020), and fairness (Song et al., 2019). However, the calculation of exact MI in (1) is well-recognized as a challenge, because the expectation w.r.t p(x,y) is always intractable, especially when only samples from p(x,y) are provided. To this end, several upper and lower bounds have been introduced to estimate the MI with samples. For MI maximization tasks (Hjelm et al., 2018; Chen et al., 2020), Oord et al. (2018) derived a powerful MI estimator, InfoNCE, based on noise contrastive estimation (NCE) (Gutmann & Hyvärinen, 2010). Given a batch of sample pairs {(xi,yi)}Ni=1, the InfoNCE estimator is defined with a learnable score function f(x,y):
  INCE := 1
  N N∑ i=1 log exp(f(xi,yi)) 1 N ∑N j=1 exp(f(xi,yj)) . (2)
  For MI minimization tasks (Alemi et al., 2017; Song et al., 2019), Cheng et al. (2020a) introduced a contrastive log-ratio upper bound (CLUB) based on a variational approximation qθ(y|x) of conditional distribution p(y|x):
  ICLUB := 1
  N N∑ i=1 [ log qθ(yi|xi)− 1 N N∑ j=1 log qθ(yj |xi) ] . (3)
  In the following, we use the above two MI estimators to induce the sentence encoder, eliminating the biased information and preserving the semantic information from the original raw text.
sections.heading: 3 METHOD
sections.text: SupposeE(·) is a pretrained sentence encoder, which can encode a sentence x into low-dimensional embedding z = E(x). Each sentence x = (w1, w2, . . . , wL) is a sequence of words. The embedding space of z has been recognized to have social bias in a series of studies (May et al., 2019; Kurita
  et al., 2019; Liang et al., 2020). To eliminate the social bias in the embedding space, we aim to learn a fair filter network f(·) on top of the sentence encoder E(·), such that the output embedding of our fair filter d = f(z) can be debiased. To train the fair filter, we design a multi-view contrastive learning framework, which consists of three steps. First, for each input sentence x, we generate an augmented sentence x′ that has the same semantic meaning as x but in a different potential bias direction. Then, we maximize the mutual information between the original embedding z = f(x) and the augmented embedding z′ = f(x′) with the InfoNCE (Oord et al., 2018) contrastive loss. Further, we design a debiasing regularizer to minimize the mutual information between d and sensitive attribute words in x. In the following, we discuss these three steps in detail.
sections.heading: 3.1 DATA AUGMENTATIONS WITH SENSITIVE ATTRIBUTES
sections.text: We first describe the sentence data augmentation process for our FairFil contrastive learning. Denote a social sensitive topic as T = {D1,D2, . . . ,DK}, where Dk (k = 1, . . . ,K) is one of the potential bias directions under the topic. For example, if T represents the sensitive topic “gender”, then T consists two potential bias directions {D1,D2} = {“male”,“female”}. Similarly, if T is set as the major “religions” of the world, then T could contain {D1,D2,D3,D4} = {“Christianity”, “Islam”, “Judaism”, “Buddhism”} as four components. For a given social sensitive topic T = {D1, . . .DK}, if a word w is related to one of the potential bias direction Dk (denote as w ∈ Dk), we call w a sensitive attribute word of Dk (also called bias attribute word in Liang et al. (2020)). For a sensitive attribute word w ∈ Dk, suppose we can always find another sensitive attribute word u ∈ Dj , such that w and u has the equivalent semantic meaning but in a different bias direction. Then we call u as a replaceable word of w in direction Dj , and denote as u = rj(w). For the topic “gender” = {“male”, “female”}, the word w = “boy” is in the potential bias direction D1 = “male”; a replaceable word of “boy” in “female” direction is r2(w) = “girl” ∈ D2. With the above definitions, for each sentence x, we generate an augmented sentence x′ such that x′ has the same semantic meaning as x but in a different potential bias direction. More specifically, for a sentence x = (w1, w2, . . . , wL), we first find the sensitive word positions as an index set P , such that each wp (p ∈ P) is a sensitive attribute words in direction Dk. We further make a reasonable assumption that the embedding bias of direction Dk is only caused by the sensitive words {wp}p∈P in x. To sample an augmentation to x, we first select another potential bias direction Dj , and then replace all sensitive attribute words by their replaceable words in the direction Dj . That is, x′ = {v1, v2, . . . , vL}, where vl = wl if l /∈ P , and vl = rj(wl) if l ∈ P . In Table 1, we provide an example for sentence augmentation under the “gender” topic.
sections.heading: 3.2 CONTRASTIVE LEARNING FRAMEWORK
sections.text: After obtaining the sentence pair (x,x′) with the augmentation strategy from Section 3.1, we construct a contrastive learning framework to learn our debiasing fair filter f(·). As shown in the Figure 1(a), our framework consists of the following two steps:
  (1) We encode sentences (x,x′) into embeddings (z, z′) with the pretrained encoder E(·). Since x and x′ have the same meaning but different potential bias directions, the embeddings (z, z′) will have different bias directions, which are caused by the sensitive attributed words in x and x′.
  (2) We then feed the sentence embeddings (z, z′) through our fair filter f(·) to obtain the debiased embedding outputs (d,d′). Ideally, d and d′ should represent the same semantic meaning without social bias. Inspired by SimCLR (Chen et al., 2020), we encourage the overlapped semantic information between d and d′ by maximizing their mutual information I(d;d′). However, the calculation of I(d;d′) is practically difficult because only embedding samples of d and d′ are available. Therefore, we use the InfoNCE mutual information estimator (Oord et al., 2018) to minimize the lower bound of I(d;d′) instead. Based on a learnable score function g(·, ·),
  the contrastive InfoNCE estimator is calculated within a batch of samples {(di,d′i)}Ni=1:
  INCE = 1
  N N∑ i=1 log exp(g(di,d ′ i)) 1 N ∑N j=1 exp(g(di,d ′ j)) . (4)
  By maximize INCE, we encourage the difference between the positive pair score g(di,d′i) and the negative pair score g(di,d′j), so that di can share more semantic information with d ′ i than other embeddings d′j 6=i.
sections.heading: 3.3 DEBIASING REGULARIZER
sections.text: Practically, the contrastive learning framework in Section 3.2 can already show encouraging debiasing performance (as shown in the Experiments). However, the embedding d can contain extra biased information from z, that only maximizing I(d;d′) fails to eliminate. To encourage no extra bias in d, we introduce a debiasing regularizer which minimizes the mutual information between embedding d and the potential bias from embedding z. As discussed in Section 3.1, in our framework the potential bias of z is assumed to come from the sensitive attribute words in x. Therefore, we should reduce the bias word information from the debiased representation d. Let wp be the embedding of a sensitive attribute word wp in sentence x. The word embedding wp can always be obtained from the pretrained text encoders (Bordia & Bowman, 2019). We then minimize the mutual information I(wp;d), using the CLUB mutual information upper bound (Cheng et al., 2020a) to estimate I(wp;d) with embedding samples. Given a batch of embedding pairs {(di,wp)}Ni=1, we can calculate the debiasing regularizer as:
  ICLUB = 1
  N N∑ i=1 [ log qθ(w p i |di)− 1 N N∑ j=1 log qθ(w p j |di) ] , (5)
  where qθ is a variational approximation to ground-truth conditional distribution p(w|d). We parameterize qθ with another neural network. As proved in Cheng et al. (2020a), the better qθ(w|d) approximates p(w|d), the more accurate ICLUB serves as the mutual information upper bound. Therefore, besides the loss in (5), we also maximize the log-likelihood of qθ(w|d) with samples {(di,wpi )}Ni=1. Based on the above sections, the overall learning scheme of our fair filter (FairFil) is described in Algorithm 1. Also, we provide an intuitive explanation to the two loss terms in our framework. In Figure 1(b), the blue and red circles represent d and d′, respectively, in the embedding space. The intersection I(d;d′) is the common semantic information extracted from sentences x and x′, while the two shadow parts are the extra bias. Note that the perfect debiased embeddings lead to coincident circles. By maximizing INCE term, we enlarge the overlapped area of d and d′; by minimizing ICLUB, we shrink the biased shadow parts.
  Algorithm 1 Updating the FairFil with a sample batch Begin with the pretrained text encoder E(·), and a batch of sentences {xi}Ni=1. Find the sensitive attribute words {wp} and corresponding embeddings {wp}. Generate augmentation x′i from xi, by replacing {wp} with {rj(wp)}. Encode (xi,x′i) into embeddings di = f(E(xi)),d ′ i = f(E(x ′ i)).
  Calculate INCE with {(di,d′i)}Ni=1 and score function g. if adding debiasing regularizer then
  Update the variational approximation qθ(w|d) by maximizing log-likelihood with {(di,wpi )} Calculate ICLUB with qθ(w|d) and {(di,wpi )}Ni=1. Learning loss L = −INCE + βICLUB.
  else Learning loss L = −INCE. end if Update FairFil f and score function g by gradient descent with respect to L.
sections.heading: 4 RELATED WORK
sections.heading: 4.1 BIAS IN NATURAL LANGUAGE PROCESSING
sections.text: Social bias has recently been recognized as an important issue in natural language processing (NLP) systems. The studies on bias in NLP are mainly delineated into two categories: bias in the embedding spaces, and bias in downstream tasks (Blodgett et al., 2020). For bias in downstream tasks, the analyses cover comprehensive topics, including machine translation (Stanovsky et al., 2019), language modeling (Bordia & Bowman, 2019), sentiment analysis (Kiritchenko & Mohammad, 2018) and toxicity detection (Dixon et al., 2018). The social bias in embedding spaces has been studied from two important perspectives: bias measurements and and debiasing methods. To measure the bias in an embedding space, Caliskan et al. (2017) proposed a Word Embedding Association Test (WEAT), which compares the similarity between two sets of target words and two sets of attribute words. May et al. (2019) further extended the WEAT to a Sentence Encoder Association Test (SEAT), which replaces the word embeddings by sentence embeddings encoded from pre-defined biased sentence templates. For debiasing methods, most of the prior works focus on word-level representations (Bolukbasi et al., 2016; Bordia & Bowman, 2019). The only sentence-level debiasing method is proposed by Liang et al. (2020), which learns bias directions by PCA and subtracts them in the embedding space.
sections.heading: 4.2 CONTRASTIVE LEARNING
sections.text: Contrastive learning is a broad class of training strategies that learns meaningful representations by making positive and negative embedding pairs more distinguishable. Usually, contrastive learning requires a pairwise embedding critic as a similarity/distance of data pairs. Then the learning objective is constructed by maximizing the margin between the critic values of positive data pairs and negative data pairs. Previously contrastive learning has shown encouraging performance in many tasks, including metric learning (Weinberger et al., 2006; Davis et al., 2007), word representation learning (Mikolov et al., 2013), graph learning (Tang et al., 2015; Grover & Leskovec, 2016), etc. Recently, contrastive learning has been applied to the unsupervised visual representation learning task, and significantly reduced the performance gap between supervised and unsupervised learning (He et al., 2020; Chen et al., 2020; Qian et al., 2020). Among these unsupervised methods, Chen et al. (2020) proposed a simple multi-view contrastive learning framework (SimCLR). For each image data, SimCLR generates two augmented images, and then the mutual information of the two augmentation embeddings is maximized within a batch of training data.
sections.heading: 5 EXPERIMENTS
sections.text: We first describe the experimental setup in detail, including the pretrained encoders, the training of FairFil, and the downstream tasks. The results of our FairFil are reported and analyzed, along with the previous Sent-Debias method. In general, we evaluate our neural debiasing method from two perspectives: (1) fairness: we compare the bias degree of the original and debiased sentence embeddings for debiasing performance; and (2) representativeness: we apply the debiased embeddings into downstream tasks, and compare the performance with original embeddings.
sections.heading: 5.1 BIAS EVALUATION METRIC
sections.text: To evaluate the bias in sentence embeddings, we use the Sentence Encoder Association Test (SEAT) (May et al., 2019), which is an extension of the Word Embedding Association Test (WEAT) (Caliskan et al., 2017). The WEAT test measures the bias in word embeddings by comparing the distances of two sets of target words to two sets of attribute words. More specifically, denote X and Y as two sets of target word embeddings (e.g., X includes “male” words such as “boy” and “man”; Y contains “female” words like “girl” and “woman”). The attribute setsA and B are selected from some social concepts that should be “equal” to X and Y (e.g., career or personality words). Then the bias degree w.r.t attributes (A,B) of each word embedding t is defined as:
  s(t,A,B) = meana∈A cos(t,a)−meanb∈B cos(t, b), (6)
  where cos(·, ·) is the cosine similarity. Based on (6), the normalized WEAT effect size is:
  dWEAT = meanx∈X s(x,A,B)−meany∈Ys(y,A,B)
  stdt∈X∪Ys(t,A,B) . (7)
  The SEAT test extends WEAT by replacing the word embeddings with sentence embeddings. Both target words and attribute words are converted into sentences with several semantically bleached sentence templates (e.g., “This is <word>”). Then the SEAT statistic is similarly calculated with (7) based on the embeddings of converted sentences. The closer the effect size is to zero, the more fair the embeddings are. Therefore, we report the absolute effect size as the bias measure.
sections.heading: 5.2 PRETRAINED ENCODERS
sections.text: We test our neural debiasing method on BERT (Devlin et al., 2019). Since the pretrained BERT requires the additional fine-tuning process for downstream tasks, we report the performance of our FairFil under two scenarios: (1) pretrained BERT: we directly learn our FairFil network based on pretrained BERT without any additional fine-tuning; and (2) BERT post tasks: we fix the parameters of the FairFil network learned on pretrained BERT, and then fine-tune the BERT+FairFil together on task-specific data. Note that when fine-tuning, our FairFil will no longer update, which satisfies a fair comparison to Sent-Debias (Liang et al., 2020).
  For the downstream tasks of BERT, we follow the setup from Sent-Debias (Liang et al., 2020) and conduct experiments on the following three downstream tasks: (1) SST-2: A sentiment classification task on the Stanford Sentiment Treebank (SST-2) dataset (Socher et al., 2013), on which sentence embeddings are used to predict the corresponding sentiment labels; (2) CoLA: Another sentiment classification task on the Corpus of Linguistic Acceptability (CoLA) grammatical acceptability judgment (Warstadt et al., 2019); and (3) QNLI: A binary question answering task on the Question Natural Language Inference (QNLI) dataset (Wang et al., 2018).
sections.heading: 5.3 TRAINING OF FAIRFIL
sections.text: We parameterize the fair filter network with one-layer fully-connected neural networks with the ReLU activation function. The score function g in the InfoNCE estimator is set to a two-layer fully-connected network with one-dimensional output. The variational approximation qθ in CLUB estimator is parameterized by a multi-variate Gaussian distribution qθ(w|d) = N(µ(d),σ2(d)), where µ(·) and σ(·) are also two-layer fully-connected neural nets. The batch size is set to 128. The learning rate is 1× 10−5. We train the fair filter for 10 epochs. For an appropriate comparison, we follow the setup of Sent-Debias (Liang et al., 2020) and select the same training data for the training of FairFil. The training corpora consist 183,060 sentences from the following five datasets: WikiText-2 (Merity et al., 201y), Stanford Sentiment Treebank (Socher et al., 2013), Reddit (V”olske et al., 2017), MELD (Poria et al., 2019) and POM (Park et al., 2014). Following Liang et al. (2020), we mainly select “gender” as the sensitive topic T , and use the same pre-defined word sets of sensitive attribute words and their replaceable words as Sent-Debias did. The word embeddings for training the debiasing regularizer is selected from the token embedding of the pretrained BERT.
sections.heading: 5.4 DEBIASING RESULTS
sections.text: In Tables 2 and 3 we report the evaluation results of debiased embeddings on both the absolute SEAT effect size and the downstream classification accuracy. For the SEAT test, we follow the
  setup in Liang et al. (2020), and test the sentence templates of Terms/Names under different domains designed by Caliskan et al. (2017). The column name Origin refers to the original BERT results, and Sent-D is short for Sent-Debias (Liang et al., 2020). FairFil− and FairFil (as FairF− and FairF in the tables) are our method without/with the debiasing regularizer in Section 3.3. The best results of effect size (the lower the better) and classification accuracy (the higher the better) are bold among Sent-D, FairFil−, and FairFil. Since the pretrained BERT does not correspond to any downstream task, the classification accuracy is not reported for it.
  From the SEAT test results, our contrastive learning framework effectively reduces the gender bias for both pretrained BERT and fine-tuned BERT under most test scenarios. Comparing with Sent-Debias, our FairFil reaches a lower bias degree on the majority of the individual SEAT tests. Considering the average of absolute effect size, our FairFil is distinguished by a significant margin to Sent-Debias. Moreover, our FairFil achieves higher downstream classification accuracy than Sent-Debias, which indicates learning neural filter networks can preserve more semantic meaning than subtracting bias directions learned from PCA.
  For the ablation study, we also report the results of FairFil without the debiasing regularizer, as in FairF−. Only with the contrastive learning framework, FairF− already reduces the bias effectively and even achieves better effect size than the FairF on some of the SEAT tests. With the debiasing regularizer, FairF has better average SEAT effect sizes but slightly loses in terms of the downstream performance. However, the overall performance of FairF and FairF− shows a trade-off between fairness and representativeness of the filter network.
  We also compare the debiasing performance on a broader class of baselines, including word-level debiasing methods, and report the average absolute SEAT effect size on the pretrained BERT encoder. Both FairF− and FairF achieve a lower bias degree than other baselines. The word-level debiasing methods (FastText (Bojanowski et al., 2017) and BERT word (Bolukbasi et al., 2016))
  have the worst debiasing performance, which validates our observation that the word-level debiasing methods cannot reduce sentence-level social bias in NLP models.
sections.heading: 5.5 ANALYSIS
sections.text: To test the influence of data proportion on the model’s debiasing performance, we select WikiText-2 with 13,750 sentences as the training corpora following the setup in Liang et al. (2020). Then we randomly divide the training data into 5 equal-sized partitions. We evaluate the bias degree of the sentence debiasing methods on different combinations of the partitions, specifically with training data proportions (20%, 40%, 60%, 80%, 100%). Under each data proportion, we repeat the training 5 times to obtain the mean and variance of the absolute SEAT effect size. In Figure 2, we plot the bias degree of BERT post tasks with different training data proportions. In general, both Sent-Debias and FairFil achieve better performance and smaller variance when the proportion of training data is larger. Under a 20% training proportion, our FairFil can better remove bias in text encoder, which shows FairFil has better data efficiency with the contrastive learning framework.
  To further study output debiased sentence embedding, we visualize the relative distances of attributes and targets of SEAT before/after our debiasing process. We choose the target words as “he” and “she.” Attributes are selected from different social domains. We first contextualize the selected words into sentence templates as described in Section 5.1. We then average the original/debiased embeddings of these sentence template and plot the t-SNE (Maaten & Hinton, 2008) in Figure 3. From the t-SNE, the debiased encoder provides more balanced distances from gender targets “he/she” to the attribute concepts.
sections.heading: 6 CONCLUSIONS
sections.text: This paper has developed a novel debiasing method for large-scale pretrained text encoder neural networks. We proposed a fair filter (FairFil) network, which takes the original sentence embeddings as input and outputs the debiased sentence embeddings. To train the fair filter, we constructed a multi-view contrast learning framework, which maximizes the mutual information between each sentence and its augmentation. The augmented sentence is generated by replacing sensitive words in the original sentence with words in a similar semantic but different bias directions. Further, we designed a debiasing regularizer that minimizes the mutual information between the debiased embeddings and the corresponding sensitive words in sentences. Experimental results demonstrate the proposed FairFil not only reduces the bias in sentence embedding space, but also maintains the semantic meaning of the embeddings. This post hoc method does not require access to the training corpora, or any retraining process of the pretrained text encoder, which enhances its applicability.
sections.heading: ACKNOWLEDGEMENTS
sections.text: This research was supported in part by the DOE, NSF and ONR.
title: FAIRFIL: CONTRASTIVE NEURAL DEBIASING METHOD FOR PRETRAINED TEXT ENCODERS