summaries: This paper is dedicated to proposing a self-supervised objective, local prior matching (LMP), for speech recognition. This approach can take advantage of vase quantities of unlabeled speech data. What' more, the objective is simple to implement and theoretically well-motivated. In the paper, based on a supervised pretrained model, it then finetunes 360 hours with unlabeled data and LPM reduces the WER consistently. They also conduct extensive ablation experiments to show the effect of their self-supervised approach.

summaries: This work proposed a distillation approach which use ASRs to generate hypotheses for unsupervised data, run a LM to get probability for the hypothesis, and perform distillation with the resulting probability. The ASRs being used for generating hypotheses can be either a model trained with the supervised data or the student model, and can switch between the two during training. In the experiments, ASR models are pre-trained with the subset of Librispeech data and use the rest of Librispeech data as unsupervised data, and the LM is trained with Librispeech LM data. The experiments shown the proposed approach improve baseline model trained with the Librispeech subset significantly.

abstractText: We propose local prior matching (LPM), a self-supervised objective for speech recognition. The LPM objective leverages a strong language model to provide learning signal given unlabeled speech. Since LPM uses a language model, it can take advantage of vast quantities of both unpaired text and speech. The loss is theoretically well-motivated and simple to implement. More importantly, LPM is effective. Starting from a model trained on 100 hours of labeled speech, with an additional 360 hours of unlabeled data LPM reduces the WER by 26% and 31% relative on a clean and noisy test set, respectively. This bridges the gap by 54% and 73% WER on the two test sets relative to a fully supervised model on the same 360 hours with labels. By augmenting LPM with an additional 500 hours of noisy data, we further improve the WER on the noisy test set by 15% relative. Furthermore, we perform extensive ablative studies to show the importance of various configurations of our self-supervised approach.

authors:
Jordi Adell
Antonio Bonafonte

title: Towards phone segmentation for concatenative speech synthesis

venue: In Fifth ISCA Workshop on Speech Synthesis,

authors:
Dario Amodei
Sundaram Ananthanarayanan
Rishita Anubhai
Jingliang Bai
Eric Battenberg
Carl Case
Jared Casper
Bryan Catanzaro
Qiang Cheng
Guoliang Chen

title: Deep Speech 2: Endto-end speech recognition in English and Mandarin

venue: In Proc. International Conference on Machine Learning (ICML),

authors:
Dzmitry Bahdanau
Jan Chorowski
Dmitriy Serdyuk
Philemon Brakel
Yoshua Bengio

title: Endto-end attention-based large vocabulary speech recognition

venue: In Proc. International Conference on Acoustics, Speech and Signal Processing (ICASSP),

authors:
Murali Karthick Baskar
Shinji Watanabe
Ramon Astudillo
Takaaki Hori
Lukáš Burget
Jan Černockỳ

title: Semi-supervised sequence-to-sequence ASR using unpaired speech and text

venue: In Proc. Annual Conference of International Speech Communication Association (INTERSPEECH),

authors:
Yoshua Bengio
Nicholas Léonard
Aaron Courville

title: Estimating or propagating gradients through stochastic neurons for conditional computation

venue: arXiv preprint arXiv:1308.3432,

authors:
Chung-Cheng Chiu
Tara N Sainath
Yonghui Wu
Rohit Prabhavalkar
Patrick Nguyen
Zhifeng Chen
Anjuli Kannan
Ron J Weiss
Kanishka Rao
Ekaterina Gonina

title: State-of-the-art speech recognition with sequence-to-sequence models

title: Knowledge of language: Its nature, origin, and use

authors:
Jan Chorowski
Navdeep Jaitly

title: Towards better decoding and language model integration in sequence to sequence models

authors:
Jia Cui
Brian Kingsbury
Bhuvana Ramabhadran
George Saon
Tom Sercu
Kartik Audhkhasi
Abhinav Sethy
Markus Nussbaum-Thom
Andrew Rosenberg

title: Knowledge distillation across ensembles of multilingual models for low-resource languages

authors:
Yann N Dauphin
Angela Fan
Michael Auli
David Grangier

title: Language modeling with gated convolutional networks

authors:
Jennifer Drexler
James Glass

title: Combining end-to-end and adversarial training for low-resource speech recognition

venue: In Proc. IEEE Spoken Language Technology Workshop (SLT),

title: Cognitive science in the era of artificial intelligence: A roadmap for reverseengineering the infant

venue: language-learner. Cognition,

title: Towards unsupervised speech processing

venue: In International Conference on Information Science, Signal Processing and their Applications (ISSPA),

authors:
Ian Goodfellow
Jean Pouget-Abadie
Mehdi Mirza
Bing Xu
David Warde-Farley
Sherjil Ozair
Aaron Courville
Yoshua Bengio

title: Generative adversarial nets

venue: In Proc. Neural Information Processing Systems (NeurIPS),

authors:
Awni Hannun
Carl Case
Jared Casper
Bryan Catanzaro
Greg Diamos
Erich Elsen
Ryan Prenger
Sanjeev Satheesh
Shubho Sengupta
Adam Coates

title: Deep Speech: Scaling up end-to-end speech recognition

venue: arXiv preprint arXiv:1412.5567,

authors:
Awni Hannun
Ann Lee
Qiantong Xu
Ronan Collobert

title: Sequence-to-sequence speech recognition with time-depth separable convolutions

authors:
Tomoki Hayashi
Shinji Watanabe
Yu Zhang
Tomoki Toda
Takaaki Hori
Ramon Astudillo
Kazuya Takeda

title: Back-translation-style data augmentation for end-to-end ASR

authors:
Takaaki Hori
Ramon Astudillo
Tomoki Hayashi
Yu Zhang
Shinji Watanabe
Jonathan Le Roux

title: Cycle-consistency training for end-to-end speech recognition

authors:
Wei-Ning Hsu
James Glass

title: Extracting domain invariant features by unsupervised learning for robust automatic speech recognition

authors:
Wei-Ning Hsu
Yu Zhang
James Glass

title: Unsupervised domain adaptation for robust speech recognition via variational autoencoder-based data augmentation

venue: In Proc. IEEE Workshop on Automfatic Speech Recognition and Understanding (ASRU),

authors:
Wei-Ning Hsu
Hao Tang
James Glass

title: Unsupervised adaptation with interpretable disentangled representations for distant conversational speech recognition

authors:
Eric Jang
Shixiang Gu
Ben Poole

title: Categorical reparameterization with gumbel-softmax

venue: In Proc. International Conference on Learning Representations (ICLR),

authors:
Jacob Kahn
Ann Lee
Awni Hannun

title: Self-training for end-to-end speech recognition

venue: arXiv preprint arXiv:1606.07947,

authors:
Shigeki Karita
Shinji Watanabe
Tomoharu Iwata
Atsunori Ogawa
Marc Delcroix

title: Semisupervised end-to-end speech recognition

title: Subword regularization: Improving neural network translation models with multiple subword candidates

venue: In Proc. Annual Meeting of the Association for Computational Linguistics (ACL),

authors:
Taku Kudo
John Richardson

title: Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing

venue: In Proc. Empirical Methods in Natural Language Processing (EMNLP),

title: Early language acquisition: cracking the speech code

venue: Nature Reviews Neuroscience,

authors:
Bo Li
Tara N Sainath
Arun Narayanan
Joe Caroselli
Michiel Bacchiani
Ananya Misra
Izhak Shafran
Hasim Sak
Golan Pundak
Kean K Chin

title: Acoustic modeling for Google home

authors:
Bo Li
Tara N Sainath
Ruoming Pang
Zelin Wu

title: Semi-supervised training for end-to-end models via weak distillation

authors:
Alexander H Liu
Hung-yi Lee
Lin-shan Lee

title: Adversarial training of end-to-end speech recognition using a criticizing language model

authors:
Vimal Manohar
Hossein Hadian
Daniel Povey
Sanjeev Khudanpur

title: Semi-supervised training of acoustic models using lattice-free MMI

authors:
Zhong Meng
Zhuo Chen
Vadim Mazalov
Jinyu Li
Yifan Gong

title: Unsupervised adaptation with domain separation networks for robust speech recognition

authors:
Zhong Meng
Jinyu Li
Yifan Gong
Biing-Hwang Juang

title: Adversarial teacher-student learning for unsupervised domain adaptation

authors:
Zhong Meng
Jinyu Li
Yifan Gong

title: Attentive adversarial learning for domain-invariant training

authors:
Ramesh Nallapati
Bowen Zhou
Caglar Gulcehre
Bing Xiang

title: Abstractive text summarization using sequence-to-sequence RNNs and beyond

venue: In Proc. ACL Conference on Natural Language Learning (CoNLL),

authors:
Vassil Panayotov
Guoguo Chen
Daniel Povey
Sanjeev Khudanpur

title: Librispeech: an ASR corpus based on public domain audio books

authors:
Sree Hari Krishnan Parthasarathi
Nikko Strom

title: Lessons from building acoustic models with a million hours of speech

authors:
Kainan Peng
Wei Ping
Zhao Song
Kexin Zhao

title: Parallel neural text-to-speech

venue: arXiv preprint arXiv:1905.08459,

authors:
Vineel Pratap
Awni Hannun
Qiantong Xu
Jeff Cai
Jacob Kahn
Gabriel Synnaeve
Vitaliy Liptchinsky
Ronan Collobert

title: wav2letter++: The fastest open-source speech recognition system

venue: arXiv preprint arXiv:1812.07625,

title: Non-mainstream languages and speech recognition: Some challenges

venue: CALICO journal,

authors:
Odette Scharenborg
Vincent Wan
Mirjam Ernestus

title: Unsupervised speech segmentation: An analysis of the hypothesized phone boundaries

venue: The Journal of the Acoustical Society of America,

authors:
Rico Sennrich
Barry Haddow
Alexandra Birch

title: Improving neural machine translation models with monolingual data

authors:
Nitish Srivastava
Geoffrey Hinton
Alex Krizhevsky
Ilya Sutskever
Ruslan Salakhutdinov

title: Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning

authors:
Sining Sun
Binbin Zhang
Lei Xie
Yanning Zhang

title: An unsupervised deep domain adaptation approach for robust speech

venue: recognition. Neurocomputing,

authors:
Andros Tjandra
Sakriani Sakti
Satoshi Nakamura

title: Listening while speaking: Speech chain by deep learning

title: End-to-end feedback loss in speech chain framework via straight-through estimator

title: Attention is all you need

authors:
Karel Veselỳ
Lukás Burget
Jan Cernockỳ

title: Semi-supervised DNN training with word selection for ASR

authors:
Yu-Hsuan Wang
Cheng-Tao Chung
Hung-yi Lee

title: Gate activation signal analysis for gated recurrent neural networks and its correlation with phoneme boundaries

title: Simple statistical gradient-following algorithms for connectionist reinforcement learning

venue: Machine learning,

authors:
Zichao Yang
Zhiting Hu
Chris Dyer
Eric P Xing
Taylor Berg-Kirkpatrick

title: Unsupervised text style transfer using language models as discriminators

authors:
Neil Zeghidour
Qiantong Xu
Vitaliy Liptchinsky
Nicolas Usunier
Gabriel Synnaeve
Ronan Collobert

title: Fully convolutional speech recognition

venue: arXiv preprint arXiv:1812.06864,

authors:
Jun-Yan Zhu
Taesung Park
Phillip Isola
Alexei A Efros

title: Unpaired image-to-image translation using cycle-consistent adversarial networks

venue: In Proc. IEEE International Conference on Computer Vision (ICCV),

heading: 1 INTRODUCTION

text:
Fully supervised learning remains the mainstream paradigm for state-of-the-art automatic speech recognition (ASR). These systems require huge annotated data sets (Li et al., 2017; Chiu et al., 2018; Hannun et al., 2014; Amodei et al., 2016), which are time-consuming and expensive to collect. This hinders the development of accurate ASR systems for low resource languages (Precoda, 2013). In fact, out of over 6,000 spoken languages, fewer than 150 are supported by commercial ASR service providers. In sharp contrast to how we teach machines to recognize speech, humans do not learn by listening to thousand hours of speech and simultaneously reading the corresponding transcriptions. Instead, as noted in (Chomsky, 1986; Kuhl, 2004; Glass, 2012; Dupoux, 2018), humans possess an inherent ability to learn from vast quantities of unlabeled speech. Consider the case of conversing with someone with a strong accent. Even when the speaker pronounces several words in an unusual way, one can often correctly understand the sentence. We argue that the source of indirect supervision in processing unlabeled speech comes from prior knowledge about the world and the context of the speech.
Inspired by this, we devise a self-supervised learning framework termed local prior matching (LPM). We apply this framework to speech recognition allowing an ASR model to learn from unlabeled speech by leveraging a strong language model, which serves as the prior for self-supervision. Given an unlabeled utterance, the ASR model proposes multiple hypotheses and the language model provides a learning signal by evaluating the plausibility of each one.
We evaluate the LPM method on the LibriSpeech corpus (Panayotov et al., 2015), using 100 hours of labeled speech to seed the proposal model. Using 360 hours of additional labeled data reduces the word error rate (WER) by 3.8% and 13.1% absolute on an easier and a more-challenging test set, respectively. Using the same 360 hours but without labels, LPM reduces the WER by 2.1% and 9.5% absolute on the same two test sets, effectively bridging the gap to fully-supervised learning by 54% and 73%. In addition, by augmenting LPM with another 500 hours, for a total of 860 hours of unlabeled speech, LPM reduces WER on the noisier test set by 14.2% in total. Hence, LPM is able to surpass the performance of using 360 hours of labeled data by taking advantage of about twice the amount of unlabeled data. We also conduct extensive ablation studies and analyses in order to demonstrate the significance of each proposed component. For reproducibility, software will be open source and made publicly available in the camera-ready version.

heading: 2 METHOD

heading: 2.1 PRELIMINARIES

text:
Let x denote an utterance of speech and y be a transcription. We assume speech is generated following a two-step process: y ∼ py x ∼ px|y, (1) where the text y is first generated from the language model (LM) py, and the speech x is then generated from a text-to-speech (TTS) model px|y conditioned on y. The posterior py|x is then the ASR model of interest. For convenience, we refer to pxy(x,y) = py(y) px|y(x | y) as the joint distribution, and px(x) = ∑ y pxy(x,y) as the marginal distribution. In a typical supervised learning setting, one has access to a labeled dataset Dl, which contains paired samples {(x(i),y(i))}Ni=1 drawn from the joint distribution. An ASR model qy|x can be trained by minimizing the marginal-weighted KL-divergence: Ex∼px [DKL(py|x || qy|x)], which can be estimated with∑
(x,y)∈Dl − log qy|x(y | x) + const from samples (i.e. the commonly used cross-entropy loss). In the self-supervised setting we only have access to unlabeled speech dataset Dsu, yet we still wish to optimize the same marginal weighted KL-divergence. Since we do not have samples from py|x=x directly, we can, under mild assumptions, estimate it.
Assumption 1 Different y can lead to the same pronunciation. One reason is the tokenization of words may not be unique. For example y can be the sequence of word-pieces “c at ” or “ca t ”, both of which result in the same spoken word. Homophones (e.g. “right” and “write”) also have the same pronunciation. To formalize this property, let f : Y → Z be a function which maps a text sequence to its pronunciation z ∈ Z (a broader definition of “pronunciation” here, where z = f(y) is a distribution over phoneme sequences for a given text y), and let px|y(x | y) = px|z(x | f(y)). We let the set Yz = {y : f(y) = z} ⊂ Y contain all text samples with the same pronunciation and hence the same conditional distribution, px|z=z .
Assumption 2 We assume that two pairs (y,y′) that have different pronunciations (i.e., f(y) 6= f(y′)) either sound similar or are very different. For instance, “lose” and “loose” result in almost the same pronunciation whereas an English speaker would rarely mistake “winter is coming” for “you know nothing”. In the former case, the conditional distributions px|y(x | y) and px|y(x | y′) may overlap substantially whereas in the latter case they barely intersect. More formally, for each z, we assume the pronunciation space Z can be partitioned by acoustic similarity asZ+z ∪Z−z . For all z+ ∈ Z+z , the conditional distributions are similar to that of z, such that DKL(px|z=z || px|z=z+) ≤ for small . On the other hand, we assume the set of dissimilar pronunciations Z−z contributes very little to the marginal probability of non-outlier speech samples. In other words, for any z and x where px|z(x | z) ≥ δ, we assume ∑ z−∈Z−z ∑ y−∈Yz− px|y(x | y−) py(y−) px(x).

heading: 2.2 LOCAL PRIOR MATCHING

text:
Equipped with the above assumptions, we next present a tractable approximation of the posterior py|x using unlabeled speech data. Figure 1 shows a toy example of the proposed approach. Let x be a non-outlier speech sample that is generated conditioned on a text sample y∗, whose pronunciation is f(y∗) = z∗. The posterior can be derived with Bayes’ rule as follows:
py|x(y | x) = py(y) px|y(x | y)∑
ŷ∈Y py(ŷ) px|y(x | ŷ) = py(y) px|y(x | y)∑ ẑ∈Z+
z∗∪Z − z∗
∑ ŷ∈Yẑ py(ŷ) px|y(x | ŷ) (2)
≈ py(y) px|y(x | y)∑
ẑ∈Z+ z∗
∑ ŷ∈Yẑ py(ŷ) px|y(x | ŷ) · 1∪ ẑ∈Z+ z∗ Yẑ (y) (3)
= py(y) px|z(x | f(y))∑
ẑ∈Z+ z∗
∑ ŷ∈Yẑ py(ŷ) px|z(x | ẑ) · 1∪ ẑ∈Z+ z∗ Yẑ (y). (4)
The set Y can be partitioned according to pronunciation to arrive at equation 2, which is then approximated using by remark 2 to get equation 3, where 1 is the indicator function. In equation 4, px|y(x | ŷ) is simply replaced with px|z(x | f(ŷ)) as stated in remark 1. Despite the simplification, estimating equation 4 requires a language model, a TTS model, and the enumeration of text sequences with similar pronunciations to that of z∗. To avoid the latter two requirements, we make the following approximations:
1. Replace px|z(x | ẑ) with px|z(x | z∗) for all ẑ ∈ Z+z∗ by exploiting the assumption that these distributions are very similar to each other (i.e., from remark 2 we have DKL(px|z=z∗ || px|z=ẑ) ≤ ).
2. Replace ∪ẑ∈Z+ z∗ Yẑ with hypotheses proposed from a beam search using a bootstrapped
ASR proposal model. We refer to the proposal model as ry|x and the set of hypotheses for x generated from the beam search as B(ry|x,x, k), where k is the beam size.
The approximated posterior p̄y|x(y | x) can then be written as:
p̄y|x(y | x) = py(y) px|z(x | z∗)∑
ŷ∈B(ry|x,x,k) py(ŷ) px|z(x | z ∗) · 1B(ry|x,x,k)(y) (5)
= py(y)∑
ŷ∈B(ry|x,x,k) py(ŷ) · 1B(ry|x,x,k)(y). (6)
The approximated posterior only requires computing language model probabilities of the beam search hypotheses, which is tractable. We refer to equation 6 as the local prior, since it is a renormalized distribution in the neighborhood of y∗. We also propose local prior matching (LPM) as an unsupervised objective for training an ASR model qy|x: Llpm(qy|x;x, py, ry|x, k) = DKL(p̄y|x=x||qy|x=x)
= −H(p̄y|x=x)− ∑
y∈B(ry|x,x,k)
py(y)∑ ŷ∈B(ry|x,x,k) py(ŷ) log qy|x(y | x),
which minimizes the KL-divergence between the local prior and the model distribution. For clarity, we term qy|x the online model, and let qy|x(y | x; θq) and ry|x(y | x; θr) denote the models that are parameterized by and θq and θr, respectively. Since the LPM objective simply re-weights hypotheses by the local prior, we do not need a continuous relaxation of y to make py differentiable with respect to θq as in Yang et al. (2018); Liu et al. (2019). The gradient of the proposed LPM objective is given by:
∂Llpm(qy|x;x, py, ry|x, k) ∂θq
= ∑
y∈B(ry|x,x,k)
py(y)∑ ŷ∈B(ry|x,x,k) py(ŷ) ∂ log qy|x(y | x) ∂θq .
We use paired audio and text to estimate ry|x and unpaired speech to compute the LPM objective. The LPM objective can also take advantage of large unpaired text corpora in order to obtain better language models which directly improve the quality of the approximate posterior.

heading: 2.3 PROPOSAL MODEL

text:
The quality of the posterior approximation p̄y|x depends on the proposal model ry|x. Instead of using a fixed proposal model throughout the entire training process, we update ry|x with qy|x if the latter is better.
On-policy beam search The first approach always uses the online model qy|x as the proposal model. This means θr = θq and is effectively a form of on-policy beam search, since the model used to generate hypotheses is also the model we update.
Off-policy beam search While the on-policy method benefits from the immediate improvement of the online model, it also suffers immediately if the gradient update from a mini-batch deteriorates performance. This can result in instability during optimization. We consider a second option which does not tie θr and θq but instead updates θr with θq every T steps only when the performance of the online model qy|x is better than that of the proposal model ry|x by some metric. We refer to the second option as off-policy beam search. To avoid overfitting to the training set, we use the character error rate (CER) on the validation set as the metric for the proposal model update. We set T = 1000 for all experiments with the off-policy beam search.

heading: 2.4 FILTERING HYPOTHESES USING ESTIMATED LENGTHS

text:
As noted in Chorowski & Jaitly (2017), sequence-to-sequence ASR models sometimes predict endof-sentence (EOS) tokens too early or generate looping n-grams, resulting in hypotheses that are significantly shorter or longer than the set of acoustically matched texts for a given utterance. Of the two failure modes, the former is more harmful to training with LPM. The reason is that the LPM objective assumes all hypotheses obtained from beam search are acoustically reasonable, and weights each of them by linguistic plausibility given by an LM. While LMs are effective in discriminating plausibility between sentences of similar lengths, we find empirically they tend to assign higher probabilities to shorter sentences than to longer sentences, even when the longer ones are more plausible and grammatically correct than the shorter ones. As a result, truncated hypotheses are assigned higher weights than those that are acoustically matched but longer, which in turn encourages earlier prediction of EOS tokens and forms a catastrophic feedback loop particularly with the on-policy beam search.
To address this issue, we propose a simple filtering heuristic based on the text length. Before training the model, a text length L is estimated for each unlabeled speech sample x. During training, only hypotheses with length close to L are retained for the LPM objective computation. Let len(y) denote the length of y. We keep a hypothesis y only if brlb · Lc ≤ len(y) ≤ drub · Le, where rlb and rub are the text length lower and upper bound ratios, respectively. Several methods can be used to estimate the text length on an unlabeled utterance, including using the average speaking rate (Peng et al., 2019) or generating a phoneme/syllable segmentation (Adell & Bonafonte, 2004; Scharenborg et al., 2010; Wang et al., 2017). In this work, we estimate the length by using using that of the best hypothesis generated from the initial proposal model, generated with either ASR-only greedy decoding or ASR+LM beam search decoding.

heading: 3 EXPERIMENTAL SETUP

text:
Dataset We evaluate our approach on LibriSpeech (Panayotov et al., 2015), a crowd-sourced audio book corpus derived from the LibriVox Project. The training set contains 960 hours of speech, officially split into three sets: train-clean-100, train-clean-360, and train-other-500, where the first two sets are easier and the third set is noisier and more accented. Similarly, the development and test sets are also split according to difficulty, resulting in four partitions: {dev, test}×{clean, other}, each of which contains roughly five hours of speech. In this work we use train-clean-100 as the paired speech data, and the other two training splits as the unpaired data.
We train the language model on the unpaired text data provided with LibriSpeech, which includes approximately 14,500 books collected from Project Gutenberg. Some of the books in the text corpus overlap with those in the LibriSpeech training set. To avoid training the LM on the ground truth text of the unlabeled speech, we exclude the 997 overlapping books from the text data. We follow the same recipe as Kahn et al. (2019) to pre-process the remaining text.
Neural Network Architecture The proposal model ry|x and the online model qy|x are sequenceto-sequence neural networks (Bahdanau et al., 2016; Chorowski & Jaitly, 2017) with the same timedepth separable (TDS) architecture proposed in Hannun et al. (2019). The encoder is fully convolutional, composed of TDS blocks which reduce the number of parameters while keeping the receptive field large. The decoder is a single layer recurrent neural network (RNN) with gated recurrent units (GRUs), equipped with a single-headed inner-product key-value attention (Vaswani et al., 2017) for querying information from the encoder outputs. We follow the recipe of Kahn et al. (2019) which uses fewer TDS blocks in the encoder compared to Hannun et al. (2019) in order to generalize better when trained on the smaller LibriSpeech train-clean-100. The output target of the decoder at each step is a posterior distribution over 5,000 word pieces, generated with the SentencePiece toolkit (Kudo & Richardson, 2018) using transcripts from train-clean-100.
To enable efficient evaluation of the language model probabilities, which is required at each training step, we use the gated convolutional language model architecture (ConvLM) proposed in Dauphin et al. (2017), which achieves competitive performances compared to recurrent models while significantly reducing the latency. We use the same 5,000 word-piece vocabulary for the LM which is trained with the same model configuration and recipe as Zeghidour et al. (2018). The trained ConvLM achieves a token perplexity of 34.24 on the development set.
Optimization We use both paired and unpaired data to optimize qy|x. To simplify the optimization procedure, the model is provided with either a paired or an unpaired batch at each step, alternated with a fixed ratio ml : mu. When given a paired batch of n samples, {(x(i),y(i))}ni=1, the model minimizes the standard cross-entropy loss, 1n ∑ i− log qy|x(y(i) | x(i)). When provided with an unpaired batch {x(i)}ni=1, the model minimizes a weighted LPM loss, αn ·∑ i Llpm(qy|x;x(i), py, ry|x, k). The weight α and the mixing ratio ml/mu are used to balance the supervised and self-training objectives. For regularization we use 20% dropout (Srivastava et al., 2014), 10% label smoothing, 1% decoder input sampling, and 1% word piece sampling (Kudo, 2018) following Kahn et al. (2019). We use SGD without momentum to train the online model with an initial learning rate of 5e-2. To achieve a good CER on the development sets, the model is trained for at least 1.6M steps (paired and unpaired ones combined) with a batch size of 16 (8 GPUs × 2 per GPU). The learning rate is annealed by a factor of two every 0.64M steps. All experiments in this paper are implemented in the wav2letter++ framework (Pratap et al., 2018).
Initialization To initialize the proposal model and the online model, we consider three checkpoints from a baseline model trained on train-clean-100 for a varying number of steps using only the supervised objective. The three checkpoints, whose parameters are denoted as θA, θB , and θC , are trained for about 300k / 40k / 16k steps, achieving average development set CERs of 13% / 20% / 38%, respectively. To obtain a better approximation p̄y|x for the posterior, it is desirable to use the best-performed checkpoint to initialize the proposal model. By contrast, as observed in Kahn et al. (2019) that training from scratch achieves consistently better performances than starting from a well-trained model, we hypothesize that initializing the online model with an earlier checkpoint may also have a similar effect. Hence, we initialize θr = θA and θq = θC if not otherwise specified.

heading: 4 RESULTS

text: The best supervised model trained only on train-clean-100 (θA) achieves a WER of 14.00%/37.02% on dev-clean/dev-other, respectively. Unless otherwise stated, we use train-clean-360 as the unpaired speech dataset, (rlb, rub) = (0.95, 1.05) for length filtering with reference lengths obtained from ASR-only greedy decoding.

heading: 4.1 BEAM SIZE, MIXING RATIO, AND LPM WEIGHTS

text:
Table 1 shows how the WER varies with the beam size and the mixing ratio. for all mixing ratios, the model improves the most from beam size of k = 1 to k = 2, showing the benefit of considering multiple hypotheses. The improvement is greater when a higher mixing ratio of unpaired-to-paired speech is used. In addition, we note that the LM is effectively unused when k = 1 because the LM probability assigned to each hypothesis is normalized within the beam. If there is only one hypothesis, it will be assigned an approximate posterior probability of one. The amount of improvement
diminishes with larger beam sizes, and the performance even starts to degrade beyond k = 4 when using a higher mixing ratio. This may result from the inclusion of worse hypotheses which have a better scored under the LM. We use the best setting for the following experiments with a mixing ratio rl : ru = 1 : 4, a beam size k = 4, and an LPM weight α = 0.2. We present detailed results varying the LPM weight in the Appendix.

heading: 4.2 PROPOSAL MODEL UPDATE AND MODEL INITIALIZATION

text:
In addition to the two proposal model update strategies proposed in Section 2.3, termed on-policy and off-policy (better), we experiment with two additional strategies. The first, off-policy (never), uses a fixed proposal model throughout training without ever updating. The second, off-policy (always), updates the proposal model with the online model every T steps (i.e., set θr ← θq) regardless of the performance.
The full results are shown in Table 2. Four key takeaways are as follows. (1) For all combinations of (ry|x, qy|x) initialization, off-policy (never) is the worst. This demonstrates the importance of updating the proposal model to generate better hypotheses during training. (2) Off-policy (always) consistently outperforms on-policy. We observe that training is significantly stabilized by reducing the proposal model update frequency from every step to every 1,000 steps. The effect is particularly prominent when initializing ry|x and qy|x from an earlier checkpoint (9.62% vs 20.02% on dev-clean, and 27.51% vs 45.62% on dev-other). (3) Off-policy (better) achieves the best WER in all settings and outperforms off-policy (always) by a larger margin when initializing from an earlier checkpoint. (4) Unlike the other strategies, off-policy (better) demonstrates consistent improvement when using a less-trained initial online model. In the following experiments, we initialize models with θr = θA and θq = θC unless otherwise specified.

heading: 4.3 LENGTH FILTERING

text:
Table 3 shows the impact of length filtering with different proposal model update strategies, where both the proposal model and the online model are initialized with θA. As discussed in Section 2.4, on-policy suffers more than off-policy without length filtering. If the proposal model is never updated, then length filtering does not affect the final WER. We hypothesize that length filtering keeps the proposal model stable during training. Table 4 compares three different reference lengths: the oracle length (Oracle), the predicted length from ASR + LM beam search decoding (ASR + LM Dec), and the predicted length from ASR-only greedy decoding (ASR-only Dec). We observe that the WER does not differ much when using different reference lengths for filtering.
Table 4: Length filtering using different reference lengths.
Reference Length L dev-{clean / other} Oracle 8.85 / 26.39 ASR + LM Dec 8.99 / 26.36 ASR-only Dec 9.00 / 26.47

heading: 4.4 CHOICE OF LANGUAGE MODELS

text:
We study how the LM affects the performance by experimenting with three alternative hypothesis weighting methods. The first method uses the LM score normalized by token length, py(y)1/len(y), which avoids favoring overly short hypotheses. The second method computes the LM probability of each hypothesis and shuffles them, which preserves the entropy of the original approximate posterior but randomizes the weights. The third assigns an equal weight for each hypothesis. Table 5 presents the results with three beam sizes: k = 2, 4, 8. Using the LM weights directly performs the best for all beam sizes. Length normalization (LenNorm) is better than shuffled and uniform, which have similar performance. This suggests that both ranking hypotheses correctly and scaling the weights proportional to the LM probability are important.
We also observe how the quality of the LM affects the results. We test LMs of various quality by using the same ConvLM but trained for a different number of steps. We quantify LM quality by the token perplexity (PPL) on the development set. Table 6 shows a clear positive correlation between the LM quality and the final WER. This is expected given that the better LM results in a more accurate posterior approximation p̄y|x.

heading: 4.5 FINAL RESULTS AND COMPARISON WITH LITERATURE

text:
The best performing model is trained for 3.2M steps, with a learning rate annealed by a factor of two every 1.28M steps when using 360 hours of unpaired speech, and every 0.64M steps when using 860 hours of unpaired speech. Reference lengths for filtering are obtained from ASR+LM beam search decoding. We compare LPM to fully supervised models and a pseudo labeling (PL) method in Table 7. Compared to LPM, PL demands extensive hyper-parameter tuning for the ASR+LM beam search as well as heuristic data filtering methods. to obtain high quality pseudo transcripts. We
chose PL as a baseline because we follow the same experimental setup, and, more importantly, it achieves the state-of-the-art results on LibriSpeech when using train-clean-100 as paired data and train-other-360 as unpaired speech. A more complete table with results from the literature is in the Appendix.
The upper half of Table 7 shows greedy decoding results without an LM. For the fully supervised model, when removing train-clean-360 the WER increases by 6.86% on test-clean and 13.36% on test-other. Using train-clean-360 speech without transcripts, LPM reduces the absolute WER by 5.64% and 12.21% on the two test sets, which recovers 82% and 91%, respectively, of the WER drop from removing the labels. Adding noisier train-other-500 to the unpaired set (total 860hr Dsu) further reduces the WER, and LPM achieves a better WER on the noisy sets (dev-other and testother) compared to the supervised model trained on 460 hours of clean paired data. In addition, LPM outperforms PL in all settings. This trend is consistent even when decoding with a strong ConvLM.

heading: 5 ANALYSIS

heading: 5.1 HYPOTHESIS QUALITY OF UNLABELED TRAINING SPEECH

text: As discussed in Section 4.2, updating the proposal model is crucial to improve the hypotheses used during training. To quantify the improvement, Table 8 shows WERs on the unlabeled data of an LPM model at the beginning (step=0) and at the end (step=3.2M) of training. Note that this is a proxy of quality for LPM, since multiple hypotheses are used when setting k ≥ 2. We compare to the WER of the pseudo transcripts used in PL. Although generating hypotheses without an LM leads to worse WER than that of PL, toward the end of training, the proposal model of LPM produces better predictions on both train-clean-360 and train-other-500 than the fixed ones used in PL (ASR+LM stable). Furthermore, the WER on train-other-500 much higher for PL (21.51%) than for LPM at the end of training (13.00%), which explains why LPM achieves much better performances than PL when using the full 860hr of unpaired data.

heading: 5.2 LINGUISTIC PLAUSIBILITY

text:
We expect models trained with LPM to generate more semantically and grammatically correct text since the ASR model receives direct supervision from the LM. Table 9 shows the proposed hypotheses for two utterances using a supervised baseline model and a model trained with LPM. The baseline model proposes erroneous hypotheses which are easy to discard even without the audio or the ground truth transcript. On the other hand, LPM generates hypotheses that are both grammatically and semantically plausible, with acceptable substitution errors in some cases (e.g., might/would).
We also notice in Table 9 that the LM probabilities correlate well with linguistic plausibility for texts of similar lengths (e.g., hypotheses of the same utterance), but not for texts with large differences in length. Motivated by this observation, we propose to quantify linguistic plausibility of an ASR model by measuring the LM token perplexity exp( ∑ y log py(y)/ ∑ y len(y)) of the hypotheses on the development set obtained using ASR-only greedy decoding. Results are shown in Table 10. The ground truth text has the lowest perplexity on both sets as expected. We also observe that while all models are worse on dev-other than on dev-clean, LPM exhibits the smallest perplexity difference between the two sets, demonstrating how it successfully distills knowledge from the LM.

heading: 6 RELATED WORK

text:
Our work builds on a large body of work in semi-supervised learning for ASR. Research in this direction can be classified based on the required modules and objectives used with unpaired data.
In Drexler & Glass (2018) and Karita et al. (2018), bi-encoder network architectures are used, which map text and speech to representations in a shared space with their corresponding encoder, and then apply a shared decoder to map from the shared space to the text space. Another line of work adds a TTS model (Tjandra et al., 2017; 2019; Baskar et al., 2019) or a text-to-encoding (TTE) model (Hayashi et al., 2018; Hori et al., 2019) in the loop of ASR training, which can be utilized for back-translation style data augmentation (Sennrich et al., 2016) or cycle-consistency training (Zhu et al., 2017). Liu et al. (2019) treats ASR as a generative model that conditions on speech instead of random noise vectors, and adopts the generative adversarial network (GAN) (Goodfellow et al., 2014) framework in order to improve the fidelity of ASR-generated texts (Liu et al., 2019). All the aforementioned methods involve additional modules that must be jointly optimized with the ASR model and require finding a careful balance between multiple training objectives. Moreover, some methods require gradient to flow through samples drawn from the ASR outputs, and hence rely on techniques like Gumbel-softmax (Jang et al., 2017), straight-through estimator (Bengio et al., 2013), or REINFORCE (Williams, 1992). The LPM objective can evaluate output quality yet is fully differentiable.
Knowledge distillation is also easy to used for semi-supervised learning (Cui et al., 2017; Li et al., 2019; Parthasarathi & Strom, 2019) because the teacher model can provide a training target for the unlabeled speech. Rather than learning from a teacher ASR model, LPM learns from a teacher LM.
Pseudo labelling (Veselỳ et al., 2017; Manohar et al., 2018; Kahn et al., 2019) is most similar to our approach. Pseudo labelling uses a seed ASR model to transcribe unlabeled speech. The ASR model is then trained on the predicted labels along with the original labeled data used to train the seed model. In addition to being well motivated theoretically, we highlight the key methodological advantages of LPM below. (1) While previous work uses fixed pseudo transcripts throughout training and focuses on data filtering, we demonstrate the benefit of generating hypotheses on-the-fly as the proposal model improves. This is efficient since we only use an end-to-end ASR model for beam search decoding. (2) We motivate and show empirically the benefit from considering multiple hypotheses rather than just the top hypothesis (Veselỳ et al., 2017; Kahn et al., 2019). (3) Manohar et al. (2018) consider multiple hypotheses in a numerator lattice and maximize the aggregated probability of the lattice regardless as to how the probability is allocated amongst the word sequences. In contrast, we verify in the LM ablation study that considering the allocation of the posterior probability amongst the hypotheses is important.
Aside from semi-supervised learning, this work is also related to unsupervised domain adaptation for ASR, where unlabeled speech of the target domain is provided. Unlike the proposed method, previous studies mainly focus on learning domain invariant features (Sun et al., 2017; Hsu & Glass, 2018; Meng et al., 2017; 2018; 2019) or data augmentation through learned speech transformation (Hsu et al., 2017; 2018).

heading: 7 CONCLUSION AND FUTURE WORK

text:
We introduce local prior matching, a self-supervised learning objective for speech recognition, and demonstrate note-able reductions in WER when with the addition of unpaired audio and text. We also perform an extensive empirical study to demonstrate the importance of various configurations of LPM. While LPM is motivated by how humans learn to recognize speech, the proposed method can be applied to other sequence transduction tasks including machine translation (Sennrich et al., 2016) and text summarization (Nallapati et al., 2016), provided a good prior for the domain.
We consider two promising directions for future in self-supervised speech recognition with LPM and in general: (1) Empower the model with more capabilities, such as learning to select whether or not to use a sample for self-supervision. (2) Using more context signals for prior estimation which can consider previous sentences as well as inputs from other modalities.

heading: A MORE COMPREHENSIVE COMPARISON WITH THE LITERATURE

heading: C CHARACTER ERROR RATES

heading: B ADDITIONAL RESULTS


summaries: This paper introduces a new method for training a classifier that simultaneously optimizes for a fairness criterion and robustness to data poisoning. The method is shown to increase measures of fairness and reduce inaccuracy on poisoned data relative to classifiers that only consider accuracy or fairness. Extensive results are shown for both synthetic and real benchmark data sets.

summaries: This paper combines adversarial fair training with adversarial robust training. The basic idea is that a classifier is combined with two adversaries: one tries to predict the sensitive attribute $Z$ from the output of the classifier (essentially the approach by Edwards&Storkey 2015) and the other adversary tries to recognize if a label was predicted or is from a clean hold-out dataset. The latter is intended to harden the classifier against data-poisoning of the training set.

abstractText: We consider the problem of fair and robust model training in the presence of data poisoning. Ensuring fairness usually involves a tradeoff against accuracy, so if the data poisoning is mistakenly viewed as additional bias to be fixed, the accuracy will be sacrificed even more. We demonstrate that this phenomenon indeed holds for state-of-the-art model fairness techniques. We then propose FR-GAN, which holistically performs fair and robust model training using generative adversarial networks (GANs). We first use a generator that attempts to classify examples as accurately as possible. In addition, we deploy two discriminators: (1) a fairness discriminator that predicts the sensitive attribute from classification results and (2) a robustness discriminator that distinguishes examples and predictions from a clean validation set. Our framework respects all the prominent fairness measures: disparate impact, equalized odds, and equal opportunity. Also, FR-GAN optimizes fairness without requiring the knowledge of prior statistics of the sensitive attributes. In our experiments, FR-GAN shows almost no decrease in fairness and accuracy in the presence of data poisoning unlike other state-of-the-art fairness methods, which are vulnerable. In addition, FR-GAN can be adjusted using parameters to maintain reasonable accuracy and fairness even if the validation set is too small or unavailable.

authors:
Alekh Agarwal
Alina Beygelzimer
Miroslav Dudı́k
John Langford
Hanna M. Wallach

title: A reductions approach to fair classification

venue: In ICML,

authors:
J. Angwin
J. Larson
S. Mattu
L. Kirchner

title: Machine bias: There’s software used across the country to predict future criminals

venue: And its biased against blacks.,

authors:
Solon Barocas
Selbst

title: Big Data’s Disparate Impact

venue: California Law Review,

authors:
Rachel K.E. Bellamy
Kuntal Dey
Michael Hind
Samuel C. Hoffman
Stephanie Houde
Kalapriya Kannan
Pranay Lohia
Jacquelyn Martino
Sameep Mehta
Aleksandra Mojsilovic
Seema Nagar
Karthikeyan Natesan Ramamurthy
John T. Richards
Diptikalyan Saha
Prasanna Sattigeri
Moninder Singh
Kush R. Varshney
Yunfeng Zhang

title: AI fairness 360: An extensible toolkit for detecting, understanding, and mitigating unwanted algorithmic bias

authors:
Battista Biggio
Blaine Nelson
Pavel Laskov

title: Support vector machines under adversarial label noise

venue: In ACML, pp

authors:
Battista Biggio
Igino Corona
Davide Maiorca
Blaine Nelson
Nedim Srndic
Pavel Laskov
Giorgio Giacinto
Fabio Roli

title: Evasion attacks against machine learning at test time

venue: In ECML PKDD,

authors:
Alexandra Chouldechova
Aaron Roth

title: The frontiers of fairness in machine

venue: learning. CoRR,

authors:
Andrew Cotter
Maya R. Gupta
Heinrich Jiang
Nathan Srebro
Karthik Sridharan
Serena Wang
Blake E. Woodworth
Seungil You

title: Training well-generalizing classifiers for fairness metrics and other data-dependent constraints

authors:
Andrew Cotter
Heinrich Jiang
Karthik Sridharan

title: Two-player games for efficient non-convex constrained optimization

venue: In ALT,

authors:
Flávio du Pin Calmon
Dennis Wei
Bhanukiran Vinzamuri
Karthikeyan Natesan Ramamurthy
Kush R. Varshney

title: Optimized pre-processing for discrimination prevention

venue: In NeurIPS,

authors:
Cynthia Dwork
Moritz Hardt
Toniann Pitassi
Omer Reingold
Richard Zemel

title: Fairness through awareness

venue: In Proceedings of the 3rd Innovations in Theoretical Computer Science Conference,

authors:
Michael Feldman
Sorelle A. Friedler
John Moeller
Carlos Scheidegger
Suresh Venkatasubramanian

title: Certifying and removing disparate impact

venue: In KDD, pp

authors:
Benoı̂t Frénay
Michel Verleysen

title: Classification in the presence of label noise: A survey

venue: IEEE Trans. Neural Netw. Learning Syst.,

authors:
Sahaj Garg
Vincent Perot
Nicole Limtiaco
Ankur Taly
Ed Huai hsin Chi
Alex Beutel

title: Counterfactual fairness in text classification through robustness

venue: AIES,

authors:
Ian J. Goodfellow
Jean Pouget-Abadie
Mehdi Mirza
Bing Xu
David Warde-Farley
Sherjil Ozair
Aaron C. Courville
Yoshua Bengio

title: Generative adversarial nets

authors:
Ian J. Goodfellow
Jonathon Shlens
Christian Szegedy

title: Explaining and harnessing adversarial examples

venue: In ICLR,

authors:
Alon Halevy
Flip Korn
Natalya F. Noy
Christopher Olston
Neoklis Polyzotis
Sudip Roy
Steven Euijong Whang

title: Goods: Organizing google’s

authors:
Moritz Hardt
Eric Price
Nati Srebro

title: Equality of opportunity in supervised learning

authors:
Dan Hendrycks
Mantas Mazeika
Duncan Wilson
Kevin Gimpel

title: Using trusted data to train deep networks on labels corrupted by severe noise

authors:
Heinrich Jiang
Ofir Nachum

title: Identifying and correcting label bias in machine learning

authors:
Christopher Jung
Michael J. Kearns
Seth Neel
Aaron Roth
Logan Stapleton
Zhiwei Steven Wu

title: Eliciting and enforcing subjective individual

venue: fairness. CoRR,

authors:
Faisal Kamiran
Toon Calders

title: Data preprocessing techniques for classification without discrimination

venue: Knowl. Inf. Syst.,

authors:
Faisal Kamiran
Asim Karim
Xiangliang Zhang

title: Decision theory for discrimination-aware classification

venue: In ICDM, pp

authors:
Toshihiro Kamishima
Shotaro Akaho
Hideki Asoh
Jun Sakuma

title: Fairness-aware classifier with prejudice remover regularizer

venue: In ECML PKDD, pp

authors:
Michael Kearns
Aaron Roth
Saeed Sharifi-Malvajerdi

title: Average individual fairness: Algorithms, generalization and experiments, 2019

authors:
Diederik P Kingma
Jimmy Ba

title: Adam: A method for stochastic optimization

venue: arXiv preprint arXiv:1412.6980,

authors:
Jon M. Kleinberg
Sendhil Mullainathan
Manish Raghavan

title: Inherent trade-offs in the fair determination of risk scores

venue: In ITCS,

authors:
Pang Wei Koh
Jacob Steinhardt
Percy Liang

title: Stronger data poisoning attacks break data sanitization

venue: defenses. CoRR,

title: Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid

authors:
Jakub Konečnỳ
H Brendan McMahan
Felix X Yu
Peter Richtárik
Ananda Theertha Suresh
Dave Bacon

title: Federated learning: Strategies for improving communication efficiency

venue: arXiv preprint arXiv:1610.05492,

authors:
Alexey Kurakin
Ian J. Goodfellow
Samy Bengio

title: Adversarial machine learning at scale

authors:
Yuncheng Li
Jianchao Yang
Yale Song
Liangliang Cao
Jiebo Luo
Li-Jia Li

title: Learning from noisy labels with distillation

venue: In ICCV,

title: Divergence measures based on the Shannon entropy

venue: IEEE Transactions on Information theory,

authors:
Nagarajan Natarajan
Inderjit S. Dhillon
Pradeep Ravikumar
Ambuj Tewari

title: Learning with noisy labels

venue: In NeurIPS, pp

authors:
Natasha Noy
Matthew Burgess
Dan Brickley

title: Google dataset search: Building a search engine for datasets in an open web ecosystem

venue: In 28th Web Conference (WebConf

authors:
Adam Paszke
Sam Gross
Soumith Chintala
Gregory Chanan
Edward Yang
Zachary DeVito
Zeming Lin
Alban Desmaison
Luca Antiga
Adam Lerer

title: Automatic differentiation in PyTorch

venue: In NIPS Autodiff Workshop,

authors:
Andrea Paudice
Luis Muñoz-González
Emil C. Lupu

title: Label sanitization against label flipping poisoning attacks

authors:
Geoff Pleiss
Manish Raghavan
Felix Wu
Jon M. Kleinberg
Kilian Q. Weinberger

title: On fairness and calibration

authors:
Mengye Ren
Wenyuan Zeng
Bin Yang
Raquel Urtasun

title: Learning to reweight examples for robust deep learning

authors:
Andreas Veit
Neil Alldrin
Gal Chechik
Ivan Krasin
Abhinav Gupta
Serge J. Belongie

title: Learning from noisy large-scale datasets with minimal supervision

venue: In CVPR,

title: Algorithmic fairness: Measures, methods and representations

venue: In PODS,

authors:
Sahil Verma
Julia Rubin

title: Fairness definitions explained

venue: In FairWare@ICSE,

authors:
Eric Wong
J. Zico Kolter

title: Provable defenses against adversarial examples via the convex outer adversarial polytope

authors:
Tong Xiao
Tian Xia
Yi Yang
Chang Huang
Xiaogang Wang

title: Learning from massive noisy labeled data for image classification

authors:
Mikhail Yurochkin
Amanda Bower
Yuekai Sun

title: Learning fair predictors with sensitive subspace robustness

authors:
Muhammad Bilal Zafar
Isabel Valera
Manuel Gomez-Rodriguez
Krishna P. Gummadi

title: Fairness constraints: Mechanisms for fair classification

venue: In AISTATS,

authors:
Richard S. Zemel
Yu Wu
Kevin Swersky
Toniann Pitassi
Cynthia Dwork

title: Learning fair representations

venue: In ICML, pp

authors:
Brian Hu Zhang
Blake Lemoine
Margaret Mitchell

title: Mitigating unwanted biases with adversarial learning

venue: In AIES,

heading: 1 INTRODUCTION

text:
As machine learning becomes widespread in the Software 2.0 era (Karpathy, 2017), there is an urgent need to address the issues of fairness and robustness. For sensitive applications like healthcare, finance, and self-driving cars, a trained model must not discriminate customers based on sensitive attributes including age, sex, or religion. In addition, as applications often rely on external data sets for their training data, the model training must be resilient against noisy or even adversarial data.
Traditionally, model fairness research (Venkatasubramanian, 2019; Chouldechova & Roth, 2018; Verma & Rubin, 2018) has focused on developing metrics such as disparate impact (Feldman et al., 2015), equalized odds (Hardt et al., 2016), and equal opportunity (Hardt et al., 2016), which capture various notions of discrimination. More recently, there has been a surge in unfairness mitigation techniques (Bellamy et al., 2018), which improve the model fairness by either fixing the training data, training process, or trained model. Unfairness mitigation usually involves some tradeoff between the model’s accuracy and fairness. Most recently, generative adversarial networks (GANs) are being adapted to a fairness setting (Zhang et al., 2018). The architecture of GANs is suitable because it can be used to compete the training between two objectives: accuracy and fairness.
Robust model training is also important and needs to be concurrently taken into consideration. There has been a proliferation of algorithms that make model training resilient against noisy and even adversarial data (Natarajan et al., 2013; Biggio et al., 2011; Frénay & Verleysen, 2014). The latter scenario may occur when the source of the training data is external, and we cannot control the data that is being added. For example, any public dataset on the Web may contain intentionallyfalse information. Moreover, one must have a robust training algorithm if the data set comes from diverse users’ inputs that cannot be fully trusted (Konečnỳ et al., 2016). Recently, data poisoning attacks have become increasingly sophisticated where it is becoming difficult to sanitize the data against all of them (Koh et al., 2018). In addition, dataset searching is becoming mainstream as demonstrated by Google Dataset Search (Goods) (Halevy et al., 2016) and its public version for searching scientific datasets (Noy et al., 2019). While data lakes within companies may consist of refined datasets, datasets in the public are easy to poison. And anyone can poison public datasets
using attacks in the literature and share them. We thus believe that it is essential to address both bias and poisoning as a preventive measure.
Solving model fairness without addressing data poisoning may lead to a worse tradeoff between accuracy and fairness. For example, consider a banking system that is giving out loans where there are two sensitive groups: men and women. According to disparate impact, the two groups must have similar ratios of positive predictions. A perfect disparate impact would have a value of 1. Suppose the clean training data has no bias and a perfect (or high) disparate impact score. Now suppose this data is poisoned where half of the positive labels of men are flipped to negative labels. If the model is fitted so as to make more negative predictions on men than women, then disparate impact would worsen. To address this problem, one may suggest using data sanitization techniques prior to the model training, but it is challenging to do so without any knowledge of the model. Even though many data sanitization methods are proposed, there have been newer attacks that can easily evade such defenses (Koh et al., 2018).
Our main contribution is the development of a new framework called FR-GAN, which trains accurate models that are also fair and robust to poisoning. FR-GAN consists of a generator used for classification, a discriminator that distinguishes predictions from one sensitive group against others, and a second discriminator that distinguishes {examples, predictions} of the training data from {examples, labels} of a separate and clean validation set. The first discriminator ensures that the prediction ŷ is independent of the sensitive attribute z. We show that variations of this approach can be used to maximize any of the following prominent fairness measures: disparate impact, equalized odds, and equal opportunity. The second discriminator ensures that the model predictions on the training data are “consistent” with labels on clean data. We theoretically show that FR-GAN optimizes fairness without requiring the knowledge of prior statistics of sensitive attributes. In addition, the parameters of FR-GAN can be adjusted to maintain reasonable accuracy and fairness even if the validation set is too small or unavailable. In the following sections, we present the related work, demonstrate the weaknesses of current fairness methods, and propose FR-GAN.

heading: 2 RELATED WORK

text:
Model fairness The notion of discrimination has many definitions and usually comes from certain social goals that one wants to guarantee. Fairness research has been focused on defining fairness measures that capture a variety of notions of fairness (Verma & Rubin, 2018). There are largely two types of fairness measures: (1) group fairness (Barocas & Selbst, 2016; Hardt et al., 2016), which ensures similar statistics between two sensitive groups; (2) individual fairness (Dwork et al., 2012), which guarantees similar prediction results across nearby examples. Group fairness measures are widely studied in general applications, even though these cannot be simultaneously satisfied in certain circumstances (Kleinberg et al., 2017). In addition, there are also recent significant works that focus on individual fairness (Garg et al., 2018; Yurochkin et al., 2019; Kearns et al., 2019; Jung et al., 2019). Among the two types of fairness with different aspects, we focus on group fairness as our initial research. Recently, there has also been a surge of research on unfairness mitigation techniques (Bellamy et al., 2018). Depending on where a fix occurs, there are mainly three approaches: (1) pre-processing techniques (Kamiran & Calders, 2011; du Pin Calmon et al., 2017; Zemel et al., 2013; Feldman et al., 2015) that fix the training data; (2) in-processing techniques (Zafar et al., 2017; Jiang & Nachum, 2019; Zhang et al., 2018; Kamishima et al., 2012; Cotter et al., 2019; 2018; Agarwal et al., 2018) that address the issue during model training; and (3) post-processing techniques (Hardt et al., 2016; Pleiss et al., 2017; Kamiran et al., 2012) that manipulate predictions while maintaining the model.
Among the three, the in-processing techniques have the advantages that one can work with any data and that there is more control on model training (Venkatasubramanian, 2019). FAIRNESS CONSTRAINTS (Zafar et al., 2017) incorporates a regularization term that reflects fairness constraints in the context of convex margin-based classifiers such as logistic regression and support vector machines (SVMs). The main hyperparameter is the λ term, which serves to balance the accuracy and fairness objectives. LABEL BIAS CORRECTION (Jiang & Nachum, 2019) assumes the existence of true yet possibly biased labels. A reweighting algorithm is proposed with theoretical guarantees that training on the resulting loss corresponds to training on the true unbiased labels, which yields a fair model. Here the hyperparameters include the number of training iterations. Finally, ADVERSARIAL
DEBIASING (Zhang et al., 2018) trains a classifier so that an adversary cannot predict the sensitive attributes based on the output of the classifier. Hence, the classifier achieves fairness when its predictions do not have any correlation with sensitive attributes. This idea of adversarial training is similar to GANs (Goodfellow et al., 2014). ADVERSARIAL DEBIASING provides insights of using adversarial training to minimize mutual information between the prediction and sensitive attributes. In this paper, we strengthen the theoretical results of ADVERSARIAL DEBIASING using information theory, and this motivates us to provide systematic methodology for various fairness metrics (Section 4.1).
As we demonstrate in Section 3, the existing techniques are not tailored for robust training, so are vulnerable to data poisoning attacks. In comparison, FR-GAN addresses both model fairness and robust training within the same model training process.
Robust training There is a heavy literature on how to make the model training robust against noisy or even adversarial data (Natarajan et al., 2013; Biggio et al., 2011; Frénay & Verleysen, 2014; Kurakin et al., 2017). A major challenge is that there can be a wide range of data poisoning attacks that keep on evolving. One defense approach is to sanitize the training data and thus fix the root cause. A fundamental limitation is that we do not know in general what kind of poisoning will occur, so a universal defense against all possible attacks does not seem feasible as demonstrated by Koh et al. (2018). A more recent trend is to develop general defense algorithms for any attack during model training using meta learning (Veit et al., 2017; Li et al., 2017; Xiao et al., 2015; Hendrycks et al., 2018). Ren et al. (2018) propose a meta-learning framework that employs a clean validation set for the purpose of preventing the model training from being influenced by poisoned data. By using the validation loss as a meta objective, it intends to defend against any poisoning attack. Our FR-GAN framework is inspired by this approach, but employs a GAN-based model to support fair and robust training without using meta learning. In particular, the design of FR-GAN’s robustness discriminator is based on mutual-information-based theoretical insights (Section 4.2). Another line of research is defending against adversarial attacks during test time (Biggio et al., 2013; Goodfellow et al., 2015; Wong & Kolter, 2018). In comparison, our focus is on defending against data poisoning on the training data.

heading: 3 VULNERABILITY OF FAIRNESS METHODS

text:
We perform experiments to demonstrate that state-of-the-art fairness methods are indeed vulnerable even to simple poisoning attacks. We generate a synthetic dataset as shown in Figure 1a (see the generation details in Section 5.1). There are two non-sensitive attributes a and b, which are reflected in the x-axis and y-axis, respectively. The examples are further divided into two classes based on the sensitive attribute z. To generate poisoned data, we measure accuracy degradation for a flipping of each z. We then choose the top-10% of the z values with the highest degradation and flip them. This approach is similar to the existing label flipping method (Paudice et al., 2018), except that the
z value is being flipped instead of the label so that the model fairness is also affected. To make a validation set, we randomly select clean examples that amount to 10% of the entire training data.
We use disparate impact (Feldman et al., 2015) as a fairness measure and evaluate one fairness method, called FAIRNESS CONSTRAINTS (Zafar et al., 2017). As this method involves a regularization factor λ that balances the accuracy and fairness objectives, we can obtain a tradeoff curve by adjusting its value. Figure 1b shows two accuracy-fairness tradeoff curves obtained with the clean and poisoned synthetic datasets. Notice that adding data poisoning clearly shifts the curve to the left, which means both accuracy and fairness decrease. This coincides with our intuition. The poisoning confuses the model so that there are more biased examples to fix, which in turn makes it overreact and thus sacrifice more on accuracy. In the supplementary, we show similar trends when the data is poisoned via label flipping. In Section 5, we will show how data poisoning affects LABEL BIAS CORRECTION (Jiang & Nachum, 2019) and ADVERSARIAL DEBIASING (Zhang et al., 2018).

heading: 4 FR-GAN

text: We now describe FR-GAN, which is shown in Figure 2. Unlike traditional GANs, the generator is a classifier that receives an example x ∈ X and returns a prediction ŷ. The other two discriminators optimize fairness and robustness as we describe in the following sections.

heading: 4.1 FAIRNESS

text:
We denote by Dtr the training data set. Suppose Dtr has m examples {(x(i), z(i), y(i))}mi=1 where x(i) contains the non-sensitive attributes, z(i) contains the sensitive attributes, and y(i) is the label. Both the sensitive attribute and label can be multi-class, i.e., they can have one of multiple values. For notational simplicity, we assume there is one sensitive attribute, which can be viewed as a merged result of multiple sensitive attributes with a larger alphabet size. For illustrative purposes, we focus on disparate impact, leaving in the supplementary our formulation for equalized odds and equal opportunity.
The first discriminator in FR-GAN distinguishes predictions w.r.t. one sensitive group from those in the others. Disparate impact intends the sensitive attribute to be independent of the model’s prediction, i.e.,
I(Z; Ŷ ) = 0.
We explain how FR-GAN can enforce the above constraint. Let PZ(z) be the distribution of Z where z ∈ Z and Z is the set of possible sensitive attribute values. Let Ŷ |Z = z ∼ PŶ |z(·) and Ŷ ∼ PŶ (·). Then PŶ (·) = ∑ z∈Z PZ(z)PŶ |z(·).
We show that mutual information is equivalent to the following expression where the optimal discriminator D?z(ŷ) = PŶ ,Z(ŷ,z)
PŶ (ŷ) and
∑ z∈Z D ? z(ŷ) = 1 ∀ŷ ∈ Y .
Theorem 1. I(Z; Ŷ ) = maxDz(ŷ):∑z Dz(ŷ)=1∀ŷ∑z∈Z PZ(z)EPŶ |z [logDz(Ŷ )]+H(Z). While deferring the detailed proof to the supplemental materials, we provide a brief overview of the proof. As the optimization problem in the RHS is convex, we find the optimal dicriminator by solving the KKT conditions. We then show that the maximum value attained by the optimal discriminator is equal to the mutual information by using the properties of mutual information and the generalized Jensen-Shannon divergence (Lin, 1991).
What is more involved than showing the above equality is designing a right optimization problem. One needs to carefully handcraft a plausible optimization problem so that its unique solution matches the desired quantity. Here, we design the above optimization problem via a ‘guess-&-check’ approach aided by the structural insights across the KL divergences that appear in an alternative expression of mutual information.
We now discuss how to implement the above expression. Since we do not know PŶ |z(·) exactly, we compute the following empirical version:
max Dz(ŷ): ∑ z Dz(ŷ)=1∀ŷ ∑ z∈Z PZ(z) ∑
i:z(i)=z
1
mz logDz(ŷ
(i)) +H(Z).
Now for sufficiently large m, the number mz of examples with z(i) = z is approximately the same as PZ(z)m. Therefore, the above expression becomes:
max Dz(ŷ): ∑ z Dz(ŷ)=1∀ŷ ∑ z∈Z ∑ i:z(i)=z 1 m logDz(ŷ (i)) +H(Z).
Interestingly, this formulation is exactly the same as that in the original GAN (Goodfellow et al., 2014) when |Z| = 2. We also remark that our formulation does not require a prior knowledge on PZ(z).

heading: 4.2 ROBUSTNESS

text:
We utilize a clean yet small validation set to ensure robust training. In particular, we add a second discriminator that distinguishes the training data with predictions {(x(i), z(i), ŷ(i))}mi=1 from the validation set {(x(i)val, z (i) val, y (i) val)} mval i=1 . Intuitively, if the classifier is confused by data poisoning in the training data, then its predictions will not be consistent with the labels of the clean data, and the discriminator would be able to detect that difference. While using a validation set is inspired by the meta learning approach by Ren et al. (2018), we take an adversarial learning approach. A key motivation behind this choice is that our GAN approach introduces a knob that controls the emphasis of robust training. We find that this knob enables FR-GAN to be more robust to the validation set size. See details in Section 5.1.
Our formulation is as follows. Let V ∼ Bern(β) denote whether an example comes from training data (V = 1) or from the validation set (V = 0). We then want to ensure that V is independent of the training data and predictions, i.e., I(V ;X,Z, Ŷ ) = 0, which means the predictions on the training data are indistinguishable from labels of the validation set. This way we can mimic the clean data set while expecting an indirect sanitization effect. We optimize this independence using a formulation similar to that of fairness. Another possible formulation is to use the hard decision value from D(Ŷ ), say Ẑ, instead of Z. However, this discriminator can only be trained after the fairness discriminator is trained. Instead, we take an approach that allows us to train the two discriminators at the same time.

heading: 4.3 ARCHITECTURE

text:
Figure 2 illustrates the architecture of FR-GAN. For the loss function in the generator, we employ the cross entropy loss:
L1 = 1
m m∑ i=1 −y(i) log ŷ(i) − (1− y(i)) log(1− ŷ(i)).
As per Theorem 1, we set the loss function w.r.t. the fairness discriminator as:
L2 = max D(·) ∑ z∈Z ∑ i:z(i)=z 1 m logDz(ŷ (i)) +H(Z)
where D(·) := (D1(·), . . . , D|Z|(·)). The condition ∑ z∈Z D ? z(Ŷ ) = 1 can be enforced by adding a softmax layer to the discriminator.
Finally, implementing I(V ;X,Z, Ŷ ) as advised by Theorem 1, we set the loss function w.r.t. the robustness discriminator as:
L3 = max Dr(·) ∑ i:v(i)=0 1 m logDr(x(i), z(i), ŷ(i)) + ∑ i:v(i)=1 1 m log(1−Dr(x(i), z(i), ŷ(i))) +H(V ).
The final objective function is to minimize the weighted sum of these value functions:
min G(·) L1 + λ1L2 + λ2L3.
Here λ1 and λ2 are tuning knobs that play roles to emphasize fair and robust training, respectively.

heading: 5 EXPERIMENTS

text: We provide experimental results for disparate impact, while leaving in the supplementary those for equalized odds and equal opportunity. All of our results are on a separate test set. More implementation details are also in the supplementary.

heading: 5.1 SYNTHETIC DATA RESULTS

text:
For the synthetic data, we generate 2,000 examples with two non-sensitive attributes a and b, a sensitive attribute z, and a label y, using a method similar to the algorithm proposed by Zafar et al. (2017). Both z and y are binary and generated uniformly at random. For the four possible (z, y) combinations, we generate different normal distributions for a and b. Finally for each example, the a and b values are sampled as per the normal distribution associated with the (z, y) pair. For data poisoning, we flip the z values of examples so as to lower the model’s accuracy the most as described in Section 3.
To make a fair comparison in the context of robust training, we also improve the existing fairness methods with data sanitization and compare them with FR-GAN. Since FR-GAN is one of the first works to address both fairness and robustness, there are few baselines to compare with. Hence, we employed one reasonable baseline, which first sanitizes the poisoned data using a well-known sanitization technique and then performs each fairness algorithm. There are several sanitization techniques surveyed in Koh et al. (2018), and we choose the loss defense method. Here examples that
are not well fit by the model and result in a high loss are considered poisoned and are subsequently discarded. Loss defense is the only method that utilizes the trained model and thus has the best chance of defending poisoning attacks.
Accuracy and fairness We compare FR-GAN with FAIRNESS CONSTRAINTS (Zafar et al., 2017), LABEL BIAS CORRECTION (Jiang & Nachum, 2019), ADVERSARIAL DEBIASING (Zhang et al., 2018), and a plain non-fairness method, logistic regression. We also compare with the meta learning approach by Ren et al. (2018), which focuses on robust training. For this comparison, we tailor our framework to a setting where λ1 = 0 (no fairness constraint), and name it R-GAN. For FR-GAN, we use a validation set that amounts to 10% of Dtr. When setting λ1 and λ2, we usually fix λ2 to some value and then adjust λ1. For the clean data, we apply proper hyperparameters so that the disparate impacts are similar (around 0.8) across all distinct methods. We do the same for the poisoned data. There is no such hyperparameter tuning for logistic regression and the meta learning by Ren et al. (2018), as the frameworks have no knob for adjusting fairness. We find that this leads to poor disparate impacts. For the three fairness methods, data poisoning aggravates performances in accuracy, fairness or both. For example, the disparate impact and accuracy of FAIRNESS CONSTRAINTS fall by 8.03% and 18.7%, respectively. On the other hand, the performance changes for FR-GAN are negligible: disparate impact increases by 2.09%, and accuracy decreases by 0.12%. Table 1 also shows that combining the fairness methods with loss defense (rows 6–8) does not always yield better accuracy and fairness. In fact, using the loss defense may lower the accuracy or fairness (e.g., LBC+LD has a disparate impact of 0.513 on poisoned data while LBC has 0.816). Perhaps this is because the data sanitization sometimes mistakenly discards clean data as well. The results demonstrate that even a data sanitization technique may not help much. In the supplementary, we also perform experiments using equalized odds and show that FR-GAN has similar benefits.
We observe how accuracy trades off with fairness on clean and poisoned datasets. The results for FAIRNESS CONSTRAINTS are shown in Figure 1b. For LABEL BIAS CORRECTION, we employ the number of training iterations as a knob to trade accuracy off fairness. As shown in Figure 3a, the tradeoff curve shifts to the left, which demonstrates a clear tradeoff degradation. For ADVERSARIAL DEBIASING, we employ the α parameter (Zhang et al., 2018) analogous to λ1 as a knob to trade accuracy off fairness. We see in Figure 3b that the tradeoff curve again shifts to the left.
Validation set size Figures 3c to 3e show how the validation set size affects the robustness of FRGAN. In particular, we compare the accuracy-fairness tradeoff of FR-GAN on clean data and that on poisoned data while varying the size of the validation set. When running on poisoned data, we fixed λ2 = 0.2 and varied λ1. We see that even 2% validation set (Figure 3d) is sufficient to maintain the accuracy and fairness obtained on the clean data. When using 0.5% (Figure 3e), the validation set is too small and has an adverse effect on the training. However, our framework includes a tuning knob λ2, so by decreasing λ2 down to 0.1, we could de-emphasize robust training, thereby avoiding
the adverse effect (Figure 3f). This is in contrast to the meta learning approach, which suffers from a non-negligible performance degradation for a very small validation set. See details in the supplementary. The results show that only a holistic framework like FR-GAN can achieve both excellent model fairness and training robustness. In comparison, other methods tailored for only one of these objectives lose either accuracy, fairness or both.
Ablation study (Without ‘R’) For a small value of λ2, we observe in Figure 3f that the accuracyfairness tradeoff curve shifts to the left, just like other fairness-only methods. To make the comparison clearer, we also plot them together in the supplementary. (Without ‘F’) When λ1 = 0, FR-GAN is expected to behave similarly to the other non-fair algorithms. The performance of FR-GAN with λ1 = 0, which we dub as R-GAN, is reported in Table 1. One can observe that R-GAN achieves a poor fairness performance just like the other non-fair algorithms.

heading: 5.2 REAL DATA RESULTS

text:
We use the following benchmark datasets: ProPublica COMPAS (Angwin et al., 2016) and Adult Census (Kohavi, 1996). For sensitive attributes, we use SEX for both datasets. We do not employ the German Credit data set (Dua & Graff, 2017) as its highly biased towards positive labels. With such dataset, it is nearly impossible to observe any meaningful tradeoff.
For data poisoning, we use the same method employed on synthetic data: flipping the z values so as to maximize the accuracy performance degradation. While there may be stronger attacks, this poisoning attack is effective enough to fool the three fairness methods. In the supplementary, we also evaluate FR-GAN against label flipping attacks and show similar results. We use the same constructions for the validation set and poisoning as described in Section 3.
Accuracy and fairness See Table 2. As in Table 1, we apply proper hyperparameters so that disparate impacts are similar across all distinct methods, both for the clean and poisoned datasets. The results are similar to Table 1: logistic regression and meta learning exhibit poor disparate impacts; the three fairness methods have worse disparate impact and accuracy due to data poisoning; and FR-GAN again shows little degradation both in fairness and accuracy. Table 2 also shows that combining the fairness methods with loss defense (rows 6–8 and 15–17) does not always yield better accuracy and fairness and may even lower them, which is consistent to the results on synthetic data.

heading: 6 DISCUSSION

text:
In this section, we discuss possible ways to extend FR-GAN.
Subjective labels FR-GAN can be generalized to a setting where labels are subjective instead of being clean or poisoned. Here the dataset without the undesirable human biases becomes the clean validation set. In general, given two datasets with different distributions where one of them is desirable, FR-GAN can train robustly against the other distribution.
Constructing real datasets We propose a general methodology for constructing a real dataset that has both bias and poisoning, as well as a “clean” validation set. Suppose that we are performing evaluation tasks (say loan decisions) where there is a high chance of human bias (i.e., poisoning) due to the high workload. We can construct a clean validation set by selecting a few tasks, assigning more evaluators, and taking a majority vote of the evaluations to minimize the bias. While employing more evaluators can be expensive, constructing a small validation set is sufficient for robust training, which makes this method practical.
Generalizing to all attacks An interesting question is whether FR-GAN can defend against all poisoning attacks. If we know all the possible attacks, we can construct a training set containing these attacks. In the more challenging case where we do not know which attacks even exist, there seems to be a fundamental limitation in protecting against the attacks. Generalization is a universal problem in machine learning where a model trained on one dataset is not guaranteed to perform well in another dataset with a different distribution. Although the generalization is a critical issue to address, it is currently beyond the scope of this paper.

heading: 7 CONCLUSION

text: We proposed FR-GAN, which is a holistic framework that enables both unfairness mitigation and robust training. Our key contribution is proposing a novel GAN architecture that enjoys the synergistic effect of combining two approaches: (1) employing a fairness discriminator that distinguishes predictions w.r.t. one sensitive group from others and (2) employing a robust discriminator that distinguishes training data with predictions from a clean yet small validation set. We showed that existing fairness methods are vulnerable to data poisoning, even when combined with data sanitization techniques. In comparison, FR-GAN is robust to the poisoning and can be adjusted to maintain reasonable accuracy and fairness even if the validation set is too small or unavailable. We showed that FR-GAN is guaranteed to maximize fairness and supports all the prominent fairness measures: disparate impact, equalized odds, and equal opportunity.

heading: A APPENDIX

heading: A.1 PROOF FOR THEOREM 1

text:
Before we present the proof of the main theorem, we first recall our notation. Let PZ(z) be the distribution of Z where z ∈ Z and Z is the set of possible sensitive attribute values. Let Ŷ |Z = z ∼ PŶ |z(·) and Ŷ ∼ PŶ (·). Then PŶ (·) = ∑ z∈Z PZ(z)PŶ |z(·). Also, let Y ∼ PY (·).
For convenience, let us repeat the statement of Theorem 1 here:
I(Z; Ŷ ) = max Dz(ŷ): ∑ z Dz(ŷ)=1,∀ŷ ∑ z∈Z PZ(z)EPŶ |z [ logDz(Ŷ ) ] +H(Z).
We now prove the theorem.
Proof. Denote byD the collection of Dz(ŷ) for all possible values of z and ŷ, and by ν the collection of νŷ for all values of ŷ. We can construct the Lagrangian function as follows:
L(D,ν) = ∑ z∈Z PZ(z)EPŶ |z [ logDz(Ŷ ) ] +H(Z) + ∑ ŷ∈Y νŷ ( 1− ∑ z∈Z Dz(ŷ) ) .
We use the following KKT conditions:
∂L(D,ν) ∂Dz(ŷ) = PZ(z) PŶ |z(ŷ) D?z(ŷ) − ν?ŷ = 0, ∀(ŷ, z) ∈ Y × Z,
1− ∑ z∈Z D?z(ŷ) = 0, ∀ŷ ∈ Y.
Solving the two equations, we obtain ν?ŷ = PŶ (ŷ) for all ŷ. Thus,
D?z(ŷ) = PZ(z)PŶ |z(ŷ)
PŶ (ŷ) .
Putting this to the above optimization,∑ z∈Z PZ(z)EPŶ |z [ log PZ(z)PŶ |z(Ŷ ) PŶ (Ŷ ) ] +H(Z)
= ∑ z∈Z PZ(z)EPŶ |z
[ log PZ(z)PŶ |z(Ŷ )
PŶ (Ŷ )
] + ∑ z∈Z PZ(z) log 1 PZ(z)
= ∑ z∈Z PZ(z)EPŶ |z
[ log PŶ |z(Ŷ )
PŶ (Ŷ ) ] = ∑ z∈Z PZ(z)DKL(PŶ |z‖PŶ )
, JSPZ (PŶ |z1 , . . . , PŶ |z|Z|) = I(Z; Ŷ ).
Here, the second last equality is due to the definition of the generalized JS divergence, and the last equality is due to its equivalence to the mutual information (Lin, 1991).

heading: A.2 EXTENSIONS TO OTHER FAIRNESS MEASURES

text:
The following theorem relates the conditional mutual information I(Z; Ŷ |Y ) to the solution of an optimization problem. Theorem 2.
I(Z; Ŷ |Y ) = max Dz|y(ŷ): ∑ z∈Z Dz|y(ŷ)=1;∀ŷ ∑ y∈Y ∑ z∈Z PY,Z(y, z)EPŶ |y,z [ logDz|y(Ŷ ) ] +H(Z|Y ).
Recall that this conditional mutual information term can be used to capture equalized odds, which is another important fairness metric. We also note that the following theorem can be modified in a straightforward manner so that it can handle I(Z; Ŷ |Y = 1), which can be used to capture equal opportunity.
We now prove the theorem.
Proof. Denote by D the collection of Dz|y(ŷ) for all possible values of (z, ŷ, and y) and by ν the collection of νy,ŷ for all values of y and ŷ. We can construct the Lagrangian function as follows:
L(D,ν) = ∑ y∈Y ∑ z∈Z PY,Z(y, z)EPŶ |y,z [ logDz|y(Ŷ ) ] +H(Z|Y ) + ∑ ŷ∈Y ∑ y∈Y νy,ŷ ( 1− ∑ z∈Z Dz|y(ŷ) ) .
We use the following KKT conditions:
∂L(D,ν) ∂Dz|y(ŷ) = PY,Z(y, z) PŶ |y,z(ŷ) D?z|y(ŷ) − νy,ŷ = 0, ∀(ŷ, y, z) ∈ Y × Y × Z
1− ∑ z∈Z D?z|y(ŷ) = 0, ∀(ŷ, y) ∈ Y × Y.
Solving the two equations, we obtain ν?y,ŷ = PY,Ŷ (y, ŷ) for all (y, ŷ) ∈ Y × Y . Thus,
D?z|y(ŷ) = PZ|y(z)PŶ |y,z(ŷ)
PŶ |y(ŷ) ,∀y, ŷ ∈ Y × Y.
Putting this to the above optimization,∑ y∈Y ∑ z∈Z PY,Z(y, z)EPŶ |y,z [ log PZ|y(z)PŶ |y,z(Ŷ ) PŶ |y(Ŷ ) ] +H(Z|Y )
= ∑ y∈Y ∑ z∈Z PY,Z(y, z)EPŶ |y,z
[ log PZ|y(z)PŶ |y,z(Ŷ )
PŶ |y(Ŷ )
] + ∑ y∈Y ∑ z∈Z PY,Z(y, z) log 1 PZ|y(z)
= ∑ y∈Y ∑ z∈Z PY,Z(y, z)EPŶ |y,z
[ log PŶ |y,z(Ŷ )
PŶ |y(Ŷ )
]
= ∑ y∈Y ∑ z∈Z PY (y)PZ|y(z)EPŶ |y,z
[ log PŶ |y,z(Ŷ )
PŶ |y(Ŷ )
]
= ∑ y∈Y PY (y) ∑ z∈Z PZ|y(z)EPŶ |y,z
[ log PŶ |y,z(Ŷ )
PŶ |y(Ŷ ) ] = ∑ y∈Y PY (y) ∑ z∈Z PZ|y(z)DKL(PŶ |y,z‖PŶ |y)
, ∑ y∈Y PY (y) · JSPZ|y (PŶ |z1,y, . . . , PŶ |z|Z|,y)
= ∑ y∈Y PY (y)I(Z; Ŷ |Y = y) = I(Z; Ŷ |Y ).
The third last equality is due to the definition of the generalized JS divergence; the second last equality is due to its equivalence to the mutual information (Lin, 1991); and the last equality is due to the definition of conditional mutual information.
We now discuss how to actually compute the mutual information. We compute the following empirical version using the example {(x(i), z(i), y(i))}mi=1.
max Dz|y(ŷ): ∑ z∈Z Dz|y(ŷ)=1;∀ŷ ∑ y∈Y ∑ z∈Z PY,Z(y, z) ∑
i:(y(i),z(i))=(y,z)
1
my,z logDz|y(ŷ
(i)) +H(Z|Y ).
Now for a sufficiently large value of m, my,z ≈ PY,Z(y, z)m. Therefore, the above expression is approximated as:
max Dz|y(ŷ): ∑ z∈Z Dz|y(ŷ)=1;∀ŷ ∑ y∈Y ∑ z∈Z ∑ i:(y(i),z(i))=(y,z) 1 m logDz|y(ŷ (i)) +H(Z|Y ).
Hence, we can set L2 (i.e., the loss w.r.t. the fairness discriminator) to the above expression. The rest of the objective function is the same. Figure 4 shows the resulting FR-GAN architecture.

heading: A.3 ADDITIONAL EXPERIMENTS

heading: A.3.1 SYNTHETIC DATA

text:
We continue our experiments from Sections 3 and 5.1.
Label flipping for data poisoning Figure 5a to 5d show the experimental results when the data is poisoned via label flipping. We used the same synthetic data, as illustrated in Figure 1a, and flipped 7% of labels from the clean data to make a poisoned data. Figures 5a, 5b, and 5c show the accuracy-fairness tradeoff curves of FAIRNESS CONSTRAINTS, LABEL BIAS CORRECTION, and ADVERSARIAL DEBIASING, respectively. While the tradeoff curves in Figure 5a to 5c considerably shift to the left, Figure 5d shows that FR-GAN is noticeably robust against label flipping.
Meta learning with different validation set sizes Table 3 shows the accuracy and fairness results for the meta learning method by Ren et al. (2018) (ML) for different validation set sizes. We observe a drastic decrease of accuracy when the validation set size is 0.1% of the training data.
Ablation study for fairness Figure 6 shows the accuracy-fairness tradeoff curves for ADVERSARIAL DEBIASING and FR-GAN on the synthetic dataset. ADVERSARIAL DEBIASING tradeoff curves are from Figure 3b, and the FR-GAN curve comes from Figure 3f. We observe in Figure 6 that the accuracy-fairness tradeoff of FR-GAN using a small value of λ2 on the poisoned data is similar to the results of fairness-only methods.

heading: A.3.2 REAL DATA

text:
We continue our experiments from Section 5.2.
Label flipping for data poisoning Table 4 uses the same experimental setting as in Table 2, except that data poisoning is label flipping (not w.r.t. sensitive attributes). We observe similar trends as in Table 2. For the non-fairness methods, the fairness results are significantly worse than other methods. For the fairness-only methods, both fairness and accuracy tend to be high on the clean data, but at least one of them decreases significantly on the poisoned data. On the other hand, FR-GAN yields high fairness and accuracy on the clean data and is also the most resilient against poisoned data. The results emphasize our holistic technique that takes both fairness and accuracy into consideration.

heading: A.3.3 FR-GAN USING OTHER FAIRNESS MEASURES

text: As we showed in Appendix A.2, FR-GAN respects equalized odds and equal opportunity. Table 5 shows the experimental results on the synthetic and real datasets for equalized odds. We see that FR-GAN significantly improves equalized odds with reasonable accuracy. The results w.r.t. equal opportunity are similar and thus not shown here.

heading: A.4 TRAINING METHODOLOGY

text:
We implement all algorithms in PyTorch (Paszke et al., 2017), and all experiments are performed on a server with Intel i7-6850 CPUs.
The generator G is a neural network with zero or one hidden layer. The discriminator Df is a single layer neural network, and the discriminatorDr is a neural network with one hidden layer. We used 8 or 16 nodes in the hidden layers. We set an Adam optimizer (Kingma & Ba, 2014) for the generator,
and a stochastic gradient descent (SGD) optimizer for each discriminator. We empirically observe that one can stabilize the training procedure by freezing the parameters of the fairness discriminator Df for the initial phase of training. Thus, we choose to freeze the parameters of the fairness discriminator Df for the first few epochs until the generator achieves a certain accuracy. We use the generator/discriminator update ratio of 1:1 for the first 300 epochs and use the update ratio of 1:3 (or 1:5) for the rest of training.
Also, we use the following details for choosing the values of λ1 and λ2. For clean data, we set λ2 as a small value (e.g., 0.01) and vary λ1 from 0 to 0.95. For poisoned data, we set λ2 as 0.2, 0.3, or 0.4, and vary λ1 from 0 to 0.95 − λ2. Given the values of λ1 and λ2, we also normalize L1 (the generator loss) by multiplying it with (1− λ1 − λ2).

heading: A.5 ADDITIONAL EXPERIMENTS ON ADULT DATASET

text: We compare FR-GAN and AD using the AdultCensus dataset and the same poisoning in Section 3. For a fair comparison, we use loss defense (LD) as data sanitization prior to running AD. We generate confusion matrices both for disparate impact (DI) and equalized odds (EO). Overall, FRGAN performs better than AD because of its robustness discriminator. The robustness discriminator successfully ignores the poisoned distribution in the training data, and FR-GAN’s TPR is higher than AD with sanitization. Table 6 shows confusion matrices as results of disparate-impact-focused experiment. The results are reported when AD achieves (Acc, DI) = (0.728, 0.821), and FR-GAN achieves (Acc, DI) = (0.827, 0.866). In addition, Table 7 represents confusion matrices as results of equalized-odds-focused experiment. The results are reported when AD achieves (Acc, EO when y = 1, EO when y = 0) = (0.788, 0.844, 0.750), and FR-GAN achieves (Acc, EO when y = 1, EO when y = 0) = (0.838, 0.824, 0.773).