summaries: The paper proposes a GAN process for training neural machine translation models. The noise generator in this approach uses a switching-aligned-words technique where they randomly switch a word in the source sentence with its translation in the target sentence. They use fast-align to get alignments between source and target sentences. The experiments show that the noisy sentence pair generator performs best with the proposed switch and align approach in comparison with other (more random) methods.

summaries: This paper describes a method for data augmentation and/or regularization for machine translation that works by running a word aligner on the parallel data, and then with some probability \gamma, replacing a source token with its corresponding target token or vice versa. A proposed variant also mixes the embeddings of the two words. Small improvements are shown over simpler noising strategies such as replacing words with placeholder tokens or with random words from the vocabulary.

summaries: The paper proposes a data augmentation technique where source sentences are perturbed by replacing (or mixing) source words with their aligned counterparts from the target language (while the target sentences remain as is). Alignments can be either obtained from an unsupervised aligner like fast-align or from the attention distribution of an NMT model. Perturbations are aimed to be semantically invariant to preserve the meaning of the source sentence. In addition to simply replacing the source word with the aligned word, authors also try out inputting a weighted combination of both the source word and the target word and refer to this method as “mixing”. Empirical observations suggest that simply replacing the source word with the aligned target yields better results.

abstractText: In neural machine translation (NMT), data augmentation methods such as backtranslation make it possible to use extra monolingual data to help improve translation performance, while it needs extra training data and the in-domain monolingual data is not always available. In this paper, we present a novel data augmentation method for neural machine translation by using only the original training data without extra data. More accurately, we randomly replace words or mixup with their aligned alternatives in another language when training neural machine translation models. Since aligned word pairs appear in the same position of each other during training, it is helpful to form bilingual embeddings which are proved useful to provide a performance boost (Liu et al., 2019). Experiments on both small and large scale datasets show that our method significantly outperforms the baseline models.

authors:
Dzmitry Bahdanau
Kyunghyun Cho
Yoshua Bengio

title: Neural machine translation by jointly learning to align and translate

authors:
Ekin D Cubuk
Barret Zoph
Dandelion Mane
Vijay Vasudevan
Quoc V Le

title: Autoaugment: Learning augmentation policies from data

venue: In arXiv preprint arXiv:1805.09501,

authors:
Anna Currey
Antonio Valerio Miceli-Barone
Kenneth Heafield

title: Copied monolingual data improves low-resource neural machine translation

venue: In WMT,

authors:
Chris Dyer
Victor Chahuneau
Noah A Smith

title: A simple, fast, and effective reparameterization of ibm model 2

venue: In NAACL,

authors:
Sergey Edunov
Myle Ott
Michael Auli
David Grangier

title: Understanding back-translation at scale

venue: In EMNLP,

authors:
Marzieh Fadaee
Christof Monz

title: Back-translation sampling by targeting difficult words in neural machine translation

authors:
Marzieh Fadaee
Arianna Bisazza
Christof Monz

title: Data augmentation for low-resource neural machine translation

venue: In ACL,

authors:
Fei Gao
Jinhua Zhu
Lijun Wu
Yingce Xia
Tao Qin
Xueqi Cheng
Wengang Zhou
Tie-Yan Liu

title: Soft contextual data augmentation for neural machine translation

authors:
Jonas Gehring
Michael Auli
David Grangier
Denis Yarats
Yann N Dauphin

title: Convolutional sequence to sequence learning

venue: In ICML,

authors:
Ian Goodfellow
Jean Pouget-Abadie
Mehdi Mirza
Bing Xu
David Warde-Farley
Sherjil Ozair
Aaron Courville
Yoshua Bengio

title: Generative adversarial nets

venue: NeurIPS,

authors:
Di He
Yingce Xia
Tao Qin
Liwei Wang
Nenghai Yu
Tie-Yan Liu
Wei-Ying Ma

title: Dual learning for machine translation

authors:
Vu Cong Duy Hoang
Philipp Koehn
Gholamreza Haffari
Trevor Cohn

title: Iterative backtranslation for neural machine translation

title: Contextual augmentation: Data augmentation by words with paradigmatic relations

authors:
Alex Krizhevsky
Ilya Sutskever
Geoffrey E Hinton

title: Imagenet classification with deep convolutional neural networks

venue: In NeurIPS,

authors:
Guillaume Lample
Alexis Conneau
Ludovic Denoyer
Marc’Aurelio Ranzato

title: Unsupervised machine translation using monolingual corpora only

authors:
Xuebo Liu
Derek F Wong
Yang Liu
Lidia S Chao
Tong Xiao
Jingbo Zhu

title: Shared-private bilingual word embeddings for neural machine translation

authors:
Tomas Mikolov
Kai Chen
Greg Corrado
Jeffrey Dean

title: Efficient estimation of word representations in vector space

authors:
Tim Salimans
Ian Goodfellow
Wojciech Zaremba
Vicki Cheung
Alec Radford
Xi Chen

title: Improved techniques for training gans

authors:
Rico Sennrich
Barry Haddow
Alexandra Birch

title: Improving neural machine translation models with monolingual data

authors:
Ilya Sutskever
Oriol Vinyals
Quoc V Le

title: Sequence to sequence learning with neural networks

authors:
Ashish Vaswani
Noam Shazeer
Niki Parmar
Jakob Uszkoreit
Llion Jones
Aidan N Gomez
Łukasz Kaiser
Illia Polosukhin

title: Attention is all you need

authors:
Shuo Wang
Yang Liu
Chao Wang
Huanbo Luan
Maosong Sun

title: Improving back-translation with uncertainty-based confidence estimation

authors:
Xinyi Wang
Hieu Pham
Zihang Dai
Graham Neubig

title: Switchout: an efficient data augmentation algorithm for neural machine translation

venue: In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,

authors:
Lijun Wu
Yiren Wang
Yingce Xia
QIN Tao
Jianhuang Lai
Tie-Yan Liu

title: Exploiting monolingual data at scale for neural machine translation

authors:
Yonghui Wu
Mike Schuster
Zhifeng Chen
Quoc V Le
Mohammad Norouzi
Wolfgang Macherey
Maxim Krikun
Yuan Cao
Qin Gao
Klaus Macherey

title: Google’s neural machine translation system: Bridging the gap between human and machine translation

venue: arXiv preprint arXiv:1609.08144,

authors:
Fengshun Xiao
Jiangtong Li
Hai Zhao
Rui Wang
Kehai Chen

title: Lattice-based transformer encoder for neural machine translation

authors:
Ziang Xie
Sida I Wang
Jiwei Li
Daniel Lévy
Aiming Nie
Dan Jurafsky
Andrew Y Ng

title: Data noising as smoothing in neural network language models

authors:
Hongyi Zhang
Moustapha Cisse
Yann N Dauphin
David Lopez-Paz

title: mixup: Beyond empirical risk minimization

heading: 1 INTRODUCTION

text:
Deep neural networks show great performances when trained on massive amounts of data. Data augmentation is a simple but effective technique to generate additional training samples when deep learning models are thirsty for data. In the area of Computer Vision, it is a standard practice to use image data augmentation methods because trivial transformations for images like random rotation, resizing, mirroring and cropping (Krizhevsky et al., 2012; Cubuk et al., 2018) doesn’t change its semantics. This presence of of semantically invariant transformation makes it easy to use image data augumentation in Computer Vision research.
Unlike image domain, data augmentation on text for Natural Language Processing (NLP) tasks is usually non-trivial as there is often a prerequisite to do some transformations without changing the meaning of the sentence. In this paper we will focus on data augmentation techniques in neural machine translation (NMT) which is special and more difficult than other NLP tasks since we should maintain semantic consistency within language pairs which is from quite possibly different domains.
Data augmentation techniques in NMT can be divided into two categories dependent on whether additional monolingual corpus is uesd. If in-domain monolingual training data for NMT is available, one successful data augmentation method is back-translation (Sennrich et al., 2016), whereby an NMT model is trained in the reverse translation direction (target-to-source) and then used to translate target-side monolingual data back to source language. The resulting synthetic parallel corpus can added to existing training data to learn a source-to-target model. Other more refined ideas of backtranslation include dual learning (He et al., 2016) or Iterative Back-translation (Hoang et al., 2018).
Sometimes when in-domain monolingual data is limited, existing methods including randomly swapping two words, dropping word, replacing word with another one (Lample et al., 2018) and so on are applied to perform transfromations to original training data without changing its semantics to the greatest extent. However, due to text characteristics, these random transformations often result in significant change in semantics. Gao et al. (2019) propose to replace the embedding of word by a weighted combination of mutiple semantically similar words. Also, Xiao et al. (2019) use a lattice structure to integrate multiple segmentations of a single sentence to perfrom an immediate data augmentation.
In this work, we propose Switching-Aligned-Words (SAW) data augmentation, a simple yet effective data augmentation approach for NMT training. It belongs to the second class of data augmentation
methods where in-domain monolingual data is limited. Different from the previous methods that conduct semantically invariant transformations within each language, we propose to use another language (target language) to help make semantically invariant transformations for current language (source language) by switching aligned words randomly. We use an unsupervised word aligner fast-align1 (Dyer et al., 2013) to pair source and target words that have similar meaning.
To verify the effectiveness of our method, we conduct experiments on WMT14 English-to-German and IWSLT14 German-to-Englisth datasets. The experimental results show that our method can obtain remarkable BLEU score improvement over the strong baselines.

heading: 2 RELATED WORK

text: We describes the related work about data augmentation for NMT with or without using additional monolingual data in this section.

heading: 2.1 WITH MONOLINGUAL DATA

text:
The most successful data augmentation techiques to leverage monolingual data for NMT training is back-translation. It requires training a target-to-source system in order to generate additional synthetic parallel data from the monolingual target data. This data complements human bitext to train the desired source-to-target system. There has been a growing body of literature that analyzes and extends back-translation. Edunov et al. (2018) demontrate that it is more effective to generate source sentences via sampling rather than beam search. Hoang et al. (2018) present iterative backtranslation, a method for generating increasingly better synthetic parallel data from monolingual data to train NMT model. Fadaee & Monz (2018) show that words with high predicted loss during training benefit most. Wang et al. (2019) propose to quantify the confidence of NMT model predictions based on model uncertainty to better cope with noise in synthetic bilingual corpora produced by back-translation. Dual learning (He et al., 2016) extends the back-translation approach to train NMT systems in both translation directions. When jointly training the source-to-target and targetto-source NMT models, the two models can provide back translated data for each other direction and perform multi-rounds back-translation.
Different from back-translation, Currey et al. (2017) show that low resource language pairs can also be improved with synthetic data where the source is simply a copy of the monolingual target data. Wu et al. (2019) propose to use noised training to better leverage both back-translation and self-training data.

heading: 2.2 WITHOUT MONOLINGUAL DATA

text: Lample et al. (2018) randomly swap the words within a fixed small window size or drop some words in a sentence for learning an autoencoder to help train the unsupervised NMT model. Fadaee et al. (2017) propose to replace a common word by low-frequency word in the target sentence, and change its corresponding word in the source word to improve translation quality of rare words. In Xie et al. (2017), they replace the word with a placeholder token or a word sampled from the frequency distribution of vocabulary, showing that data noising is an effective regularizer for NMT. Kobayashi (2018) propose an approach to ues the prior knowledge from a bi-directional language model to replace a word token in the sentence. Gao et al. (2019) try to replace the ids of word by a soft ids and they train Transformer language models in original training data to get soft words. Wang et al. (2018) introduce a data augmentation method for NMT called SwitchOut to randomly replace words in both source and target sentences with other words.

heading: 3 OUR APPROACH

text:
We first describe the background and our proposed switching-aligned-words data augumentation approach. The framework can be seen as an adversarial training process like Generative Adversarial Networks (GAN) (Goodfellow et al., 2014; Salimans et al., 2016), see Figure 1 for an overview. For
1https://github.com/clab/fast align
I want to thank my friends
Ich möchte meinen Freunden danken
Figure 2: The illustration for alignment model. English sentence is ”I want to thank my friends.”, and corresponding German sentence is ”Ich möchte meinen Freunden danken”.
image generation, in which a discriminator and a generator compete with each other: the generator aims to generate images similar to the natural ones, and the discriminator aims to detect the generated ones from the natural ones. For data augmentaion methods in NMT, the noise generator can be any model that produces noise over parallel sentences, in our method it is an alignment model which is shown in Figure 2. Finally, the NMT model is trained as a discriminator to distinguish generated sentences from the original ones and the process of detection noise offers NMT model an ability to learn bilingual alignment information.

heading: 3.1 BACKGROUND

text:
Given a source and target sentence pair (x,y), where x = (x1, x2, · · · , x|x|) is a source-language sentence and y = (y1, y2, · · · , y|y|) is a target-language sentence. A neural machine translation system models the conditional probability:
P (y|x) = |y|∏ j=1 P (yj |y<j ,x) (1)
based on an encoder-decoder framework with an attention mechanism (Sutskever et al., 2014; Bahdanau et al., 2014). Encoder and decoder can be specialized using different neural architectures including GRU (Bahdanau et al., 2014), LSTM (Wu et al., 2016), CNN (Gehring et al., 2017) and Transformer (Vaswani et al., 2017), among which the self-attention based Transformer is the stateof-the-art architecture for NMT.
The decoder predicts a corrresponding translation y = (y1, · · · , y|y|) step by step based on the last decoding state and source context. The translation probability can be formulated as follows:
P (yj |y<j ,x) = q(yj−1, sj , cj) (2)
where sj and cj denote the decoding state and the source context at the j-th time step respectively. Here, q(·) is the softmax layer. Sepcifically,
sj = g(yj−1, sj−1, cj) (3)
where g(·) is the corresponding neural architecture unit. The context vector cj is calculated as a weighted sum of the source annotations hi on the basis of attention mechanism:
cj = |x|∑ i=1 αjihi (4)
The alignment model αji measures the similarity between sj and hi. The whole model is jointly trained to seek the optimal parameters that can be used to correctly encode the source sentences and decode them to corresponding target sentences.

heading: 3.2 ALIGNMENT

text:
NMT models learn the alignment between source words xi and target word yj mainly deponds on these two aspects: attention and word embeddings. Since attention weight αji measures the similarity between sj and hi, it has been widely used to evaluate the word alignment between yj and xi, so that the word alignment is explicitly modeled.
NMT models also try to learn word alignment information by updating word embeddings when training. In monolingual vector space, similar words tend to have commonalities in the same dimensions of their word vectors (Mikolov et al., 2013). These commonalities include: (1) a similar degree (value) of the same dimension and (2) a similar positive or negative correlation of the same dimension. In bilingual vector space, Liu et al. (2019) assume that the source and target words that have similar meanings should also have similar embedding vectors. Hence, they propose to perform a sharing techique between source and target word embedding space resulting significantly imporvement in alignment quality and translation performance.
Motivated by their findings, we propose to generate new training samples by replacing one word in the original sentences with its alinged word in corresponding target sentences. According to the characteristic of bilingual embeddings, aligned words tend to have similar meanings even in different language, so our replacing method will preserve the original meaning of the sentence to a great extend. Also, when training the model we put a aligned target word in the similar context of source sentence, it is helpful for source and target words with similar meanings to learn similar embedding representation.

heading: 3.3 SWITCHING ALIGNED WORDS BY REPLACEMENT

text:
Inspired by the above intuition, we propose to augment NMT training data by replacing a randomly chosen word in a sentence by its aligned target word. Suppose we have an extra alignment model A(·|·) such as intrinsic attention mechanism (Bahdanau et al., 2014) or unspervised word aligner (Dyer et al., 2013). Given a sentence pair (x,y), each source word xi is aligned with a target word ŷi that has the highest alignment probability among the candidates, and is computed as follows:
ŷi = argmax y∈a(x)
logA(y|xi) (5)
where a(·) denotes the set of aligned candidates. So the conditional probability can be written as:
P (y|x) = |y|∏ j=1 P (yj |y<j , C(x))
= |y|∏ j=1 P (yi|y<j , x1, . . . , ŷk, . . . , x|x|)
(6)
where k-th source word is replaced by corresponding target word. In experiments, we randomly choose a word in the training data with probability γ1 and replace it by its aligned target word.

heading: 3.4 SWITCHING ALIGNED WORDS BY MIXUP

text:
Mixup is a simple yet effective image augmentation techique introduced by Zhang et al. (2017). The idea is to combine two random images in a mini-batch in some proportion to generate synthetic examples for training. Bringing this idea to our work, we do not directly replace source word with corresponding aligned target word with probability γ1, instead we mix up these two word embeddings to form a combined embedding which contain both source and target information:
E(xi) = (1− γ2)E(xi) + γ2E(C(x)) = (1− γ2)E(xi) + γ2E(ŷi)
(7)
where E is the embedding lookup table, γ2 is the mixup ratio which is a hyper-parameter.
The intuition behind mixup is that random linear interpolations between the embeddings of source word and corresponding target word let neural models regularize the representation of word embeddings. Mixing the aligned word pairs do not interrupt the representaion of word embeddings far from its original ones.

heading: 4 EXPERIMENT

text: In this paper, data augmentation will only process source data of the training data.

heading: 4.1 DATASETS

text:
Two translation tasks, IWSLT14 German-to-English (De-En) and WMT14 English-to-German (EnDe), are used for our evaluation.
IWSLT14 German-English IWSLT14 De-En dataset contains 153K training sentence pairs. We randomly select 7K data from the training set as validation set and use the combination of dev2010, dev2012, tst2010, tst2011 and tst2012 as test set with 7K sentences which are preprocessed firstly. BPE algorithm is used to process words into subwords, and number of subword tokens in the shared vocabulary is 10k.
WMT14 English-German We use the WMT14 En-De dataset with 4.5M sentence pairs for training. We randomly select 40K data from the training set as validation set and use newstest2014 as test set. Dataset is segmented by BPE and the number of subword tokens in the shared vocabulary is 32K. The sentences longer than 250 subword tokens are removed from the training dataset.

heading: 4.2 BASELINES

text:
We compare our approach with following baselines:
• Base: The original training strategy without any data augmentation; • Swap: Randomly swap words in nearby positions with a window size k (Lample et al.,
2018);
• Dropout: Randomly drop word tokens (Lample et al., 2018); • Blank: Ramdomly replace word tokens with a placeholder token (Xie et al., 2017); • Smooth: Randomly replace word tokens with a sample from the unigram frequency distri-
bution over the vocabulary (Xie et al., 2017);
All above introduced methods except Swap incorporate a hyper-parameter, the probability γ of each word token to be replaced in training phase. We set γ with different values in 0,0.05,0.1,0.15,0.2, and report the best result for each method. As for Swap, we use 3 as window size following (Lample et al., 2018);

heading: 4.3 MODEL

text: We use the transformer base setting following Vaswani et al. (2017) for WMT14 En-De datasets, with a 6-layer encoder and 6-layer decoder. The dimensions of word embeddings, hidden states and the position-wise feed-forward networks are 512, 512, 2048 respectively. The dropout is 0.1 and attention head is 8. For IWSLT14 De-En datasets, we use the transformer small setting which has a 6-layer encoder and 6-layer decoder, but the dimensions of word embeddings, hidden states and the position-wise feed-forward networks are 512, 512, 1024 respectively. The dropout is 0.3 and attention head is 4. Word embeddings between the source, target and output softmax embeddings are tied as it is a normal setting. We set γ1 and γ2 with different values in {0, 0.05, 0.1, 0.15, 0.2}, and report the best result for each method. For all experiments, hyperparameters are optimized on a development set and then tested using only a single hyperparameter. We use beam size 4 and length penalty 0.6 for inference, and use multi-bleu2 to evaluate the quality of translation.

heading: 4.4 TRAINING

text:
All our models are trained on one TITAN RTX GPU. The implementation of model is based on fairseq toolkit3. We choose Adam optimizer with β1 = 0.9, β2 = 0.98, = 10−9 and the learning
2https://github.com/moses-smt/mosesdecoder/blob/master/scripts/generic/multi-bleu.perl 3https://github.com/pytorch/fairseq
rate setting strategy, which are all the same as Vaswani et al. (2017), lr = d−0.5·min(step−0.5, step· warmup−1.5step ) where d is the dimension of embeddings, step is the step number of training and warmupstep is the step number of warmup. When the number of step is smaller than the step of warmup, the learning rate increases linearly and the decreases. Significantly, our replacing or mixing decision is made at runtime allowing different transformations for the same sentence pair.

heading: 4.5 RESULTS

text:
The evalution results on IWSLT14 De-En and WMT14 En-De datasets are shown in Table 1. As we can see, the Replacement method can achieve 0.69 and 0.39 BLEU scores improvement over the Transformer small and the Transformer base baselines and the Mixup method improve the two baselines by 0.47 and 0.33 BLEU scores respectively.
Compared with other augmentation methods, we can see that (1) the Replacement method achieves the best results on all the datasets and (2) the Mixup method can achieve comparable or better results. Specially, we find that our method works better on relatively small scale datasets. As small scale datasets lack bilingual information compared to large scale datasets and are easy to fall into the overfitting problems, these results clearly demonstrate the effectiveness of our approach.

heading: 5 STUDY

text:
5.1 IMPACT OF γ1 AND γ2
We set different replacing probability value γ1 and mixup probability value γ2 to see the effect of our approach.
Figure 3 shows the BLEU scores on IWSLT14 De-En dataset of each method with different replacing probability, from which we can see that our method can obtain a consistent BLEU improvement within a large probability range and achieve the best performance when γ1 = 0.1 in each method. However, the performance begins to drop when γ1 > 0.1, we think the reason is that the semantic meanings of original sentence begin to be destroyed greatly. Also we find that Mixup is more stable than Replacement.
As we can see from Figure 4, the Mixup method can obtain a consistent BLEU improvement above baseline within a large probability range and the best BLEU socre is achieved in mixup probability γ2 = 0.1 when γ1 = 0.1.

heading: 5.2 ANALYSIS OF BILINGUAL EMBEDDINGS

text:
Since we suppose that aligned word pairs appear in the same position of each other during training will be helpful to form bilingual embeddings which are proved useful to provide a preformance boost (Liu et al., 2019), we study whether our approach is truly useful for bilingual embeddings. We randomly sample some words and their corresponding aligned words to analyze the relation within them. Specifically, we compare the cosine similarity between the embeddings of aligned words to figure out the changes of bilingual embeddings. Formally, we have aligned word pairs (xi,yj) and their embeddings E(xi) = (e(xi)1, e(xi)2, · · · , e(xi)d), E(yj) = (e(yj)1, e(yj)2, · · · , e(yj)d), where d is the embedding dimension. The cosine similarity can be defined as:
cos θ(E(xi),E(yj)) = ∑d k=1 e(xi)k · e(yj)k√∑d
k=1 e(xi) 2 k · √∑d k=1 e(yj) 2 k
(8)
where θ(E(xi),E(yj)) is the angle between embedding pairs. We finally normalize the results to 0 and 1, and the larger the value, the more similar the two embeedings are.
From Figure 5 we can see that (1) The embedding vectors between aligned word pairs have a very strong positive correlation since the normalized cosine similarity values are all above 0.5. (2) The
Replacement method significantly imporves the positive correlation between aligned word pairs which proves our hypothesis that switching aligned words is helpful to from bilingual embeddings. (3) The Mixup method does not seem to improve the quality of bilingual embeddings. We suppose that the improvement of translation quality mainly come from the introduction of noise to word embeddings.

heading: 6 CONCLUSION

text:
In this work, we have presented Switching-Aligned-Words (SAW) data augmentaion for NMT, which randomly replace words or mixup with their aligned alternatives in another language when training. It is simple yet effective and can be extremely useful when extra in-domain monolingual data is limited. Results on both small and large scale datasets have verified the effectiveness of our method.
In the future, besides focusing bilingual machine translation tasks, we are interested in extending our method to a multilingual scenario which needs more complex replacement and training strategies. In addition, we plan to study our approach in other cross-lingual NLP tasks.

title: SWITCHING-ALIGNED-WORDS DATA AUGMENTATION


summaries: The paper is to measure each client’s contribution to training the federated learning model. In particular, the contribution is measured by the distance between the local model and the global model in each iteration. The targeting problem is interesting, and the use of attention-based model divergence is also an interesting idea to measure the contribution. However, the paper lacks strict theoretical discussion to prove the proposed solution is a reasonable one rather than a heuristic method. Moreover, the experiment is too weak to support the claims. The paper’s technique contribution and originality are also limited.

summaries: The paper proposes a low computational complexity method for weighting contributions of clients in a federated learning setting. The main contributions are to compare the weighting method with Shapley values and their sensitivity to low data volume and quality. The paper is based on the FedAtt paper that calculates weights based on the Euclidean distance between the server model and each client and for each layer.

summaries: The paper proposes a new contribution measurement approach for federated learning. The basic idea is that the agent with a larger model update has a larger contribution. Specifically, based on FedAtt [1], the impact of a client is computed as the local updates plus the impact of the previous round times a decay rate. The experiments on a dataset show that the proposed approach can have a similar contribution measurement compared with Shapley Value.

abstractText: Federated learning is a framework for protecting distributed data privacy and has participated in commercial activities. However, there is a lack of a sufficiently reasonable contribution measurement mechanism to distribute the reward for each agent. In the commercial union, if there is no mechanism like this, every agent will get the same reward. This is unfair to agents that provide better data, so such a mechanism is needed. To address this issue, this work proposes a real-time contribution measurement method. Firstly, the method defines the impact of each agent. Furthermore, we comprehensively consider the current round and the previous round to obtain the contribution rate of each agent. To verify effectiveness of the proposed method, the work conducts pseudo-distributed training and an experiment on the Penn Treebank dataset. Comparing the Shapley Value in game theory, the comparative experiment result shows that the proposed method is more sensitive to both data quantity and data quality under the premise of maintaining real-time.

authors:
Yoshinori Aono
Takuya Hayashi
Lihua Wang
Shiho Moriai

title: Privacy-preserving deep learning via additively homomorphic encryption

venue: IEEE Transactions on Information Forensics and Security,

authors:
Eugene Bagdasaryan
Andreas Veit
Yiqing Hua
Deborah Estrin
Vitaly Shmatikov

title: How to backdoor federated learning, 2018

authors:
Xianglin Bao
Cheng Su
Yan Xiong
Wenchao Huang
Yifei Hu

title: Flchain: A blockchain for auditable federated learning with trust and incentive

venue: 5th International Conference on Big Data Computing and Communications (BIGCOM),

authors:
H Brendan McMahan
Eider Moore
Daniel Ramage
Seth Hampson
Blaise Agüera y Arcas

title: Communication-efficient learning of deep networks from decentralized data

venue: arXiv preprint arXiv:1602.05629,

authors:
Yiqiang Chen
Xin Qin
Jindong Wang
Chaohui Yu
Wen Gao

title: Fedhealth: A federated transfer learning framework for wearable healthcare

venue: IEEE Intelligent Systems,

authors:
Mingshu Cong
Xi Weng
Han Yu
Jiabao Qu
Siu Ming Yiu

title: Optimal procurement auction for cooperative production of virtual products: Vickrey-clarke-groves meet cremer-mclean, 2020

authors:
Vukasin Felbab
Péter Kiss
Tomás Horváth

title: Optimization in federated learning

venue: In ITAT,

authors:
Robin C Geyer
Tassilo Klein
Moin Nabi

title: Differentially private federated learning: A client level perspective

venue: arXiv preprint arXiv:1712.07557,

authors:
Stephen Hardy
Wilko Henecka
Hamish Ivey-Law
Richard Nock
Giorgio Patrini
Guillaume Smith
Brian Thorne

title: Private federated learning on vertically partitioned data via entity resolution and additively homomorphic encryption

venue: arXiv preprint arXiv:1711.10677,

authors:
Sepp Hochreiter
Jürgen Schmidhuber

title: Long short-term memory

venue: Neural computation,

authors:
Shaoxiong Ji
Shirui Pan
Guodong Long
Xue Li
Jing Jiang
Zi Huang

title: Learning private neural language modeling with attentive aggregation

venue: In 2019 International Joint Conference on Neural Networks (IJCNN),

authors:
Jiawen Kang
Zehui Xiong
Dusit Niyato
Shengli Xie
Junshan Zhang

title: Incentive mechanism for reliable federated learning: A joint optimization approach to combining reputation and contract theory

venue: IEEE Internet of Things Journal,

authors:
Hyesung Kim
Jihong Park
Mehdi Bennis
Seong-Lyun Kim

title: Blockchained on-device federated learning

venue: IEEE Communications Letters,

authors:
Jakub Konečnỳ
H Brendan McMahan
Felix X Yu
Peter Richtárik
Ananda Theertha Suresh
Dave Bacon

title: Federated learning: Strategies for improving communication efficiency

venue: arXiv preprint arXiv:1610.05492,

authors:
Yunlong Lu
Xiaohong Huang
Yueyue Dai
Sabita Maharjan
Yan Zhang

title: Blockchain and federated learning for privacy-preserved data sharing in industrial iot

venue: IEEE Transactions on Industrial Informatics,

authors:
Brendan McMahan
Eider Moore
Daniel Ramage
Seth Hampson
Blaise Aguera y Arcas

venue: In Artificial Intelligence and Statistics,

authors:
Jianji Ren
Haichao Wang
Tingting Hou
Shuai Zheng
Chaosheng Tang

title: Federated learningbased computation offloading optimization in edge computing-supported internet of things

venue: IEEE Access,

authors:
Guan Wang
Charlie Xiaoqian Dang
Ziye Zhou

title: Measure contribution of participants in federated learning

venue: IEEE International Conference on Big Data (Big Data),

authors:
Zirui Xu
Zhao Yang
Jinjun Xiong
Jianlei Yang
Xiang Chen

title: Elfish: Resource-aware federated learning on heterogeneous edge devices

authors:
Qiang Yang
Yang Liu
Tianjian Chen
Yongxin Tong

title: Federated machine learning: Concept and applications

venue: ACM Trans. Intell. Syst. Technol.,

authors:
Yufeng Zhan
Peng Li
Zhihao Qu
Deze Zeng
Song Guo

title: A learning-based incentive mechanism for federated learning

heading: 1 INTRODUCTION

text:
There are lots of data generate, collect, and access every day by smart terminals. But cause of the privacy of these data, it is usually difficult to use them. Such as the language model to predict the next word or even entire reply(Ion et al., 2016).
The emergence of federated learning breaks this data barrier. It can use agent computing power to conduct model training while maintaining data localization and privacy protection, and obtain an excellent global model.
But in a commercial federation, each agent should get corresponding rewards based on its contribution to the model, not the same rewards. There are many methods for contribution measurement. Such Wang et al. (2019) measured the contribution of each group features in vertical federated learning, and Zhan et al. (2020) proposed an incentive mechanism to make each agent willing to contribute better data. But most of them need to consume large computing resources and many methods are calculated offline.
In order to address this problem, this paper proposes a method for obtaining the contribution of each agent in real time with a small amount of calculation in horizontal federated learning.
Our contributions in this paper are as follows:
• We propose a method to measure agents’ contributions and compare this method with Shapley Value.
• The method we propose is sensitive to data volume and data quality, and can be used for mutual comparison between agents.
• In the training process, the contribution to each agent can be obtained in real time, with low computational complexity.

heading: 2 RELATED WORK

text: In this section, we will introduce the current research situation and application of federated learning, as well as the existing problems of reward distribution in the commercialization of federated learning framework, and introduce the necessity and importance of our work.

heading: 2.1 FEDERATED LEARNING

text:
Distribute the training data on each mobile device to maintain the localization of the data, instead of transmitting the data to the central server, updating the model locally, and uploading the update results to the server. While maintaining data localization and privacy, it can aggregate the data of each agent. The central server collects agent data and uses FedSGD, FedAVG(Brendan McMahan et al., 2016) and other algorithms to maintain the central model in combination with the different optimizer(Felbab et al., 2019), and sends the updated model to each agent. During the transmission process, methods such as homomorphic encryption are used to protect the security of data transmission and maintain the continuous iterative update of the model. This method is federated learning.
At present, for different datasets, federated learning framework can be classified into horizontal federated learning, vertical federated learning, and federated transfer learning(Yang et al., 2019). Horizontal federated learning is suitable for situations where the data provided by the agents has more of the same characteristics. In (McMahan et al., 2017), Google proposed a solution to update the horizontal federated learning model in Android phones. vertical federated learning is suitable for situations where there is less feature overlap but more user id overlap. Hardy et al. (2017) proposed a vertical federated learning method for training a logistic model. Chen et al. (2020) proposed the FedHealth method, which uses federated learning to aggregate data and uses transfer learning to obtain a personalized model.
Under the framework of federated learning, there are many different directions of research. Aono et al. (2017) use the data transmitted by the homomorphic encryption agents and the central server for model training, which further strengthens the privacy of the agents’ data. In (Lu et al., 2019; Kim et al., 2019), data verification is carried out in conjunction with the blockchain to prevent the gradient information from being maliciously tampered with. Konečnỳ et al. (2016) research on reducing the consumption of communication resources in federated learning.

heading: 2.2 APPLICATION AND COMMERCIALIZATION OF FEDERATED LEARNING

text:
Since federated learning was proposed, federated learning has been successfully applied to more and more scenarios. When considering data privacy issues, many companies will choose to use federated learning to protect data privacy to achieve cooperation. Such WeBank has successfully used federated learning in bank federations for credit evaluation and other financial aspect. In (Ren et al., 2019), federated learning is applied to dynamic IoT system.Lu et al. (2019); Kim et al. (2019)
use blockchain for data verification as an alternative to the commercialization of federated learning. Bao et al. (2019) achieved model commercialization by providing a healthy marketplace for collaborative-training models.
In the commercialization process of federated learning, the method of using this decentralized training model provides privacy, security, supervision and economic benefits, but also brings many fairness and systemic challenges.

heading: 2.3 CONTRIBUTION MEASUREMENT AND INCENTIVE MECHANISM IN FEDERATED LEARNING

text:
Wang et al. (2019) proposed grouped instance influence to measure contribution of agents in horizontal federated learning and grouped feature importance in vertical federated learning which equitably and accurately realized the measurement of contributions of all agents. But this method cannot take into account the amount of data.
Geyer et al. (2017) proposed a federated learning optimization algorithm based on differential privacy protection of the agents that can hide the contribution of the agents. In order to balance the contribution of the soft training model and ensure the collaborative convergence, Xu et al. (2019) proposed the corresponding parameter aggregation scheme. Kang et al. (2019) used the weighted subjective logic model to design a reputation-based worker selection plan to achieve reliable federated learning. In (Zhan et al., 2020), designed an incentive mechanism based on deep reinforcement learning to determine the optimal pricing strategy for parameter servers and the optimal training strategy for agents.
Our work focus on real-time contribution measurement mechanism for each agent in vertical federated learning. Current research has problems such as poor real-time performance and high resource consumption by contribution measurement methods. The method proposed in this paper can obtain real-time contribution measurement to each agent in the process of federated learning. At the same time, it verifies that the method proposed in this paper has high sensitivity on data quantity and data quality.

heading: 3 METHODOLOGY

text: In order to solve the above issues, we propose a corresponding method which does not need to obtain agents data and data scale, but can measure agents contribution. This method of measuring agents contributions is suitable for evaluating agents in real time.

heading: 3.1 FEDERATED LEARNING

text:
In this section, we will introduce the basic framework of federated learning and how parameters are updated.
Federated learning is a distributed learning method in which the server maintains an central model and distributes it to individual agents. To solve privacy problems, the server does not need to obtain the data of the agents, so the computing power of the agents is used to learn at the agents local environment.
The server will set up a fraction C, to select the agents proportionally for the server’s central model update. Then the selected agents’ updated model parameters are uploaded to the server for parameter updating of the server model. It is then distributed to the each agent to improve the model of the agents.
In this way, it continue to improve the central model of the server and the local model of the agents. Under the premise of ensuring the correctness and privacy of the agents, using this method can make full use of the computing power of the agents and a large amount of private data for learning, and at the same time maintain an excellent global model.

heading: 3.2 ATTENTION AGGREGATION

text:
In this section, we will introduce a FedAtt algorithm proposed by Shaoxiong Ji, Shirui Pan, Guodong Long et al.Ji et al. (2019), which is outperformed to the common FedAvg and FedSGD algorithm. The proposed attention mechanism in Federated Learning is used after the agents transmitted the parameters to the server, the server uses Formula equation 1 to calculate the value of attention α of each layer parameter that should be allocated to the agent, and multiplies each layer parameter by the corresponding attention to update the central model. The server integrates the update parameters passed by the agents according to theirs own parameter information, and finally completes an update of the central model through the communication between the server and the agents.
αlk = softmax(s l k) = es l k∑
i e sli
(1)
The slk is the norm difference from the central model. We use w l to represent layer l parameters of the server and wlk for lth layer parameters of the agent model k. So the definition of s l k is as follows
slk = ||wl − wlk||p (2)
So for m agents that are selected to update the central model, the method of updating becomes
wt+1 ← wt − m∑
k=1
5(wkt ) = wt − m∑
k=1
αk(wt − wkt ) (3)
To protect the agents’ data privacy, you can add the randomized mechanism before the agent passes parameters to the server. Randomly generate a random vector that obeys the standard distribution N (0, σ2), multiply the corresponding weight β, the results of the final update parameters as shown in Formula equation 4
wt+1 ← wt − m∑
k=1
αk(wt − wkt + βN (0, σ2)) (4)
The implementation process of the whole algorithm is shown as Algorithm 1
Algorithm 1 Attentive Federated Optimization 1: l is the ordinal of neural network layers; is the stepsize of server optimization 2: Input: server parameters wt at t, agents parameters w1t+1, w2t+1, · · · , wmt+1 at t+ 1 3: Output: server parameters wt+1 at t+ 1 4: Initialize attention α = {α1, α2, · · · , αm} 5: for each layer l in model do 6: for each agents k ∈ St from 1 to m do 7: slk = ||wl − wlk||p 8: end for 9: αlk = softmax(s l k) = es l k∑
i e sl i
10: end for 11: αk = {α1k, α2k, · · · , αlk} 12: wt+1 ← wt − ∑m k=1 αk(wt − wkt + βN (0, σ2))

heading: 3.3 AGENTS CONTRIBUTIONS

text:
In this section, the method for measuring agents contribution proposed in this paper will be introduced in detail.
Before calculating each agent contribution, we make the following assumptions: we think each terminal does not tamper with the updated gradient itself during the previous process of passing and updating parameters between the server and the agents.
The agents contribution to the server should be calculated after each layer of agents’ parameter attention. When the server has calculated the attention of agent K, it can calculate the impact of this agent K on the server model parameters in the T update. We define impkt as follows:
impkt = αk(wt − wkt + βN (0, σ2)) + γ · impkt−1 (5)
where γ ∈ (0, 1) is forgetting coefficient, because large variations in the early stage of the training model, the model is not stable, and the impact of the previous endpoint should be reduced after multiple iterations. If the agent k is not in the selected M agents this time, we think that the impact of the agent t round is impkt = imp k t−1, that is, only the number of rounds that the agent participates in the update is calculated. Note that the server’s attention to the agents are the attention of each layer, so when calculating the impact it also calculates the impact of each layer by reweighting and averaging.
We will be each agents attention MinMaxScaler normalized limited range, then Softmax, to obtain the contribution of each agent. We use conkt to represent the contribution of agent k at t time. The whole method of measuring agents contribution proposed in this paper is shown in Algorithm 2
Algorithm 2 Measure Agents Contributions 1: Input: l is the ordinal of neural network layers; is the stepsize of server optimization, server
parameterswt at time t+1, agents parametersw1t+1, w 2 t+1, · · · , wmt+1 at time t+1, γ is forgetting
coefficient, St is selected agents set. 2: Output: Agents contributions cont at t 3: for each agents k from 1 to K do 4: if k ∈ St then 5: impkt = αk(wt − wkt + βN (0, σ2)) + γ · impkt−1 6: else 7: impkt = imp k t−1 8: end if 9: end for
10: cont = softmax(MinMaxScaler(impt))

heading: 4 EXPERIMENT

text: In this section, we will introduce the verification experiments performed for our proposed agents’ contribution measurement method.

heading: 4.1 EXPERIMENTAL ENVIRONMENT AND CONFIGURATION

text: The system used in our verification experiment is Ubuntu 18.04 LTS, the backend used is the GPU version of Pytorch, with the NVIDIA GTX1660Ti GPU acceleration for model calculation. The GRU-based agents model used the language processing model of RNN. For a detailed model description, see the subsection C under this Section.

heading: 4.2 DATASET

text: We perform experimental verification on public language datasets of Penn Treebank1 to verify the effectiveness of our proposed algorithm and its sensitivity to special variables.

heading: 4.3 MODEL

text:
In natural language processing, the LSTM model is often used for processing. We used a smaller GRU-based agent language model. First take texts as input, and convert words into word vectors according to a pre-built dictionary. The converted word vector is then used as an input to the LSTM model, and the prediction result is finally output.
1Penn Treebank is available at https://github.com/wojzaremba/lstm/tree/master/data

heading: 4.4 PARAMETERS

text: This section enumerates the meanings and values of the various parameters mentioned. At the same time, we believe that the data of each agent is independent and identically distributed. The parameters and their descriptions are listed in Tab. 1

heading: 4.5 EXPERIMENTAL RESULTS

text:
We use testing perplexity as an indicator of the evaluation model. In information theory, perplexity is a measurement of how well a probability distribution or probability model predicts a sample. The perplexity of a discrete probability distribution is defined as:
ppl(x) = 2H(p) = 2− ∑ x p(x)log2p(x) (6)
In the above formula, H(p) is the expected probability distribution. If the prediction result of our model is m(x), then the perplexity of the language model is defined as:
ppl(x) = 2H(p,m) = 2− ∑ x p(x)log2m(x) (7)
Obviously, H(p) 6 H(p,m). Therefore, the smaller the perplexity, the more representative the probability distribution can be to better predict the sample distribution.
And we compared with the results of Shapley Value evaluation. Shapley Value is originated from coalitional game theory and has proven theoretical properties. It provides a way to measure the impact and contribution of various agents. The definition of Shapley Value is:
ϕi(x) = ∑
Q⊆S\{i}
|Q|!− (|S| − |Q| − 1)! |S|! (∆Q∪{i}(x)−∆Q(x)) (8)
S is the set of all agents, Q ⊂ S = 1, 2, · · · , n is a subset of the agent set S, i is the index of the agents, || represents the size of the set, ∆Q(x) = impQ denotes the impact of agent set Q. Because the complexity of directly calculating Shapley value is too high, we use the following estimation method to get the ϕi(x) of each Agents
ϕi(x) = 1
M M∑ m=1 (∆Qm∪{i}(x)−∆Qm(x)) (9)
where M is the number of iterations. ∆Qm(x) denotes the impact of random set Qm. Finally, all the data obtained by Shapley Value is subjected to the same regularization processing as ours.
To ensure the fairness of the measurement contribution in this experiment, we ensure that each agent is drawn the same number of times.
We randomly divided the data in the dataset, and distributed them evenly to all agents. Then, the corresponding special processing is performed on the data of the last few agents: reduce the amount of data by 30% and 70%, randomly generate the word sequence. Comparing the measurement results after special processing with the results when unprocessed data, it is concluded that the sensitivity of variables such as data size and data quality is evaluated.

heading: 4.5.1 EXPERIMENTAL RESULTS OF NORMAL MEASUREMENT

text: We did not do any special treatment to any agents, and randomly divided the data set into each agent. The result is shown in Fig. 2. In the figure, the abscissa is the index of agents, and the ordinate is the agents’ contribution ratios. It can be seen that the deviation of each agent is not large, only individual agents are too high or too low. And the measurement result is similar to Shapley Value.

heading: 4.5.2 EXPERIMENTAL RESULTS WITH RANDOMIZE THE WORD SEQUENCE

text: In this experiment, we modified the datasets of the next four agents into random word sequences. These should be regarded as dirty data by the model, so as to get a smaller return. The results are shown in Fig. 3. Both the method proposed in this paper and the Shpaley Value method can identify these bad agents and give them a small contribution.

heading: 4.5.3 EXPERIMENTAL RESULTS WITH REDUCE THE AMOUNT OF DATA

text:
To demonstrate the sensitivity of our measurement method to the amount of data, we performed the following experiments.
In this experiment, we reduced the data amount of the last 4 agents by 70%, and the data amount of other agents remained unchanged. As you can see in Fig. 4, the contribution of the specially treated agents are significantly reduced. But it is relatively not obvious in the measurement results of Shapley Value.
In order to reflect the relative relationship between the amount of data, we processed the data of the last 4 agents: agents with index 16 to 17 reduced the amount of data by 30%, agents with index 18 to 19 reduced the amount of data by 70%.
It can be seen in Fig. 5 that the method proposed in this paper can reflect the reduced amount of data, while Shapley Value cannot show it well. Note that the method proposed in this paper can better reflect the difference of smaller data volume.

heading: 5 CONCLUSION

text:
A reasonable and fair completion of the measurement of the contribution of each participant in federated learning and the establishment of an incentive mechanism are essential for the development of federated learning. In this paper, we use a “FedAtt” method to train the models on Penn Treebank dataset. At the same time, we perform several special processing on the agents data, and compare the measurement results of the special processing with the measurement results of the unprocessed data. Experimental results show that we use this method to reasonably establish a measurement contribution mechanism to evaluate the sensitivity of indicators such as data size and data quality. The data is evaluated and trained by this method, and the calculated results are real-time and fast, and the contribution rate reflects accurately.
For future work, It is hoped that a better algorithm can be found that can more accurately measure the contribution to malicious agents, and at the same time prevent the contribution confusion caused by more than half of malicious agents. It also should consider that the negative contribution of agents measurement and the comprehensiveness of the measurement. Preliminary classification (i.e., positive and negative) is carried out at the end to avoid attacks under the federated learning framework mechanism(Bagdasaryan et al., 2018); at the same time, the concept of game theory is introduced, such as the establishment of a PVCG mechanism on the supply side(Cong et al., 2020). Improvements in these directions will promote the implementation and application of the federated learning incentive mechanism.