Distant supervision has been widely used in relation extraction tasks without hand-labeled datasets recently. However, the automatically constructed datasets comprise numbers of wrongly labeled negative instances due to the incompleteness of knowledge bases, which is neglected by current distant supervised methods resulting in seriously misleading in both training and testing processes. To address this issue, we propose a novel semi-distant supervision approach for relation extraction by constructing a small accurate dataset and properly leveraging numerous instances without relation labels. In our approach, we construct accurate instances by both knowledge base and entity descriptions determined to avoid wrong negative labeling and further utilize unlabeled instances sufficiently using generative adversarial network (GAN) framework. Experimental results on real-world datasets show that our approach can achieve significant improvements in distant supervised relation extraction over strong baselines. Relation extraction aims to identify relations for a pair of entities in a sentence to construct relation triples like [ Steve Jobs, Founder, Apple ]. It has been well studied by supervised approaches with hand-labeled data. However, supervised methods are limited to costly hand-labeled training sets and  Corresponding authors: Weijia Jia, Hai Zhao, { jia-wj, zhaohai } @cs.sjtu.edu.cn. This work is supported by National China 973 Project No. 2015CB352401; Chinese National Research Fund (NSFC) Key Project No. 61532013 and No. 61872239. 0007/2018/A1, DCT-MoST Jointproject No. 025/2015/AMJ,FDCT,SAR Macau, China, and University of Macau Grant Nos: MYRG2018-00237-RTO, CPG2019-00004-FST and SRG2018-00111-FST, National Key Research and Development Program of China (No. 2017YFB0304100), Key Projects of National Natural Science Foundation of China (U1836222 and 61733011), and Key Project of National Society Science Foundation of China (No. 15-ZDA041). hard to be extended to large-scale relations. To break the bottleneck of hand-labeled training set, distant supervision (Mintz et al., 2009) automatically construct datasets with knowledge bases. It assumes that if two entities have a known relation in a knowledge base, all sentences that mention these two entities will probably express the same relation and can be called positive instances. At the same time, it treats sentences as negative instances whose entity pairs do not have a known relation in knowledge bases. Due to the strong assumption, instances are likely to be mislabeled. To alleviate the wrong labeling problem, distant supervised methods have been implemented with multi-instance learning and neural networks (Riedel et al., 2010; Hoffmann et al., 2011; Zeng et al., 2015; Lin et al., 2016, 2017). However, previous works focus on positive instances and few methods have addressed the issue of false negative instances. The false negative instances, which contain true relations, are misclassified sentences in the negative set due to the incomplete nature of knowledge bases. For example, over 70% of people included in Freebase have no known place of birth (Dong et al., 2014). As shown in Figure 1, S1 presents the relation place of birth , while it is labeled as a negative instance. The other three sentences are mislabeled in the same way. The missing relation triples in knowledge bases yield numbers of false negative instances in the automatically labeled dataset. These instances will not only mislead the training method to an unreliable convergence but also make the measurement criteria inaccurate in the testing process. Table 1 compares the precision of automatic and manual evaluation methods for top N predictions by the previous relation extractor (Lin et al., 2016) on the NYT dataset. From the table, we can see that manual evaluation is more precise than automatic evaluation by over 19.8%. The huge bias mainly comes from false negative instances in the testing set, which severely limits the upper bound of accuracy for relation extraction. Therefore, handling false negative instances is a pivotal issue to improve the performance of distant supervised relation extraction. To alleviate the effect of false negative instances, there are two possible ways. One is improving the accuracy of the automatically labeled dataset, and the other is properly leveraging unlabeled instances which cannot be labeled as positive or negative. The former way is to construct an accurate dataset by filtering credible negative instances but limited by high annotation cost and the resulting dataset size. The latter way is to train relation extraction models with abundant unlabeled instances but restricted by the prerequisite of an accurate dataset used as ground truth. Therefore, we propose a novel semi-distant supervised approach by integrating both ways to decrease the influence of false negative instances for better relation extraction. In our approach, we additionally use entity descriptions together with a knowledge base to construct an accurate dataset. Supervised by the dataset as ground truth, to effectively exploit numbers of unlabeled instances, we train our relation extractor using a generative adversarial network (GAN) framework. In detail, We propose a three-player min-max game to generate proper relation representations for unlabeled instances in an adversarial way which minimizes the difference between labeled and unlabeled data and maximizes the probability of distinguishing from each other at the same time. The experiments demonstrate that our approach is effective and outperforms the state-of-the-art work. In summary, we make the following major contributions:  We propose a novel semi-distant supervision method for relation extraction to alleviate the influence of false negative instances. To the best of our knowledge, we are the first to generate valid relation representations for sentences by an adversarial algorithm. Numbers of unlabeled instances are used to improve the performance of relation extraction. Moreover, our generative adversarial training strategy is proved effective on an additional sentiment classification with sixteen real-world datasets. We construct a new accurate dataset for relation extraction extended from the NYT dataset. Our approach increases the area of the Precision-Recall curve from 0.39 to 0.56 over the baselines. To extend relation extraction to large-scale datasets, distant supervision (Mintz et al., 2009) automatically labeled training sets with knowledge bases such as Freebase. Although this method is working well for large-scale relation extraction, it is trapped in the wrong labeling problem for positive instances. To deal with this problem, multi-instance learning was combined with distant supervision (Riedel et al., 2010; Hoffmann et al., 2011). Inspired by the pioneering work, a series of later studies were conducted to further improve distant supervised relation extraction with methods such as multi-instance multi-label learning (Surdeanu et al., 2012), graph model for label generation (Takamatsu et al., 2012), partial supervision (Angeli et al., 2014), matrix completion with low-rank criterion (Fan et al., 2014) and modeling the neighbor consistency with Markov logic (Han and Sun, 2016). However, the performance of the methods mentioned above strongly depends on the quality of human-designed features. With the development of neural models, relation features with semantic meaning can be accurately, simply and automatically extracted. Zeng et al. (2015) proposed the first neural relation extraction with distant supervision. Mnih et al. (2014), Lin et al. (2016), Zhang et al. (2018), Han et al. (2018) and Du et al. (2018) showed that attention model could improve the accuracy of neural relation extraction. Another similar work (Ji et al., 2017) assigned better attention weights with extra data like entity descriptions. DSGAN (Qin et al., 2018a), a GAN-based method, was also used to recognize true positive instances from noisy datasets. To further alleviate the effect of wrong labeling problem, soft-label training algorithm (Liu et al., 2017b), reinforcement learning methods (Feng et al., 2018; Qin et al., 2018b) and additional side information (Vashishth et al., 2018; Wang et al., 2018) have been used. Most recently, a few methods focused on the pre-training embeddings for word tokens and relations including adversarial training (Wu et al., 2017), transfer learning (Liu et al., 2018) and relation decoder (Su et al., 2018). All the above methods mainly pay attention to positive instances. Whereas, few studies work on the quality of negative instances, which is exactly the focus of this paper. We effectively construct a reliable dataset with both entity descriptions and a knowledge base, and thus propose a novel semi-distant supervised method to extract relations precisely. In the distant supervised relation extraction paradigm, all sentences labeled by a relation triple constitute a bag, and each sentence is called an instance. The relation triple is described as [ head, relation, tail ], where head and tail are both entities. We extract relation features from labeled training bags and then predict relations for unseen bags in the test set. This section presents our method about constructing an accurate dataset, the sentence encoder for relation representation and the semi-supervised way for relation extraction. To reduce false negative instances, we construct a new reliable dataset extended from a widely used dataset NYT (Riedel et al., 2010) with entity descriptions. Entity descriptions are crawled from Wikipedia with entity name matching 1 . We assume that if an entity is relevant to another entity, its name is possibly mentioned in the description of the other entity. For example, the entity Apple Inc. is mentioned in the description of Steve Jobs . To verify the assumption, we count the number of all the accurate positive instances whose entity descriptions mention the other entity name in the NYT corpora. There are 163,108 positive sentences in total, in which 161,392 ones contain entity pairs that related to each other in their descriptions at least once. In other words, over 98.9% instances in positive set fitting our assumption indicates that most entity pairs in positive instances contain each other in their descriptions. Therefore, a former negative instance has a big chance to be credible negative if any of its entities is not mentioned in the description of the other one. Excluding instances that contain entity pairs related to each other in their descriptions, we can obtain more confident negative instances. Finally, we filter credible positive and negative instances from the dataset, and the other instances are unlabeled ones that cannot be labeled as positive or negative. We pre-train input embeddings of word tokens including word and position embeddings. Word embeddings are distributed representations that map each word to a vector word  R w , where the parameter w indicates the dimension of the vector. The vectors are trained in advance by word 2 vec in the setting of Skip-gram (Mikolov et al., 2013). In the task of relation extraction, the relative positions of input tokens are important information. 1 One entity name may refer to multiple entities which have their own pages. In our work, all the matched pages are collected together to obtain its description. Position embeddings are defined as the combination of the relative distances from the current word to head and tail . For instance, the relative distances from [ co-founder ] to [ Steve Jobs ] and [ Apple ] are respectively 3 and -6 in the sentence Steve Jobs was the co-founder and the CEO of the Apple . We encode distances to vectors position  R p , where p is the dimension. The position embeddings are initialized randomly and updated in the training process. Finally, word embeddings and position embeddings are concatenated together to feed the neural model. We denote all the words in an instance as an initial vector sequence b  = { x 1 ,    , x i ,    , x q } , where x i  R w + p and q is the number of words in the instance b  . Convolutional Neural Network (CNN) is a widely used structure for sentence encoder as shown in Figure 2. With the input embeddings, the convolutional layer extracts local features with a sliding window of length k over the input tokens. In the figure, we extract local features from 3 ( k = 3 ) adjacent word tokens with dot production between convolutional kernels and input embeddings. The convolutional kernels are weight vectors represented by W  R d  k ( w + p ) and the number of kernels is d . In summary, the convolutional operation follows the equation, f ij = W i  [ x j  1 ; x j ; x j +1 ] , (1) where [ x ; y ] denotes the vertical concatenation of x and y . f ij presents j -th value of the i th filter, where i and j are in range [1 , d ] and [1 , q ] respectively. Out-of-range input values such as x 0 and x q +1 are taken to be zero. A max-pooling operation selects the most important features of each f i with f  i = max ( f ij ) , where f   R d . Furthermore, PCNN (Zeng et al., 2015) improves the max-pooling operation with a piecewise method whose outputs of convolutional fil-ters are divided into three segments by head and tail entities. Therefore, the max pooling procedure is performed in three segments separately. Then, we summarize f  to  h by a non-linear function such as the hyperbolic tangent. The final feature vector  h is fed into output layer after the softmax method  p = softmax ( W r  h + b r ) , where W r  R z  d and b r  R z are variables,  p  R z is the estimated probability for each class and z is the number of relations. A cost function for one instance is the negative log-likelihood of the relations, J truth ( p, y,  ) =  1 z z (cid:88) j =1 y j log  p j , (2) where y  R z is the one-hot represented ground truth and  presents all the parameters. The architecture of our semi-distant supervision is shown in Figure 3. To sufficiently utilize the reconstructing dataset including accurately labeled instances and unlabeled ones, we propose a generative adversarial training strategy, which transforms unlabeled instances ( x ul ) to labeled data ( x l ) space by generating valid relation representations ( x gen ) and making the distribution of labeled instances p ( x l ) equal to that of generative data p ( x gen ) in relation space 2 . Inspired by Goodfel-low et al. (2014), we further devise a three-player min-max game to generate valid data distribution p ( x gen ) with sentence encoder, generative and discriminative modules. The generative module minimizes the difference of p ( x l ) and p ( x gen ) , and the discriminative module maximizes the probability of distinguishing from each other at the same 2 p ( x l ) and p ( x gen ) represents the data distribution of labeled and generative instances. time. Sentence encoder is proposed as the third player, which extracts relation features from all the instances and produces a pre-trained relation representation for unlabeled instances. With the sentence encoder, we can control relation features contained in the generated representations. Therefore, the discriminative module D will try to distinguish labeled data from generative data, while the generative module G makes p ( x gen )  p ( x l ) . In addition, the sentence encoder S extracts relation features with all the training instances p all . The training procedure is a three-player min-max game as the following equation, min S,G max DV ( S, D, G ) = E x  p all [log S ( x )] + E x  p xl [log D ( x )] + E c  p xgen [log(1  D ( G ( c )))] , (3) In generative adversarial training, the discriminative module is trained by maximizing the gap between labeled data and generative data with the following equation, JD ( x, c,  d ) = log D ( x ) + log(1  D ( G ( c ))) , (4) where x and c are instances from accurately labeled set ( x l ) and unlabeled set ( x ul ) respectively. d presents parameters for the discriminator. D ( x ) and D ( G ( c )) are defined as follows, D ( x ) =  ( W d  h x ) , (5) D ( G ( c )) =  ( W d ( h c + W g )) , (6) where W d and W g are variables for discriminative and generative modules respectively. is the sigmoid function. The generative module is trained to make the generated relation representations more similar to real ones by the following loss function, where  g presents parameters. JG ( c,  g ) = log(1  D ( G ( c ))) (7) Finally, we train our sentence encoder S by optimizing the following loss function, JS ( x, c,  s ) =  1 z z (cid:88) j =1 y lj log  p ( y j | x )  1 z z (cid:88) j =1 y gj log  p ( y j | G ( c )) , (8) where y g means a one-hot vector which labels the most possible relation for unlabeled instances generated by the sentence encoder. s represents parameters of S . The complete training procedure for generative adversarial training is shown as Algorithm 1. 4 Experiments The experiments are proposed to answer the following three questions, 1) Is the proposed semi-distant supervision method effective for the task of relation extraction? 2) Is the constructed dataset credible enough? 3) Is the generative adversarial training helpful to relation extraction and other semi-supervised tasks? We conduct experiments on a widely used dataset NYT (Riedel et al., 2010) and its new version Accurate-NYT (A-NYT). A-NYT is a credible dataset filtered by our data construction module. We follow the previous work (Lin et al., 2016) to partition training and testing sets for NYT and A-NYT. Besides, we apply sixteen real-world datasets 3 (Liu et al., 2017a) to further verify the effectiveness of our generative adversarial training strategy on the task of sentiment classification. The dataset details are shown in Table 2. 3 The datasets are Amazon product reviews and movie reviews. On the dataset NYT and A-NYT, we evaluate our method in the classical held-out evaluation. It evaluates our models by comparing relation facts discovered from the test sentences with those in Freebase. Specifically, we report both the aggregate Precision-Recall (PR) curves and Precision at top N predictions (P@N) in our experiments. For the other datasets, we compute the precision of all the predictions. supervised relation extraction. Zeng et al. (2015) extracted relation features with piecewise convolutional neural network (PCNN). Lin et al. (2016) integrated PCNN with selective attention mechanism (PCNN+ATT). Wu et al. (2017) added adversarial noise at the level of the word embeddings (PCNN+ATT+AT). Liu et al. (2017b) relabeled the training instances dynamically by the relation extractor (PCNN+ATT+SL). Liu et al. (2018) shortened the training instances with the parser tree and pre-trained word embeddings with transfer learning, which is the latest state-of-the-art work. Self-Training (ST) is a semi-supervised method that can be integrated with PCNN+ATT for unlabeled data, which generates relation types for unlabeled instances with the model itself. In our experiments, we use the word 2 vec in the setting of Skip-gram to train the word embeddings on NYT set. To train our model efficiently, we iterate by randomly selecting a batch from the training set until convergence and apply sentence-level attention mechanism following the previous work (Lin et al., 2016). The parameter n and m are batch sizes for accurate and unlabeled datasets respectively. We update the gradient with adaptive moment estimation (Kingma and Ba, 2015). Furthermore, L 2 regularization and dropout (Srivas-tava et al., 2014) are adopted to avoid overfitting. Finally, we use a grid search and cross-validation to determine the optional parameters as shown in Table 3. The hyper-parameters s i , s j and s k are training steps for different modules of generative adversarial training. Since the other parameters have little effect on the results, we follow the settings as the previous work (Lin et al., 2016). The overall performance of our method compared with baselines for distant supervised relation extraction is shown in Table 4. We can see that our semi-distant supervised method achieves much better results than the baselines on all metrics. The huge improvement comes from both the accurate dataset and the effective training strategy which leverages unlabeled instances properly. In this section, we apply two previous methods Zeng et al. (2015) and Lin et al. (2016) on NYT and A-NYT. PR curves for NYT are reported in their papers, while PR curves for A-NYT come from our implementations of the two baselines. NYT and A-NYT share the same positive instances, while A-NYT set has less and credible negative instances. As shown in Figure 4(a), P@N 100 200 300 Mean PR Zeng et al. (2015) 72.3 69.7 64.1 68.7 0.33 Lin et al. (2016) 76.2 73.1 67.4 72.2 0.35 Wu et al. (2017) 81.0 74.5 71.7 75.7 0.34 Liu et al. (2017b) 87.0 84.5 77.0 82.8 0.34 Qin et al. (2018a) 78.0 75.5 72.3 75.3 0.35 Liu et al. (2018) 87.0 83.0 78.0 82.7 0.39 Our Method 96.0 93.5 93.0 94.2 0.56 Table 4: Overall performance at P@Ns(%) and PR curve areas methods on A-NYT always obtain better performance. The huge gap between PR curves is caused by false negative instances in NYT, which are not used for training and testing in A-NYT. To prove that results on A-NYT are according to the actual situation, we do manual evaluations at P@Ns. As shown in Table 5, the huge bias caused by false negative instances on NYT is dramatically alleviated on the dataset A-NYT. To further demonstrate the effectiveness of our training strategies, we compare Generative Adversarial Training (GAT) with other baselines on the partially labeled dataset A-NYT as shown in Figure 4(b). The figure gives the following insights, 1) PCNN+ATT+ST and PCNN+ATT+AT do not work well, which is caused by the low quality of unlabeled instances. 2) PCNN+ATT+SL works as well as our models at low recall rate because of its excellent ability to extract notable features. Unfortunately, it falls far behind all the baselines at high recall rate, which means it tends to converge to a local optimum. 3) Our model achieves solid PR curves at all range of recall rate. Meanwhile, we propose a detailed comparison of baselines with P@Ns and PR curve areas as shown in Table 6. From the table, we can see that our training strategy achieves much better result than the other baselines, which indicates that abundant unlabeled instances are helpful to extract relations only if used appropriately. Going deeper in the table, PCNN+ATT+SL works well at top predictions but obtains the worst PR curve. Our semi-distant supervised model with adversarial generations is useful for leveraging unlabeled instances properly. To verify the expandability of generative adversarial training, we conduct additional experiments on the task of sentiment classification. We implement our model and three baselines based on the Long Short-Term Memory (LSTM) network 5 . The results are shown in Table 7, from which we 4 Results on NYT are reported as their papers except Wu et al. (2017) and Qin et al. (2018a). Results of these two methods on NYT and all results on A-NYT are obtained by our implementations. 5 Our generative adversarial training strategy is model-independent, meaning that it could be applied to other neural models. see that, 1) Self-training obtains poor results compared with the basic LSTM model, which means they fail to utilize unlabeled data correctly. 2) Adversarial training improves the performance on only three of the datasets and performs poorly on others, which means they possibly rely on the quality of unlabeled data. 3) LSTM+GAT achieves better results than the baselines on most of the datasets because of generating high-quality representations for unlabeled sentences. In this paper, we propose a novel semi-distant supervision approach that is capable of jointly exploiting limited accurate and abundant unlabeled ones. We first construct a reliable dataset with a knowledge base and additional entity descriptions. With the dataset, the generative adversarial training strategy is proposed to deal with plenty of unlabeled instances, which generates valid relation representations. Our experiments show that the proposed approach achieves significant improvement over previous state-of-the-art baselines.
abstain abstain objective method result abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain objective method method objective objective objective objective objective abstain objective abstain other other other other other other other other other other other other other abstain objective abstain abstain method method method abstain method abstain result abstain method abstain result method method abstain abstain abstain abstain method abstain abstain method abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain method method method abstain abstain abstain method abstain abstain other abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain abstain method method abstain method abstain abstain abstain abstain abstain abstain abstain method method abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain other abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain method abstain abstain abstain objective objective abstain objective