Knowledge base (KB) embeddings have been shown to contain gender biases (Fisher et al., 2020b). In this paper, we study two questions regarding these biases: how to quantify them , and how to trace their origins in KB ? Specifically, first, we develop two novel bias measures respectively for a group of person entities and an individual person entity. Evidence of their validity is observed by comparison with real-world census data. Second, we use influence function to inspect the contribution of each triple in KB to the overall group bias. To exemplify the potential applications of our study, we also present two strategies (by adding and removing KB triples) to mitigate gender biases in KB embeddings. Gender biases have been shown to have noticeable presence in a wide range of NLP models. For example, we can observe that the word embedding of engineer is closer to he than she (Bolukbasi et al., 2016), and co-reference systems associate surgeon more with masculine pronouns than with feminine ones (Rudinger et al., 2018). These biases are brought to our models from training data by our algorithms. Hence, besides revealing the existence of gender biases, it is important to quantify them and explain their origins in data. Knowledge bases (KB, e.g. Freebase, Bollacker et al., 2007) provide accessible organizations of human knowledge by the form of triples. Each triple consists of a head entity, a relation, and a tail entity. For example, the fact that Marie Curie is a chemist is represented as (cid:104) Marie Curie , people.person.profession , chemist (cid:105) . KB embeddings encode these knowledge into dense vector representations. It is important to understand gender biases in KB embeddings for two major reasons. First, KB embeddings serve as sources of prior knowledge in many downstream NLP models (e.g. pre-trained language models, Zhang et al., 2019). Clearly, if biases exist in KB embeddings, they can easily propagate into these models, and drive these models more biased. Second, Garg et al. (2018) observe that word embeddings reflect biases in the training corpora, and hence our society. Likewise, we suspect KB embeddings to reflect biases encoded in KBs, as also suggested by Radstok et al. (2021). In this paper, we propose two novel gender bias measures for KB embeddings, one for a group of person entities ( group bias ) and the other for an individual person entity ( individual bias ). Furthermore, with influence function (Koh and Liang, 2017), we explain the origins of group bias at the fact triple level (i.e. how each triple in KB contribute to group bias). In practice, we use TransE (Bordes et al., 2013) to demonstrate our methods, for its popularity and simplicity. Nevertheless, most of our study can generalize to other embedding algorithms. Specifically, we make four contributions. First, regarding a group of person entities with a shared relation-tail pair (e.g. of the same occu-pation), using correlation analyses , we measure their gender biases by the differences between different genders' link prediction errors. Second, to understand the origins of the group bias, we use influence function to find its highly-influential triples in KB (i.e. triples that will change the bias most if being removed during training). Third, regarding a single person entity , using counterfactual analyses , we develop a bias measure by measuring the change of the link prediction error when we keep everything else the same and perturb its gender. To avoid the intractable computational cost of re-training, we propose to use influence function to approximate the results. Fourth, to further facilitate large-scale influence function based analyses, we derive a closed-form approximation of the Hessian matrix of TransE loss. We therefore improve the time complexity of computing influence function from O ( n ) (stochastic approximation) to (1) . Moreover, in further analyses, we show that both group and individual bias correlate well with real-world biases. We argue that this suggests the validity of our bias measures. We also show the accuracy of our influence function approximation by comparing with the brute-force strategy (i.e., leave-one-out re-training). Finally, to exemplify the applications of our study, we propose two simple de-biasing strategies, and demonstrate their effectiveness. Knowledge Base KB is a set of structural human knowledge represented by triples G = {(cid:104) h, r, t (cid:105)} , where h is a head entity, r is a relation type, and t is a tail entity. Moreover, these triples form a graph with entities as nodes (denoted by E , where e  E is an entity) and relations as edges. In this work, since we are particularly interested in person entities and their gender, we use (cid:104) s, r g , m (cid:105) or (cid:104) s, r g , f (cid:105) to represent a person s with gender male or female, where r g is the relation of gender. 1 TransE The entities and relations in KB can be represented with embedding vectors. These embeddings can serve in many NLP task as a source of prior knowledge. In this work, we focus on the widely used TransE (Bordes et al., 2013). Given a triple (cid:104) h, r, t (cid:105) , the key idea of TransE is to make vectors of h , r and t close to each other in the sense of small link prediction error. Concretely, TransE embeddings are learned by minimizing a margin-based ranking loss, L = (cid:80) (cid:104) h,r,t (cid:105)G [ m +  ( h, r, t )   ( h (cid:48) , r, t (cid:48) )] + , (1) where m is a scalar margin and  is a distance measure. The lower  ( h, r, t ) is, the more likely (cid:104) h, r, t (cid:105) forms a fact. h (cid:48) and t (cid:48) are two randomly sampled entities. The triple (cid:104) h (cid:48) , r, t (cid:48) (cid:105) is called a negative sample because it is not in G . This loss function basically says that the dissimilarity of a positive triple (cid:104) h, r, t (cid:105) should be smaller than a negative sample by a margin m . 2 Specifically, in this paper, we take  to be the L 2 -norm distance 1 We operate with binary gender here, because it is naturally encoded in KB. 2 For simplicity, we consider only linear loss in the rest of this paper. This is a feasible choice both empirically and theoretically in our analyses. Empirically, in experiments we observe that the link prediction errors of all triples converge  ( h, r, t ) = (cid:107) h + r  t (cid:107) 22 , where h , r , t  R d are the embeddings of h , r and t , respectively. In this paper, we use Freebase's (Bollacker et al., 2007) subset FB5M (Bordes et al., 2015) as the KB for training TransE embeddings and performing our analyses. See Appendix A for detailed setup. Influence Function (Cook and Weisberg, 1982; Koh and Liang, 2017) provides an efficient way to approximate each training sample's impact on correctly predicting a test sample. Formally, let L ( z,  ) be a convex loss function on a training set { z i } ni =1 with parameters  . The empirical risk minimizer (ERM) is given by   = arg min  1 n (cid:80) ni =1 L ( z i ,  ) . We are interested in a training sample z 's impact on   , with a weight of  . In this case, the new ERM is given by   z, = arg min  1 n (cid:80) ni =1 L ( z i ,  ) + L ( z,  ) (Note that if  =  1 n , it equals to removing z ). Influence function provides an efficient method of approximating the difference between   z, and   , without retraining the model,   z,      I up,param ( z ) , (2) where I up,param ( z ) def =  H  1     L ( z,   ) . H  = 1 n (cid:80) i  2 L ( z i ,  ) is the Hessian matrix of the original loss function. Moreover, we are interested in the change of the test performance, which is a function F of the test sample z test and the model parameter (LHS). By applying chain rule to F and Equation 2, we can obtain the difference of test performance. Formally, F (  z, , z test )  F ( , z test )   I up ,F ( z, z test ) , (3) where I up ,F ( z,z test ) def =   F ( z test ,   ) (cid:62) I up,param ( z ) . Similarly, by splitting perturbation to first remove then add, we can also inspect the change of F when a training sample z is perturbed to z (cid:48) . Denote   z,z (cid:48) , = arg min  1 n (cid:80) ni =1 L ( z i ,  )  L ( z,  ) + L ( z (cid:48) ,  ) , and apply Equation 3 twice, we obtain F (   z,z (cid:48) , , z test )  F ( , z test )   I up ,F ( z (cid:48) , z test )   I up ,F ( z, z test ) def =  I pert ,F ( z, z (cid:48) , z test ) . Finally, besides single sample estimation, we are also interested in inspecting the influence of removing a group of training samples. In these cases, at a larger-than-margin value. Theoretically, when link prediction errors converge at values smaller than the margin, the gradients become 0. Its influence thus becomes 0, too. following Koh and Liang (2017), we simply add up the influence of each removed training sample. However, as noted by Koh and Liang (2017), when handling a group of samples, although influence function approximation still holds a strong correlation with the ground truth change of the parameters, the estimation can suffer from larger errors. In this section, based on link prediction, we take two views to quantify gender biases in KB embeddings. First, using correlation analysis, we take a macro view to inspect gender biases of a group of person entities (e.g., how gender influences the overall occupation prediction accuracy of a group of engineer entities). Second, under the framework of counterfactual analysis, we take a micro view to assess gender biases of an individual person entity (e.g., how a specific engineer entity's gender influences its occupation prediction accuracy). Afterwards, we build connections between them. In this following, we adopt occupation prediction as our running example. The reason is two fold. First, among all of the relations connected with person entities, occupation relation has the highest coverage rate (i.e. connect with the most person entities). Second, most previous relevant studies also focus on occupation. Our choice makes it easier to perform comparative studies (Garg et al., 2018; Fisher et al., 2020b). To see whether a group of entities exhibits bias, one direct solution is to deploy methods analog to those applied to analyze bias in word embeddings (Bolukbasi et al., 2016). For example, we can compute the projection of the TransE embedding of an occupation o to the difference between male and female entities (Bourli and Pitoura, 2020), B wo = o (cid:62) ( m  f ) , where m and f are the embeddings of male and female entity respectively. However, we argue that because TransE follows a different type of learning objective (link prediction style objective instead of the vector-similarity-based ones in word embedding algorithms), directly adopt existing settings may not fully explore the semantics of TransE embeddings. Therefore, we propose to detect group bias based on the correlation between genders and link prediction errors. Intuitively, given an occupation o , person entities of o 's privileged gender will link to o with lower errors than those of unprivileged gender. Formally, we define the group bias of o as B gr = 1 | F | (cid:80) s  F  ( s, r p , o )  1 | M | (cid:80) s  M  ( s, r p , o ) , where M and F are the sets of all male and female person entities with o respectively, and r p is the relation people.person.profession . The higher B gr is, the more o 's embedding is biased towards male. Table 1 lists B gr of some occupations, as well as the gender frequency of this occupation in KB. We make two observations. First, we observe the existence of gender biases in KB embeddings, and note their consistency with real-world biases. For example, engineer and nurse have more extreme bias scores respectively towards male and female, while singer and animator have more moderate ones (quantitative analyses in 4). Second, although the gender ratio of person entities has a great impact on B gr , it is not the only decisive factor. For example, animator has a gender ratio of 5.7:1, but its B gr is biased towards female. Inspecting the Origins of Biases The second observation motivates us to trace the origins of biases. More concretely, in the context of KB: how do different triples contribute to B gr ? To answer this question, we apply influence function (Equa-tion 3) with F = B gr and observe how removing a training triple changes the overall group bias score. One appealing property of TransE is that we are able to derive a closed-form Hessian matrix. Moreover, by further analyses, we can directly obtain a diagonal approximation of the Hessian matrix, and thus the Hessian inverse I up, param . Taking advantage of this, we can reduce the computation of I up , B gr to constant time complexity (w.r.t. training set size), which is much faster than the LiSSA algorithm (Agarwal et al., 2017) applied in (Koh and Liang, 2017), which requires O ( n ) time complexity to obtain a Hessian inverse approximation. Concretely, using basic calculus, we have the following lemma and remarks. We include their detailed proof and derivations in Appendix B. Lemma 1. Suppose we generate the corresponding negative sample of a positive sample (cid:104) h, r, t (cid:105) by randomly choosing h or t and corrupting it to a random entity in E , we can derive the closed-form Hessian matrix of TransE with entries E H   =   e e (cid:48) r r (cid:48) ... ... ... ... ... e   ee I d  ee (cid:48) I d  er I d ... e (cid:48)   ee (cid:48) I d  ee I d  er I d where e , e (cid:48) and r , r (cid:48) are different entities and relations,  ee ,  er ,  ee (cid:48) are three different coefficients dependent on the frequencies of the corresponding entities and relations, and I d is the identity matrix of R d  d . Remark 2. In practice, we approximate the closed-form Hessian from Lemma 1 with its diagonal elements, EH    diag {   ,  ee I d ,    (cid:124) (cid:123)(cid:122) (cid:125) entities ,    , 0 ,    (cid:124) (cid:123)(cid:122) (cid:125) relations } . Remark 3.  e could be zero or negative, which breaks the positive definiteness of H   . Following Koh and Liang (2017), we add I (  > 0 ) to H   (i.e.,  ee   ee +  ), which equals adding an L 2 regularization on parameters. Following Equation 3 (  =  1 / |G| ), we can compute the change of group bias (denoted by  z B gr ) after removing a training triple z = (cid:104) h, r, t (cid:105) , 3  z B gr = 1 |G|  B (cid:62) gr (cid:16) E H  1     L ( z,   ) (cid:17) . A triple z with positive  z B gr means that re-training without it will increase B gr (i.e., towards mascu-line) and vice versa. We note that due to the diagonal Hessian, z will have a non-zero influence iff it is reachable from o in two hops (i.e., entities of z take part in the computation of B gr ). In practice, we calculate  z B gr of each triple in KB regarding B gr of each occupation, and make three observations. First, regarding relations in KB, we find most of the highly-influential triples (i.e. triples with highest absolute  z B gr values) to be of the profession relation (i.e., r p ) and its inverse 4 . For example, regarding the occupation of singer , these two relations occupy 74% of the top 1% positive triples and 78% of the top 1% negative triples. It suggests that compared with indirectly (i.e. two-hop) connected triples, triples directly connect with an entity have larger impact on it, which matches our intuitions. Second, regarding gender, we find that most person entities in triples with high positive  z B gr are of female gender, and vice versa. Figure 1 take the occupation of actor as an example to illustrate this. 5 This observation agrees with previous observation: triples with person entities of male gender will drive the overall biases towards masculine, and removing them will reverse this effect. Third, regarding graph substructure, we find that if a triple contains a high degree person entity, it usually has a nearly zero  z B gr (i.e. has small impact on other triples, see Figure 1), We suspect the reason to be as follows: the more neighbors 3 More precisely, z is a pair of triples ( (cid:104) h, r, t (cid:105) , (cid:104) h (cid:48) , r, t (cid:48) (cid:105) ) . To handle randomness of negative samples, we adopt two strategies in our implementation. First, we freeze negative samples in training epochs to get consistent results. Second, we use EH   to replace random H   in influence functions. 4 i.e. people . person . people _ with _ this _ profession 5 Similar patterns are observed in other occupations for both this observation and the next one. an entity has, the more constraints its embedding needs to put on others (by link prediction). It makes the embedding less optimal to represent each constraint, and hence less influential to each triple. Group-level correlation analyses offer us a coarse portrayal of biases. However, we are also interested in finer characterization (for each group member). Moreover, because of the complexity of KB structures, there very likely exist confounders between person entities and occupations (e.g. if two person entities of the same occupation are connected themselves, they are confounders of each other). In this case, correlation does not imply causation . In other words, gender differences are not guaranteed to be the only cause of B gr . Therefore, in this section, we study a further question: can we perform analyses on a specific person entity and measure its gender biases based on how its gender change its link prediction error (i.e. causality)? By virtue of the structured knowledge in KB, we are able to conduct this individual-level analysis in a tractable way. The key idea is, what if we keep everything else identical and perturb only the gender? Intuitively, given an occupation o , if flipping a person entity's gender from female to male will make it easier to connect the person with o , o should be biased towards male. Formally, we define individual bias B in of (cid:104) s, r p , o (cid:105) as B in =  ( s, r p , o ) | f   ( s, r p , o ) | m , where  | f (  | m ) is  computed on a version of TransE where s 's gender is female (male). B in means that, it is more difficult to predict s 's occupation if s is female. We would thus argue that (cid:104) s, r p , o (cid:105) is biased toward male. Because we keep all other attributes identical, this counterfactual definition naturally offers us causation. The practical issue of B in is the computation of the counterfactual: for each triple, this definition naively requires the re-training of the entire embedding model. This is intractable for large-scale analyses because of the extremely high computational cost. To avoid this issue, we apply influence function (Equation 4) for a fast evaluation of B in . Indeed, using Lemma 1 and Remark 2, we can obtain a closed-form B in (proof in Appendix B). Corollary 4. Assume that for each person entity s , we have the same negative sample for (cid:104) s, r p , f (cid:105) and (cid:104) s, r p , m (cid:105) , then B in   4  s |G| ( s + r p  o ) (cid:62) ( m  f ) , (5) One important observation of B in is that it is essentially a mixture of local graph substructure information (  s , the degree of s in KB), and a projection of link prediction residual ( s + r p  o ) onto the gender difference ( m  f , a reminiscence of word embedding gender subspace proposed in Bolukbasi et al., 2016). Compared with directly projecting o onto B wo (a hard generalization of word embedding bias measure), the link prediction residual is more compatible with the TransE learning objective. Figure 2 exhibits the distributions of B in of several occupations. We make two observations from the results. First, although there are a number of outliers, most B in are tightly distributed. It shows the consistency of the individual bias scores among different triples. Second, the bias scores correlate well with real-world gender stereotypes: engineer and lawyer lean more towards male, while model and actor lean more towards female. It suggests the validity of B in in describing biases in KB. Differences with Fisher et al. (2020b) A similar definition of bias is proposed in Fisher et al. (2020b) (denoted as B (cid:48) in ). B (cid:48) in is defined as follows: After training the embedding model to convergence, they perform one extra step of updating on the gender direction. The bias score is defined as the difference of the link prediction error before and after the update. We would like to note here the two aspects of differences between B in and B (cid:48) in . First, compared with B (cid:48) in , B in offers better interpretability. Concretely, in our definition, we approximate a purely counterfactual setting: flip the gender and re-train the entire model until convergence. In contrast, Fisher et al. (2020b) update the embedding after the convergence, which may not happen in real-world training. Compared with Equation 6, Equation 5 (approxi-mation of B in ) has an additional graph information term  s . Intuitively,  s serves as a normalization term: entities with more connections will be less affected by a single perturbation. In other words, the more connections an entity has, the less its link prediction error relies on one of them (i.e. gender). Again, take the occupation of journalist as an example, we show the relationship between B in and B (cid:48) in in Figure 3 and make two observations. First, there is a strong correlation between these two bias measures: points are approximately distributed along the diagonal. Second, we notice that there exist a substantial number of data points with positive B (cid:48) in but near zero B in . This suggests that the normalization term  s can prevent the overestimation of biases of person entities with many connections. This also corresponds to our third observation regarding the distribution of  z B gr (3.1). After obtaining B in , a remaining question is: given a group of person entities, how to use individual biases to characterize the group's overall bias? The rationale behind is that, if we can accurately measure biases of individuals, we should be able to aggregate them to represent biases of the group. A natural solution to this question is to directly average B in . However, in practice, we find that the averaged B in of all occupations align poorly with B gr ( r  0 . 27 ). We suspect this inconsistency to originate from the mismatches among different person entities' contexts in KB (i.e. different connections and local substructure). In other words, without controlling backdoor variables, simply averaging associations observed from each individual may not be suitable for representing association of the entire group (Pearl et al., 2016). 6 In our analyses, because of the complexity of KB, it is infeasible to control all factors. Nevertheless, we can control some of them to alleviate this issue. Here, we focus on controlling gender for two reasons. First, occupations in KB are often of very imbalanced gender ratios (e.g., nurse connects with more female entities than male entities). At the same time, different genders usually have different distributions of B in : female entities mainly have above zero B in , while B in of male entities distributes in a wider range. 7 Therefore, controlling gender should be able to effectively reduce the context mismatch. Second, because we treat the average link prediction error of each gender equally in group bias (  3.1), controlling gender can help us to obtain more comparable results. We thus propose to average scores of each gender separately to calibrate this mismatch ( weighted averaging instead of vanilla averaging ). Formally, 1 | F | (cid:80) s  FB in ( (cid:104) s, r p , o (cid:105) ) + 1 | M | (cid:80) s  MB in ( (cid:104) s, r p , o (cid:105) ) . We find weighted averaging align much better with B gr ( r  0 . 50 ) and real-world biases (  4.1). One method of inspecting the validity of a bias measure is to analyze its connection with real-world statistics (e.g. gender ratios of occupations). However, most existing datasets fail to meet our needs, 6 Other examples of this phenomenon include Simpson's Paradox and ecological fallacy . 7 We show B in distribution of the occupation of journalist as an example in Figure 5 in Appendix C, and find similar trends in other occupations. because they have different occupation categories with FB5M (e.g. Garg et al., 2018; Du et al., 2019). Accordingly, we take the following steps to build a new dataset. First, we collect the gender distributions of occupations in 2018 by the U.S. census data (Ruggles et al., 2020). Afterwards, we calculate their log proportions 8 and manually pair up them with occupations in KB. 9 We use it as our validation data and refer it as census data . Table 2 shows the Pearson's r values and p values between census data and all five bias measures described in  3 ( B gr , B in and B (cid:48) in with both averaging strategies). Our observations are two fold. First, both B gr and B in exhibit significant correlations (especially under weighted averaging ) with census data ( p < . 01 ), indicating their validity of measuring gender biases in KB embeddings. Second, individual bias measures ( B in and B (cid:48) in ) align better with census data under weight averaging than under vanilla averaging. This backs up our suspicion regarding contexts' mismatches, as well as our solution strategy (weighted averaging). Because the Hessian matrix we derived for the calculation of influence function is a diagonal approximation, and influence function of a group of training samples is only an approximation of the test performance change after re-training, one may concern the accuracy of our influence function. Therefore, in this section, we perform a validation experiment to address this concern. Specifically, for each occupation o , we first remove k triples with highest  z B gr , then re-train the TransE model from scratch, and calculate their B gr regarding o . Af-8 log-prop = p 1  p , where p is % of men in occupation. terwards, we compare the sum of  z B gr with the ground truth changes in B gr . In practice, we take k s to be a arithmetic progression from 500 to 5000, with a common difference of 500. We show a few occupations' alignment results as examples in Figure 4a-4c. We observe strong correlations between our approximation and the ground truth ( r > 0 . 95 for all occupations). It suggests the accuracy of our approximation (some additional results in Appendix C). Our study can broadly benefit relevant future research regarding societal biases and KB. As examples of such applications, based on our study in  3.1, we explore two strategies for de-biasing KB embeddings. We note that these two strategies aim to exemplify the potential impacts of our previous study, and are not necessarily the best method to de-bias KB embeddings. 10 Instead, we highly encourage future studies to build better de-biasing methods on the basis of our findings. Strategy 1: De-biasing by Adding In Table 1, we observe that gender ratio has a substantial impact on B gr . Based on this, one natural idea of debiasing is to balance gender proportion by adding dummy triples. The advantage of this strategy is that, because we do not remove triples, we are able to keep the information of the original KB intact. Specifically, suppose an occupation o with M male entities and F female entities, where M is larger than F . To alleviate bias, we create c ( M  F ) 11 dummy entities and connect them with only the female gender and o . Afterwards, we re-train TransE and observe the B gr regarding o . Table 3 lists a few examples of the results. We find that this de-biasing strategy overall works well. It is worth noting that the changes of biases of some occupations (e.g. nurse ) are smaller, which matches our previous observation: gender ratio is not the only decisive factor of B gr . Strategy 2: De-biasing by Removing Based on our study on the origins of biases, and inspired by the validation results in  4.2, we investigate a straightforward de-biasing strategy: we simply remove the top k most influential triples based on the approximation of influence function (IF-REMOVE). Again, we take k s to be [500 , 1000 , 1500 , ..., 5000] . Besides, for the purpose of controlling variable, we compare it to a naive baseline method, in which we randomly delete triples of all entities (Random-REMOVE). Figure 4d-4f exhibit some examples of the results. We observe that comparing with the baseline, where B gr rarely change, this de-biasing strategy is able to mitigate biases very effectively. Several additional examples are included in Appendix C. 5 Related Work Various measures have been proposed to quantify gender biases in word embeddings (Bolukbasi et al., 2016; Caliskan et al., 2017; Swinger et al., 2019). Many of them are based on vector similarity (e.g. cosine similarity) between words, which matches the training objective of most word embedding algorithms (maximize the vector similarities between similar words, Mikolov et al., 2013; Pennington et al., 2014). Moreover, Garg et al. (2018) suggest that word embedding can reflect biases in the training corpora and hence our society. Recently, a few studies have explored gender biases in KBs and their embeddings. A pioneer study by Klein et al. (2016) investigates gender gap in Wikidata across time, space, culture, occupation and language. A following study (Shaik et al., 2021) further analyzes the race and country of citizenship bias in KB regarding STEM representation. Moreover, Janowicz et al. (2018) analyze the potential bias issues in KBs from both data and schema viewpoints. Fisher et al. (2020b) propose a KB embedding bias measure based on the change of link prediction error after a one-step update towards male. Fisher et al. (2020a) and Arduini et al. (2020) propose to use adversarial training objective to mitigate biases in KB embeddings. Influence function is a commonly used technique in robust statistics (Cook and Weisberg, 1982). Koh and Liang (2017) first use it to inspect each training point's influence on a neural network's prediction. A following study by Koh et al. (2019) investigate the accuracy of influence function on measuring the effect of removing a group of training samples, and show that its approximation has strong correla-1388 tions with actual effects. Afterwards, Brunet et al. (2019) apply influence function as a differential bias measure to study gender bias in word embedding. Moreover, Pezeshkpour et al. (2019) use an simplification of influence function to perform adversarial attack on link prediction. In this paper, we study the gender biases in KB embeddings. First, we develop two bias measures to quantify biases: one from the group level and the other from the individual level. Evidence of their validity are obtained in comparison with real-world biases. Second, to understand the origins of biases, we adopt influence functions for triple-level analysis and develop an efficient method for fast evaluation. The accuracy of this method is validated by comparing our approximation with group-truth changes after re-training. Moreover, as examples of the potential applications of our find-ings, we propose two de-biasing strategies for KB embeddings and obtain promising performance. Although we focus on Freebase (FB5M) and TransE in this paper, we note that our analyses are theoretically generalizable to other commonly-used KBs and embedding training algorithms. For instance, Wikidata, another commonly-used KB, uses a different hierarchical structure to organize its data (Vrandecic and Krtzsch, 2014; Tanon et al., 2016; Wang et al., 2021). However, it still loosely follows the triple structure used in Freebase, and therefore can be pre-processed to fit in our analyses. Also, because our bias measures and bias tracing methods are built on simple and generalizable defi-nitions (i.e., differences between link predictions errors and influence function), they can naturally be adapted to other KB embedding algorithms (Lin et al., 2015; Yang et al., 2015; Peng et al., 2021). However, we recognize that such generalizations are not trivial efforts. Take Wikidata again for an instance, although a simple transformation is adequate for running the embedding algorithm, it is far from fully eliminating the differences between Freebase and Wikidata. For example, Wikidata does not have an inverse predicate for each relation, and has a much smaller number of overall relations (Azmy et al., 2018; Diefenbach et al., 2017). Such differences might have a large impact on the final results. Also, to perform the same analyses with other embedding algorithms, we will need to develop algorithms to facilitate the computation of their influence function (as Lemma 1), too. Therefore, we consider such generalizations to be promising future directions but out of the scope of our work. We thank Dong Nguyen for her meticulous and valuable suggestions, as well as productive discussions. We also thank all anonymous reviewers for their constructive and helpful feedback. This research was (partially) supported by NSFC (62076097), STCSM(20511101205), and Shanghai Key Laboratory of Multidimensional Information Processing, ECNU (2020KEY001). The corresponding authors are Yuanbin Wu and Yan Yang. Intended Usage Our work intend to provide insights of gender biases in KB and its embeddings, on how to measure these biases and how to trace the origins of them. Moreover, as discussed in  4.3, future studies can build better de-biasing methods based on our findings. In this way, our framework can contribute to the development of models that are less biased and hence potentially less harmful. Limitations In this study, we use gender information already encoded in KB to measure and trace gender biases. However, because only binary gender is recorded in the KB that we use (Freebase), we take a narrow view of binary gender in our analyses. We hope to see more future studies on gender biases in KB embeddings that consider non-binary gender identities as well as intersectional identities.
abstain method objective abstain objective method abstain result method abstain abstain abstain abstain abstain abstain abstain abstain result method objective abstain abstain method objective objective result objective objective abstain result result abstain result objective abstain abstain method abstain abstain method abstain abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other method abstain abstain method abstain method abstain method method method abstain abstain abstain method abstain method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain method abstain method method abstain method abstain abstain abstain method abstain abstain abstain other method abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method abstain abstain method abstain abstain abstain abstain abstain method abstain other abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain method abstain abstain method method abstain abstain abstain abstain abstain abstain method abstain abstain method abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method objective abstain objective method objective method abstain method method abstain abstain abstain abstain objective method other other other other method method abstain method method abstain