Knowledge graphs (KG) have become increasingly important to endow modern recommender systems with the ability to generate traceable reasoning paths to explain the recommendation process. However, prior research rarely considers the faithfulness of the derived explanations to justify the decision-making process. To the best of our knowledge, this is the first work that models and evaluates faithfully explainable recommendation under the framework of KG reasoning. Specifically, we propose neural logic reasoning for explainable recommendation (LOGER) by drawing on interpretable logical rules to guide the path-reasoning process for explanation generation. We experiment on three large-scale datasets in the e-commerce domain, demonstrating the effectiveness of our method in delivering high-quality recommendations as well as ascertaining the faithfulness of the derived explanation. Compared with traditional recommender systems (RS), explainable recommendation is not only capable of providing high-quality recommendation results but also offers personalized and intuitive explanations (Zhang and Chen, 2020). Incorporating a knowledge graph (KG) into recommender systems has become increasingly popular, since KG reasoning is able to generate explainable paths connecting users to relevant target item entities. At the same time, there is increasing demand for systems to ascertain the faithfulness of the generated explanation, i.e., assess whether it faithfully reflects the reasoning process of the model and is consistent with the historic user behavior. However, previous work has largely neglected faithfulness in KG-enhanced explainable recommendation (Xian et al., 2020a; Fu et al., 2020a). A number of studies (Lakkaraju et al., 2019; ter Hoeve et al., 2018; Wu and Mooney, 2018) argue that  Equal contribution faithful explanations should also be personalized and gain the capability to reflect the personalized user historic behavior. However, to the best of our knowledge, none of the existing explainable recommendation models based on KGs have considered faithfulness in the explainable reasoning process and its evaluation on the generated explainable paths. For instance, PGPR (Xian et al., 2019; Zhao et al., 2020) infers explainable paths over the KG without considering personalized user behavior, and its prediction on next potential entities is merely based on the overall knowledge-driven rewards. CAFE (Xian et al., 2020b) builds user module profiles to guide the path inference procedure. However, as illustrated in Subramanian et al. (2020), such neural module networks only implicitly abstract the reasoning process and lack of considering the faithfulness of explanations. In this paper, we propose a new KG-enhanced recommendation model called LOGER to produce faithfully explainable recommendation via neural logic reasoning. To fully account for heterogeneous information and rules about users and items from the KG, we leverage an interpretable neural logic model for logical reasoning, enhanced by a general graph encoder that learns KG representations to capture semantic aspects of entities and relations. These two components are iteratively trained via the EM algorithm by marrying the merits of interpretability of logical rules and the expressiveness of KG embeddings. Subsequently, the learned rule weights are leveraged to guide the path reasoning to generate faithful explanations. The derived logical rules are expected to be consistent with historic user behavior and the resulting paths genuinely reflect the decision making process in KG reasoning. We experiment on three large-scale datasets for e-commerce recommendation that cover rich user behavior patterns. The results demonstrate the superior recommendation performance achieved by our model compared to the state-of-the-art baselines, with the guarantee of the faithfulness on the generated path-based explanations. The contributions of this paper are threefold. We highlight the significance of considering faithfulness in explainable recommendation. We propose a novel approach that incorporates interpretable logical rules into KG path reasoning for recommendation and explanation generation. We experiment on three large-scale datasets showing promising recommendation performance as well as faithful path-based explanation. A knowledge graph (KG) for recommendation is defined as G = { ( e h , r, e t ) | e h , e t  E , r  R} , where E denotes the entity set consisting of sets of users U , items I , and other entities, while R denotes the relation set. Each triplet ( e h , r, e t ) represents a fact indicating head entity e h interacts with tail entity e t via relation r . In recommendation tasks, we are particularly interested in useritem interactions { ( u, r ui , v ) | u  U , r ui  R , v  I} with the special relation r ui meaning purchase in e-commerce or like in movie recommendation. The problem of KG reasoning for explainable recommendation is formulated as follows. Given an incomplete KG G with missing useritem interactions, for every user u  U , the goal is to select a set of items as recommendations { v | ( u, r ui , v ) (cid:54) G , v  I} along with a set of paths as explanations connecting each pair of the user and a predicted item. The key challenge is to not only guarantee the recommendation quality with the rich information in KG, but also generate faithful explanations that reflect the actual decision-making process of the recommendation model and are consistent with historic user behavior. We introduce the novel neural LOGic Explainable Recommender (LOGER) for producing faithfully explainable recommendations with a KG. As illustrated in Fig. 1, it consists of three components: (i) a KG encoder for learning embeddings of KG entities and relations to capture their semantics, (ii) a neural logic model for conducting interpretable logical reasoning to make recommendations, and (iii) a rule-guided path reasoner for generating faithfully explainable paths. Both KG encoder and neural logic model are trained iteratively via the EM algorithm (Neal and Hinton, 1998) so that they mutually benefit to make recommendations via logical reasoning. Additionally, personalized rule importance scores are derived for every user and leveraged to guide the path reasoning for faithful explanation generation. Let X hrt be a binary random variable indicating whether a triplet ( e h , r, e t ) is true or not, XG = { X hrt | ( e h , r, e t )  G} be a random variable regarding all observed triplets in the KG G , and XH = { X hrt | ( e h , r, e t )  H } be a random variable of hidden useritem interactions in H = { ( u, r ui , v ) | u  U , v  I , ( u, r ui , v ) (cid:54) G} . The KG encoder is generally defined as a triplet-wise function f  : E  R  E (cid:55) [0 , 1] parametrized by  that maps each triplet to a real-valued score. For any triplet ( e h , r, e t )  G  H , we can interpret its truth probabilistically via the KG encoder f  as q ( X hrt |  ) = Bernoulli ( X hrt | f  ( e h , r, e t )) . The KG encoder f  can be instantiated with any existing KG embedding (Ji et al., 2020) or graph neural network (Wu et al., 2020) model. We focus on composition rules for useritem interactions, i.e., r ui is a composition of relations r 1 , . . . , r j if ( u, r 1 , e 1 )      ( e j  1 , r j , v )  ( u, r ui , v ) ,  u  U , v  V , e 1 , . . . , e j  1  E . Given a set of logical rules L mined from the KG, the goal of this component is, for every user u  U , to emit a set of personalized rule importance scores y u = { y u,l } l  L to capture the historic user behavior. To achieve this, we build upon Markov Logic Networks (Qu and Tang, 2019), an interpretable probabilistic logic reasoning method that models the joint distribution of all triplets via a set of logical rules L , i.e., p ( XG , XH | w ) = 1 Z exp (cid:0)(cid:80) l  L w l n l (cid:1) , where w = { w l } l  L with w l being the global weight of rule l  L , and n l denotes the number of true groundings of rule l over observed and hidden triplets. Accordingly, we de-fine the personalized rule importance score to be y u,l = w l n l ( u ) (cid:80) l (cid:48) L n l (cid:48) ( u ) , where n l ( u ) is the number of groundings of rule l over the observed triplets in { ( u, r ui , v )  G} . However, it is intractable to directly maximize the log likelihood of observed triplets to learn the global weights w , i.e., max w log p ( XG | w ) . Instead, we employ the EM algorithm to iteratively optimize the objective to acquire optimal global weights. E-Step We introduce a mean-field variational distribution q ( XH |  )  (cid:81) ( e h ,r,e t )  H q ( X hrt |  ) over hidden useritem interactions in H . The goal of the E-step is to estimate q ( XH |  ) by minimizing the KL divergence between q ( XH |  ) and the posterior distribution p ( XH | XG , w ) with fixed w . For each triplet ( e h , r, e t )  H , we denote by L hrt the set of rules associated with the triplet and by G hrt the corresponding groundings of all logical rules in L hrt . Following Qu and Tang (2019), the optimal q ( XH |  ) can be achieved under the fixed-point condition, i.e., q ( X hrt |  )  p ( X hrt | XG hrt , w ) , for all ( e h , r, e t )  H . Here, q ( X hrt |  ) is approximated by the KG encoder f  , and p ( X hrt | XG hrt , w ) can be estimated with the global weights w of the rules in L hrt from the last iteration: p ( X hrt = 1 | XG hrt , w ) =  (cid:32)(cid:80) l  L hrt w l | L hrt | (cid:33) , (1) where  (  ) is the sigmoid function. In other words, if a hidden triplet ( e h , r, e t ) is asserted to be true by the rules (e.g., p ( X hrt = 1 | XG hrt , w ) > 0 . 5 ), the probability q ( X hrt = 1 |  ) given by the KG encoder is also expected to be high. Therefore, to learn the parameter  , we aim to maximize the log-likelihood function over all observed triplets in G and the plausibly true hidden triplets in H + = { ( e h , r, e t ) | p ( X hrt = 1 | XG hrt , w )   } , which leads to the objective (cid:96) (  ) = (cid:88) ( e h ,r,e t ) G H + log q ( X hrt = 1 |  ) , (2) where  is a hyperparameter. M-Step The goal of the M-step is to learn the global rule weights w by maximizing the log-likelihood function E q ( XH ) [log p ( XG , XH ; w )] given a fixed  from the E-step. Since the log-likelihood term models the joint distribution over all triplets, which is hard to compute for a large KG, we approximate it with the pseudolikelihood (Besag, 1975): (cid:96) PL ( w ) = (cid:80) ( e h ,r,e t ) G HE q ( XH |  ) [log p ( X hrt | XG hrt , w )] . Then, we can invoke gradient ascent to acquire the optimal w , with the gradient defined as:  w l (cid:96) PL ( w l ) = (cid:88) ( e h ,r,e t ) G 1  p hrt | L hrt | + (cid:88) ( e h ,r,e t )  H q ( X hrt = 1 |  )  p hrt | L hrt | , (3) where p hrt = p ( X hrt = 1 | XG hrt , w ) . Once the optimal global weights are acquired, we can make a recommendation by calculating the ranking score of a user u  U and an item v  I as q ( X urv |  ) +  p ( X urv = 1 | XG urv , w ) , where r = r ui and   R is a hyperparameter. We draw on the KG encoder f  and the personalized rule importance scores y u from the last two steps to generate explainable paths for every user u . Specifically, we train an LSTM-based path reasoning network  that takes the start user embedding as input and predicts a sequence of entities and relations to form a path. For every user u , we restrict the reasoner to generate the paths that follow the rules with the largest scores in y u . The details of  and path reasoning are described in the Appendix. Dataset We experiment on three domain-specific e-commerce datasets from Amazon, namely Cellphones , Grocery , and Automotive . There are two requirements that lead to the selection of these categories in our experiments. First, the constructed KG should contain rich user behavior patterns, e.g., user mentioned features or preferred styles, etc. This is the major difference from most of the existing work (Zhao et al., 2019), which only extends knowledge on the item side. Second, the KGs are assumed to be large-scale. We select several large subsets from Fu et al. (2020b), where the constructed KG can be regarded as an updated version of those of Ai et al. (2019) based on the Amazon review dataset (Ni et al., 2019). The remaining three datasets are the ones that satisfy both of the aforementioned requirements. Statistical details of datasets are provided in the Appendix. Baselines & Metrics We consider several state-of-art baselines in the following experiments. CKE (Zhang et al., 2016) uses semantic representations derived from TransR (Lin et al., 2015) to enhance the matrix factorization process. RippleNet (Wang et al., 2018) is a hybrid method combining regularization and path formats, and augmenting user representations with a memory-network-like approach. PGPR (Xian et al., 2019) designed a policy-guided graph search algorithm for recommendation over KGs. HeteroEmbed (Ai et al., 2018) aims to learn the embeddings of a heterogeneous graph including users, items, and relations for recommendation. KGAT (Wang et al., 2019) explicitly models higher-order KG connectivity and learns node representations by propagating the embedding of neighbors with corresponding importance discriminated by an attention mechanism. We adopted the same metrics as Ai et al. (2018) to evaluate the recommendation performance of all models: Precision , Recall , Normalized Discounted Cumulative Gain ( NDCG ), and Hit Rate ( HR ). We first evaluate the recommendation quality of our model. The results of all methods across all three datasets are reported in Table 1. In general, our method significantly outperforms all state-of-the-art baselines on all metrics. Taking Cellphones as an example, our method achieves an improvement of 6.01% in NDCG against the best baseline (underlined), and an improvement of 5.82% in Hits@10. Similar trends can be observed on other benchmarks as well. Note that both our model and HeteroEmbed adopt TransE for KG representation learning, yet our model achieves better performance, mainly attributed to the iterative learning of graph encoder and neural logic model. Measuring Faithfulness Inspired by previous work (Maaten and Hinton, 2008; Serrano and Smith, 2019; Subramanian et al., 2020), we de-fine the faithfulness to be the JensenShannon (JS) divergence of rule-related distributions from training and test sets. Specifically, we randomly sample 50 users from the training set. For each user u , we further sample around 1,000 paths between the user and the connected item nodes, and calculate the rule distribution over these paths, denoted by F ( u ) . We compare the proposed LOGER with two baselines, PGPR, and KGAT, each of which is used to generate 20 explainable paths for every selected user in the test phase. Similarly, we can calculate the rule distribution over these 20 paths, denoted by Q f ( u ) . The JS scores are defined as follows. Here, Q w ( u ) is the rule weight distribution derived from the personalized rule importance scores of our method or the path weights of baselines. Smaller values of two JS scores correspond to better faithfulness of the explainable paths. This faithfulness evaluation is motivated in terms of the consistency of the explainable paths with respect to the user historic behavior. User Study Additionally, we conduct a user study to evaluate the faithfulness of the explainable paths. We display 50 sampled KG paths starting from one user towards purchased items in the training set to represent examples of user historical behaviors. For comparison, we also present 10 explainable paths generated by three methods for the same user in the test dataset. We ask 20 human subjects to rank these methods based on whether Cellphones Grocery Automotive Precision Recall NDCG HR Precision Recall NDCG HR Precision Recall NDCG HR CKE 0.0360 0.1760 0.1847 0.3067 0.0612 0.2528 0.3070 0.4511 0.0458 0.1871 0.2257 0.3621 RippleNet 0.0419 0.2141 0.2177 0.3715 0.0591 0.2682 0.2858 0.4800 --PGPR 0.0462 0.2148 0.2366 0.3801 0.0649 0.2710 0.3174 0.4926 0.0589 0.2315 0.2804 0.4409 KGAT 0.0476 0.2274 0.2365 0.3835 0.0702 0.2916 0.3381 0.5020 0.0601 0.2500 0.2859 0.4514 HeteroEmbed 0.0527 0.2543 0.2626 0.4226 0.0785 0.3316 0.3701 0.5572 0.0695 0.2923 0.3314 0.5082 LOGER 0.0622 0.2977 0.3227 0.4808 0.0906 0.3754 0.4370 0.6121 0.0743 0.3091 0.3653 0.5346 Table 1: Recommendation quality of all methods on three datasets. paths obtained by three methods. Bold numbers indicate the best results. the generated paths are consistent with those from the training set. Then, we calculate the average ranking scores ( Avg. Rank ) by averaging the rank given by each human tester on each method. Results The results on the Cellphones and Grocery datasets are reported in Table 2. We observe that our method LOGER achieves the lowest JS scores and average ranking score, which reveal the effectiveness of our model in producing more faithful explanations in both quantitative measurements and in the user study. We further study how hidden triplets used in training KG encoder (Eq. 2) influence the recommendation performance. We experiment on the Cellphones data under different sizes of hidden triplet sets H + . We choose the sizes of { 10 , 20 , 30 , 40 , 50 } and keep all other settings unchanged. The results are plotted in Fig. 2, including our model (red circles) and the best baseline HeteroEmbed (blue crosses). We find that our model consistently outperforms the baseline in all the metrics under different numbers of hidden triplets. Better recommendation performance can be achieved with more hidden triplets included in training the KG encoder, because more candidate items will enhance the capability of our model to discern the logical rules of good quality and hence benefit the recommendation prediction. In this paper, we propose LOGER for faithfully explainable recommendation, which generates explainable paths based on personalized rule importance scores via neural logic reasoning that adequately captures historic user behavior. We experiment on three large-scale datasets for e-commerce recommendation showing superior recommendation quality of LOGER as well as the faithfulness of the generated explanations both quantitatively and qualitatively. We hope to encourage future work that values explainability and in particular the faithfulness of explanations. Our code is available at https://github.com/orcax/LOGER . We thank the reviewers for the valuable feedback and suggestions. This work was supported in part by NSF IIS-1910154 and IIS-2007907. Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect those of the sponsors.
abstain abstain objective objective objective abstain abstain abstain abstain abstain method abstain abstain abstain objective abstain abstain abstain abstain method objective objective result objective result abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain other abstain method abstain method abstain method method abstain method method abstain method abstain other abstain abstain method abstain method method method method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result result abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain result abstain abstain abstain result result result objective result method other other other other