Utilizing reviews to learn user and item representations is useful for recommender systems. Existing methods usually merge all reviews from the same user or for the same item into a long document. However, different reviews, sentences and even words usually have different informativeness for modeling users and items. In this paper, we propose a hierarchical user and item representation model with three-tier attention to learn user and item representations from reviews for recommendation. Our model contains three major components, i.e., a sentence encoder to learn sentence representations from words, a review encoder to learn review representations from sentences, and a user/item encoder to learn user/item representations from reviews. In addition, we incorporate a three-tier attention network in our model to select important words, sentences and reviews. Besides, we combine the user and item representations learned from the reviews with user and item embeddings based on IDs as the final representations to capture the latent factors of individual users and items. Extensive experiments on four benchmark datasets validate the effectiveness of our approach. Learning accurate user and item representations is very important for recommender systems (Tay et al., 2018). Many of existing recommendation methods learn user and item representations based on the ratings that users gave to items (Koren et al., 2009; Mnih and Salakhutdinov, 2008). For example, Koren et al. (2009) proposed a matrix factorization method based on SVD to learn latent representations of users and items from the rating matrix between users and items. However, since the numbers of users and items in online platforms are usually huge, and the rating matrix between users and items is usually very sparse, it is quite diffiDefragand cleanup, then you have a great laptop! Style: Laptop Only Verified Purchase I bought this laptop yesterday. This is a great laptop if you immediately run maintenance checks on it (defrag, disk cleanup} and remove a little bloatware. It is not a laptop to game with, but as a working/school laptop, you're getting a great bang for your buck. Only giving it four stars just because of the above mentioned things I did afterward, but by no means is this a ho  riblelaptop. 106 people found this helpful ***** MULTIMEDIA LAPTOP June 22, 2018 Style: Laptop Only Verified Purchase This Laptop Great! One person found this helpful Figure 1: Two example reviews. cult for those rating based recommendation methods to learn accurate user and item representations (Zheng et al., 2017; Tay et al., 2018). Luckily, in many online platforms such as Amazon and IMDB, there are rich reviews written by the users to express their opinions on items. These reviews can provide rich information of items. For example, if sentences like bad battery life and battery capacity is low frequently appear in the reviews of a smartphone, then we can infer the performance of this item in battery life is not good. The reviews also contain rich information of users. For example, if a user frequently mentions the price is too high and very expensive in his/her reviews for different items, then we can infer this user may be sensitive to price. Thus, these reviews can help enhance the learning of user and item representations especially when ratings are sparse, which is beneficial for improving the performance of recommender systems (Zheng et al., 2017). Utilizing reviews to learn user and item representations for recommendation has attracted increasing attentions (Zheng et al., 2017; Catherine and Cohen, 2017). For example, Zheng et al. (2017) proposed a DeepCoNN method to learn the representations of users and items from reviews using convolutional neural networks (CNN), and achieved huge improvement in recommendation performance. These methods usually concatenate the reviews from the same user or the same item into a long document. However, different reviews usually have different informativeness in representing users and items. For example, in Fig. 1 the first review is much more informative than the second one. Distinguishing informative reviews from noisy ones can help learn more accurate user and item representations. In addition, different sentences in the same review may also have different informativeness. For example, in Fig. 1 the sentence it is not a laptop to game with contains more important information than I bought this laptop yesterday. Besides, different words in the same sentence may also have different importance. For example, in this is a great laptop if you ... the word great is more important than you in modeling this item. In this paper, we propose a hierarchical user and item representation model with three-tier attention ( HUITA ) to learn informative user and item representations from reviews for recommendation. In our approach, the hierarchical user and item representation model contains three major components, i.e., a sentence encoder to learn sentence representations from words, a review encoder to learn review representations from sentences, and a user/item encoder to learn user/item representations from the all reviews posted by this user or for this item. In addition, we propose to incorporate a three-tier attention network into our model to select important words, sentences and reviews to learn more informative user and item representations. Besides, we combine the user and item representations learned from the reviews with the user and item embeddings based on their IDs as the final representations to capture the latent factors of each individual users and items. We conduct extensive experiments on four benchmark datasets. The results show our approach can effectively improve the performance of recommendation and outperform many baseline methods. Learning user and item representations from reviews for recommendation has attracted many attentions (McAuley and Leskovec, 2013; Ling et al., 2014; Bao et al., 2014; Zhang et al., 2014; Diao et al., 2014; He et al., 2015; Tan et al., 2016; Ren et al., 2017). Many of the existing methods focus on extracting topics from reviews to model users and items. For example, McAuley and Leskovec (2013) proposed a Hidden Factors as Topics (HFT) method to use the topic modeling technique LDA to discover the latent aspects of users and items from the reviews. Ling et al. (2014) proposed a Ratings Meet Reviews (RMR) method to enhance the representations of users and items by extracting topics from review texts and aligning the dimensions of these topics with the latent user representations obtained from the rating matrix using matrix factorization. Bao et al. (2014) proposed a TopicMF approach to jointly model user and item representations using rating scores via matrix factorization and using review texts via non-negative matrix factorization (NMF) to obtain topics. However, these methods only extract the topic information from reviews, and a large amount of important semantic information is not captured. In addition, these methods are usually based on topic models and cannot effectively model the contexts and orders of words in reviews, both of which are important for inferring user preferences and item properties. In recent years, several deep learning based methods have been proposed to learn user and item representations from reviews for recommendation (Zhang et al., 2016; Zheng et al., 2017; Catherine and Cohen, 2017; Seo et al., 2017b,a; Chen et al., 2018; Tay et al., 2018). For example, Zheng et al. (2017) proposed a DeepCoNN method which uses CNN to learn representations of users and items from their reviews. Catherine and Cohen (2017) proposed a TransNets method to learn user and item representations from reviews using CNN and regularize these representations to be close to the representations of the review written by the target user to the target item. Seo et al. (2017b) proposed to learn user and item representations via CNN network as well as attention network over word embeddings. These methods concatenate all the reviews from the same user or for the same item into a long document, and cannot distinguish informative reviews from noisy ones. Chen et al. (2018) proposed to model the usefulness of reviews using review-level attention to enhance the learning of user and item representations. However, their method regards each review as a long sentence, and cannot distinguish informative sentences and words from less informative ones. Different from the aforementioned methods, in our approach we propose a hierarchical framework to learn user and item representations from reviews for recommendation. Our model first learns sentence representations from words, then learns review representations from their sentences, and finally learns user/item representations from their reviews. Our model also contains a three-tier attention network to jointly select important words, sentences and reviews to learn more informative user and item representations. Experiments on benchmark datasets validate the advantage of our approach over existing methods in recommendation. In this section, we introduce our HUITA approach to learn user and item representations from reviews for recommendation. The architecture of our approach is shown in Fig. 2. There are three major modules in our approach. The first one is sentence encoder which learns representations of sentences from words. The second one is review encoder which learns representations of reviews from sentences. And the third one is user/item encoder , which learns the representations of users and items from their reviews. Next we introduce each module in detail. The sentence encoder module is used to learn representations of sentences from words. According to Fig. 2, there are three layers in this module. The first layer is word embedding. It is used to convert a sequence of words into a sequence of low-dimensional dense vectors which contain semantic information of these words. Denote a sentence s contains M words [ w 1 , w 2 , ..., w M ] . Through the word embedding layer the sentence s is transformed into a vector sequence [ e 1 , e 2 , ..., e M ] using a word embedding matrix E  RV  D , where V and D represent the vocabulary size and the word embedding dimension, respectively. The word embedding matrix E is initialized using pretrained word embeddings, and fine-tuned during model training. The second layer is a convolutional neural network (CNN). CNN is an effective neural architecture for capturing local information (LeCun et al., 2015). We employ a word-level CNN to capture the local contexts of words to learn their contextual representations. Denote c w i as the contextual representation of the word w i , which is computed as follows: c wi = ReLU( U w  e ( i  K w ):( i + K w ) + b w ) , (1) where e ( i  K w ):( i + K w ) is the concatenation of the word embedding vectors from the position i  K w to i + K w . U w  RN w  (2 K w +1) D and b w  RN w are the parameters of the filters in CNN network, where N w is the number of CNN filters and 2 K w + 1 is the window size. ReLU is the nonlinear activation function (Glorot et al., 2011). The output of the CNN layer is a sequence of contextual word representations [ c w 1 , c w 2 , ..., c wM ] . The third layer is a word-level attention network. Different words in the same sentence may have different informativeness for modeling users and items. For example, in the sentence The laptop I bought yesterday is too heavy, the word heavy is more informative than the word yes-terday in representing this laptop. Thus, we use a word-level attention network to help our model select and attend to important words based on their contextual representations to build more informative sentence representations for user and item modeling. The attention weight of the i th word in the sentence s is computed as follows: a wi = tanh( v w  c wi + b w ) , (2)  wi = exp( a wi ) (cid:80) Mj =1 exp( a wj ) , (3) where v w  RN w and b w  R are the parameters in the attention network. i indicates the relative importance of the i th word evaluated by the attention network. The final representation of the sentence s is the summation of the contextual word representations weighted by their attention weights as follows: s = M (cid:88) i =1  wi c wi . The review encoder module aims to build the representations of each review based on the representation of sentences in these reviews. There are two major layers in the review encoder module. The first layer is a sentence-level CNN network. Neighboring sentences usually have some relatedness with each other. For example, in a laptop review It is not a laptop to game with. But as a working laptop, you will get a great bang for your buck, the two neighboring sentences have close relatedness and they both describe the performance of the laptop in different scenarios. Figure 2: The framework of our HUITA approach for recommendation. Thus, we employ a sentence-level CNN network to learn the contextual sentence representations by capturing the local contexts of sentences. Denote a review r contains N sentences [ s 1 , s 2 , ..., s N ] . Denote the contextual representation of sentence s i as c si , which is computed as follows: c si = ReLU( U s  s ( i  K s ):( i + K s ) + b s ) , (5) where U s  RN s  (2 K s +1) N w and b s  RN s are parameters of the sentence-level CNN filters. s ( i  K s ):( i + K s ) is the concatenation of sentence representation vectors from position i  K s to i + K s . N s is the number of filters in sentence CNN network and 2 K s + 1 is the window size. The second layer is a sentence-level attention network. Different sentences in a review may have different informativeness for modeling users and items. For example, the sentence it is not a laptop to game with is more informative than the sentence I bought this laptop yesterday in learning the representation of this laptop. Thus, we use sentence-level attention network to help our model select and attend to important sentences to learn more informative review representations. The attention weight of sentence s i in the review r is formulated as follows: a si = tanh( v s  c si + b s ) , (6)  si = exp( a si ) (cid:80) Nj =1 exp( a sj ) , (7) where v s  RN s and b s  R are the parameters of the attention network. The final contextual representation of the review r is the summation of the contextual representations of sentences weighted by their attention weights, which is formulated as: r = N (cid:88) i =1  si c si . The user/item encoder module is used to build the representations of users or items based on the representations of their reviews. Different reviews usually have different informativeness in modeling users or items. For example, in Fig. 1, the first review contains much more information of the laptop than the second review, and should has more contributions in building the representation of this laptop. Thus, we use a review-level attention network to distinguish informative reviews from less informative ones. Denote a user u has P reviews [ r 1 , r 2 , ...., r P ] . Then the attention weight of the review r i is computed as follows: a ri = tanh( v r  r i + b r ) , (9)  ri = exp( a ri ) (cid:80) Pj =1 exp( a rj ) , (10) where v r  RN s and b r  R are the parameters of the review-level attention network. The user representation learned from the reviews is the summation of the contextual representations of reviews weighted by their attention weights: u r = P (cid:88) i =1  ri r i . Although the user representation u r learned from reviews contain rich information of users, there are some latent characteristics of users which are not described in their reviews but can be inferred from the rating patterns. Thus, we also represent users using the embedding of their IDs to capture the latent factors of users, which are motivated by traditional recommendation methods (Koren et al., 2009). The final representation of user u is the concatenation of the user representation u r learned from reviews and the user embedding u d inferred from user ID, as follows: u = [ u r , u d ] . The representations of items can be computed in a similar way. Denote the representation of item t learned from reviews as t r , and the item embedding inferred from item ID as t d . Then the final representation of this item is as follows: t = [ t r , t d ] . In recommender systems the recommendations are made based on the predicted ratings that a user will give to an item. In our HUITA approach, the rating score of a user-item pair is predicted based on the representations of users and items as follows:  y = ReLU( w T ( u (cid:12) t ) + b ) , (14) where (cid:12) is item-wise dot product, w and b are parameters in the rating prediction layer. In the model training stage, we optimize the model parameters to minimize the difference between gold rating and predicted ratings. We use the mean squared error as the loss function: L = 1 NP NP (cid:88) i =1 ( y i  y i ) 2 , (15) where NP denotes the number of user-item pairs in training data,  y i and y i are the predicted rating score and the gold rating score respectively of the i th user-item pair. We conducted experiments on four widely used benchmark datasets in different domains to evaluate the effectiveness of our approach. Following (Chen et al., 2018), we used three datasets from the Amazon collection 1 (He and McAuley, 2016), i.e., Toys and Games , Kindle Store , and Movies and TV . Another dataset is from Yelp Challenge 2017 2 (denoted as Yelp 2017 ), which is a large-scale restaurant review dataset. Following (Chen et al., 2018), we only kept the users and items which have at least 5 reviews. The detailed statistics of the four datasets are summarized in Table 1. The ratings in these datasets are in [1, 5]. In our experiments, the dimension of word embeddings was set to 300. We used the pre-trained Google embedding (Mikolov et al., 2013) to initialize the word embedding matrix. The word-level CNN has 200 filters and their window size is 3. The sentence-level CNN has 100 filters with window size of 3. We applied dropout strategy (Srivastava et al., 2014) to each layer of our model to mitigate overfitting. The dropout rate was set to 0.2. Adam (Kingma and Ba, 2014) was used as the optimization algorithm. The batch size was set to 20. We randomly selected 80% of the user-item pairs in each dataset for training, 10% for validation and 10% for test. All the hyperpa-rameters were selected according to the validation set. We independently repeated each experiment for 5 times and reported the average performance in Root Mean Square Error (RMSE). We evaluate the performance of our approach by comparing it with several baseline methods. The methods to be compared include:  PMF : Probabilistic Matrix Factorization, which models users and items based on 1 http://jmcauley.ucsd.edu/data/amazon 2 https://www.yelp.com/dataset challenge PMF NMF SVD++ HFT DeepCoNN Attn+CNN NARRE HUITA Rating score (cid:88) (cid:88) (cid:88) (cid:88) (cid:88) (cid:88) (cid:88) (cid:88) Review text (cid:88) (cid:88) (cid:88) (cid:88) (cid:88) Word context & order (cid:88) (cid:88) (cid:88) (cid:88) Review attention (cid:88) (cid:88) Word/sentence attention (cid:88) * (cid:88) Table 2: Information used in different methods. ratings via matrix factorization (Mnih and Salakhutdinov, 2008). NMF : Non-negative Matrix Factorization for recommendation based on rating scores (Lee and Seung, 2001). SVD++ : The recommendation method based on rating matrix via SVD and similarities between items (Koren, 2008). HFT : Hidden Factor as Topic (HFT), a method to combine reviews with ratings via LDA (McAuley and Leskovec, 2013). DeepCoNN : Deep Cooperative Neural Networks, a neural method to jointly model users and items from their reviews via CNN (Zheng et al., 2017). Attn+CNN : Attention-based CNN, which uses both CNN and attention over word embeddings to learn user and item representation from reviews (Seo et al., 2017b). NARRE : Neural Attentional Rating Regression with Review-level Explanations, which uses attention mechanism to model the informativeness of reviews for recommendation (Chen et al., 2018). HUITA : our proposed hierarchical user and item representation approach with three-tier attention for recommendation with reviews. In Table 2, we show a simple comparison of different methods in terms of the information considered in each method. Traditional recommendation methods such as PMF , NMF and SVD are solely based on rating scores, and other methods HFT , DeepCoNN , Attn+CNN , NARRE and HUITA can exploit both rating scores and reviews for recommendation. Among the latter methods, HFT is based on topic models and cannot capture the contexts and orders of words. DeepCoNN and Attn+CNN simply concatenate reviews into a long document, and cannot model the informativeness of different reviews. Although NARRE can model review helpfulness via attention, it simply merges all sentences in a review together, and does not model the informativeness of different sentences and words. Different from these methods, our HUITA approach learns user and item representations from reviews in a hierarchical manner, and uses a three-tier attention network to select and attend to important words, sentences and reviews. The results of different methods are shown in Table 3. We have several observations from the results. First, the methods which exploit reviews (i.e., HFT , DeepCoNN , Attn+CNN , NARRE and HUITA ) usually perform better than the methods only based on rating scores (i.e., PMF , NMF and SVD++ ). It validates reviews can provide rich information of user preferences and item properties, and is important to learn informative user and item representations and can benefit recommendation. Second, among the method which can exploit reviews, the neural network based methods (e.g., DeepCoNN , Attn+CNN , NARRE and HUITA ) usually outperform the HFT method which is based on topic models. This is probably because in HFT the reviews are represented using bag-of-words features, and the contextual information and the orders of words are lost. This result validates the neural network based method can better capture the semantic information in reviews to model users and items for recommendation. Third, the methods considering review helpfulness (i.e., NARRE ) and word importance (i.e., Attn+CNN ) usually outperform DeepCoNN . This result implies that different words and different reviews have different importance for modeling users and items from reviews. Distinguishing important reviews and words from less important ones is beneficial to learn more accurate user and item representations for recommendation. Fourth, our approach can consistently outperform all the baseline methods compared here. This is because different from baseline methods such as Attn+CNN and DeepCoNN which merge all reviews into a long document and NARRE which merges all sentences into a long sentence, our HUITA approach learns user and item representations in a hierarchical manner. HUITA first learns sentence representations from words, then learns review representations from sentences, and finally learns user/item representations from reviews. Besides, our approach incorporates a three-tier attention network to jointly select and attend to important words, sentences and reviews. Thus, our approach can learn more informative user and item representations from reviews for recommendation. In this section, we conducted experiments to explore the effectiveness of the three-tier attention network in our approach. We compare three variants of our model by removing one kind of attention each time to evaluate its contribution to the performance. The results are shown in Table 4. According to Table 4, the word-level attention can effectively improve the performance of our approach. This is because different words in reviews have different importance in modeling users and items. Therefore, recognizing and highlighting the important words using the word-level attention network can help learn more informative sentence representations. In addition, the sentence-level attention is also useful. This may be because different sentences have different informativeness. For example, in a laptop review the sentence this laptop is expensive is more informative than I bought this laptop yesterday in representing this laptop. The sentence-level attention network can help to select important sentences to build review representations. Besides, the review-level attention is also useful in our HUITA approach. This is because different reviews have different informativeness in representing users and items. And distinguishing informative reviews from the less informative ones can help learn more accurate representations of users and items. Moreover, combining all the three levels of attentions can further improve the performance of our approach, which validates the effectiveness of our three-tier attention architecture. In this section, we conducted several case studies to further explore whether our approach can select informative words, sentences and reviews to learn informative user and item representations for recommendation. First, we want to explore the effectiveness of the wordand sentence-level attention networks. The visualization of the attention weights in the wordand sentence-level attention networks is shown in Fig. 3. From Fig. 3 we can see that our word-level attention network can effectively select and attend to important words. For example, in Fig. 3(a) the words Good, quality and recommend are assigned higher attention weights than bought and dad, since Good, quality and recommend can better model the properties of the film. In addition, our model can I bought this for my father. I purchased this book on a whim. I never cared for short stories when I was younger but I'm always willing to give something a try again, if I didn't have a horrible experience before. (b) Kindle Store Figure 3: Visualization of attention weights in two randomly selected reviews from the Movies and TV and Kindle Store datasets respectively. Red boxes to the left of the reviews represent sentence-level attention weights, and blue boxes on the individual words represent word-level attention weights. Darker color represents higher attention weights. effectively select informative sentences using the sentence-level attention network. For example, in Fig. 3(b) the sentence From reading I found short stories are still not my style is assigned high attention weight since it is informative for representing this user and is important for recommendation, while the sentence I purchased this book on a whim has low attention weight since it contains limited information of users and items. Thus, these results validate that our approach is effective in selecting informative words and sentences in reviews for recommendation through the wordand sentence-level attention networks. Second, we want to explore the effectiveness of the review-level attention in our HUITA approach. The visualization of the review-level attention weights is shown in Fig. 4. From Fig. 4 we can see that our approach can effectively select and attend to informative reviews. For example, the second review in Fig. 4(a) is assigned high attention weight by our approach since it reveals rich information of user preferences. However, the first review in Fig. 4(a) receives low attention weight since it contain limited information of users. Thus, these results validate the effectiveness of our approach in selecting informative reviews to learn more accurate representations of users and items from reviews for recommendation. In this paper, we propose a hierarchical user and item representation model with three-tier attention to learn user and item representations from reviews for recommendation. In our approach, we use a sentence encoder to learn sentence representations from words, a review encoder to learn review representations from sentences, and a user/item encoder to learn user/item representations from reviews. In addition, we incorporate a three-tier attention network into our model to select and attend to informative words, sentences and reviews to learn more accurate representations of users and items. Besides, we combine the user and item representations learned from the reviews with the embeddings of user and item IDs as the final representations of users and items to capture the latent factors of individual users and items. The experiments on four benchmark datasets validate that our approach can effectively improve the performance of recommendation and consistently outperform many baseline methods. This work was supported by the National Key Research and Development Program of China under Grant number 2018YFC1604002, and the National Natural Science Foundation of China under Grant numbers U1836204, U1705261, U1636113, U1536201, and U1536207.
abstain abstain abstain objective abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain objective abstain objective method method result other other other other other other other other other other other other other other objective objective abstain abstain method method method abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method other abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain method method method abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain other abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain abstain result abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain abstain result abstain result abstain result abstain abstain result objective method abstain method result other