Talent Recommendation on LinkedIn User Profiles

With the increasing amount of information on the Internet, recommender systems are becoming increasingly crucial in supporting people to find and explore relevant content. This is also true in the online recruitment space, with websites such as LinkedIn, Indeed.com, and Monster.com all using recommender systems. In online recruitment, it can often be challenging for companies to find suitable candidates with appropriate skills because of the huge volume of user profiles available. Identifying users which satisfy a range of different employer needs is also a difficult task. Thus, effective matching of user-profiles and jobs is becoming crucial for companies. This research project applies a wide range of recommendation techniques to the task of user profile recommendation. Extensive experiments are conducted on a large-scale real-world LinkedIn dataset to evaluate their performance, with the aim of identifying the most suitable approach in this particular recommendation scenario.


Summary
The aim of this research project is to identify appropriate recommendation approaches for the task of user profile recommendation on online recruitment environments based on existing recommendation techniques. In particular, the research conducted on this project focuses on the popular professional recruitment platform: LinkedIn. On LinkedIn, the primary approach for users to find jobs is by making applications for positions using their resumes/profiles provided on the platform. Companies, on the other hand, often use LinkedIn's Talent Solutions, which aims to help find and engage the most appropriate candidates, and user recommendation is one of its main features.
This research project first conducts extensive analysis on an large number of user profiles, i.e. resumes of LinkedIn users, and attempts to understand the distributions of needs of both job seekers and recruiters. Then, experiments are conducted on a largescale LinkedIn dataset to evaluate the performance of a range of existing recommendation approaches. Detailed analysis of experimental results is then presented in order to identify the most suitable recommendation approach in this particular application scenario.
v Introduction This chapter will introduce and discuss the challenge of talent recommendation, specifically using the LinkedIn user profiles. We will first outline the motivation for this research, then we will detail the research question addressed by this project. Finally, the research objectives of the recommender system to be developed will be provided.

Motivation
Recommender systems have been widely used in many application scenarios on the modern web because of the booming information stream that must be navigated. This research focuses on the domain of recruitment, specifically Job and Candidate recommendation. From the company's perspective, an effective recommender system could greatly reduce the workload involved in finding and engaging with appropriate candidates from the large number of user profiles available on job websites, such as Indeed, Monster, and job.com. In recruitment websites, a recommender system acts like a headhunter, identifying information such as: which user is the most suitable candidate for a specific job? Which candidates skills best fit a particular company's needs? Which company is a good target for a certain user? From the company's perspective, they need employees with certain specialized skills and work experience, who best fit their advertised positions. Meanwhile, users of online job websites also hope to find their dream job, which could upgrade their skills, expand their career opportunities and allow them to achieve their goals. 1 3 Online talent recommendation has recently become an important research topic. There is a significant number of existing works in this area, such as finding talent through recommendation in social networks by text-matching (Balog et al.2006). However, these approaches tend to have focused on recommending talent from general information, instead of recommending jobs/candidates using professional user profiles/resumes that include information such as user work experience, education, and skills. Thus our research aims to examine the performance of existing recommendation techniques when applied to this novel challenge.
Thus our motivation is to improve the process of talent recommendation in online recruitment sites through the adaption and application of existing approaches from the recommender systems and machine learning literature.

Research Question
We will try to investigate effective approaches for recommending talent on LinkedIn to companies that are looking to hire for a specific job using LinkedIn user profiles.

Thus, our research question is:
To what extent can recommendation techniques support talent recommendation, based on LinkedIn user profiles?

Project Objectives
This aim of this research project is to identify suitable recommendation approaches for the job/candidate recommendation task using professional user profiles/resumes, with a focus on the LinkedIn platform. In order to achieve this goal, this project has the following three major objectives: 1) Experimental Dataset Construction and Analysis. In this project, a large number of LinkedIn user profiles were harvested and used to construct an experimental dataset, in order to simulate the application scenario of job/candidate recommendation using user profiles in an online recruitment platform. On LinkedIn, a user profile consists of multiple fields, each of which details a particular aspect of information about the user, such as work experiences, educational experiences and skills, associations, etc. Thus, the specific aim of this project is to recommend suitable users for a particular position by using the structured textual information provided in their LinkedIn pro-files. In the experiments, we assume that the users who actually specified the first job experience in a job position are taken as ground truth users for the recommendation of that job position. Furthermore, an extensive analysis on the dataset is also needed to understand the needs of the proposed recommendation task, such as the distribution of job positions taken by the users and the distribution of the number of work/educa-tional experiences of users, and help to select and design appropriate recommendation approaches.
Draft of 2:58 am, Thursday, August 30, 2018 1 4 2) Recommendation Approach Selection and Design. Data Analysis on the constructed dataset illustrated the diversity of domains of job positions, e.g. there are over 200,000 different job positions discovered from the work experiences of the 158096 user profiles in the dataset, and the uncertainty in terms of the richness and completeness of provided user profiles, e.g. the number of work experiences a user specified in his/her profile ranges from 0 up to 7. Thus, it suggests that a learning-based recommendation approach would be well suited to the targeted application scenario, rather than rule-based or process-oriented approaches. This is because learning-based approaches have the advantage of automatically learning recommendation models for different domains of job positions, rather than separately designing different models for hundreds of thousands of job positions. Specifically, this project targets two categories of learning-based recommendation approaches in the proposed talent recommendation task: content-based recommendation approaches and collaborative filtering-based recommendation approaches. A content-based recommendation approach generally consists of two components: feature engineering based on the input content and selection of learning algorithms. A collaborative filtering-based approach also usually consists of two key components: construction of a rating matrix based on the input content and the selection of collaborative filtering algorithms. Thus, different feature engineering approaches, learning algorithms and collaborative filtering recommendation approaches need to be experimented with to examine the performance of the two categories of recommendation approaches.
3) Experimental results and analysis. Detailed experimental methods/steps need to be designed and implemented in order to identify the most suitable recommendation approach in the proposed talent recommendation task. They include defining the experimental environment, preprocessing of the dataset, selecting the evaluation metrics, and targeted evaluation aspects, e.g. the impact of the training size on the recommendation approaches, or the impact of the dimension of feature representation.

Project Outline
In the chapter2 we will present the literature review of the relevant researches about techniques that could be used in the job recommendation. We will discuss the state-ofthe-art and limitations of those techniques.
Chapter 3 will present and discuss the type of machine learning, approaches for recommender system, the statistic LinkedIn user data used by this research, detail of the design of the recommender system approaches and description of techniques used.
In the chapter 4 we will present results generated by various approaches from chapter 3. Then results will be evaluated based on selected metrics. Finally, we will indicate which approach can support the candidate recommendation for jobs.
Chapter 5 describe the conclusion drawn from this research, we will conclude the performance of selected approaches and give the conclusion of the recommender system.

Draft of 2:58 am, Thursday, August 30, 2018
1 5 We will also discuss the future works of our project. .

Introduction
Talent is considered as a candidate's capacity to improve their future career (Ashton & Morton2005). We will focus on techniques used for recommender system because the aim of the research is mainly concerns identifying approaches suitable for candidate recommendation for jobs. The term recommender system was defined for a long time ago by the research of (Resnick & Varian1997). However, the term recommender system represent a broad research field and there is ambiguity over what knowledge and techniques are involved in the field of job recommendation. (Paparrizos et al.2011), stated the term job recommendation refers to recommending candidate for jobs with machine learning techniques. Thus, we will give an overview of current techniques suitable for the realm of job recommendation. Based on a literature review, we discover that many techniques in job/candidate recommendation have focused on machine learning and feature representation. These researches will be presented in this chapter, from techniques of experimental data construction to techniques of learning algorithms for recommendation. This chapter will conclude with a discussion of what we have learned for the design of our recommendation system.

Talent Recommendation Techniques in Researches of Recommender System
There are a variety of traditional methods for talent recommendation on the web. (Becerra-Fernandez 2006). surveyed a large amount of talent-locator systems on the web, and a number of talent locator approaches that have been proposed are similar to CONNEX, which is a database containing a huge amount of user profiles with which companies can find users with desirable skills or other talents through a web browser interface. However, these approaches lack context information extraction and they can not predict the future jobs of candidates. Thus data mining techniques for user data is crucial for our research project. Based on the boom in research into feature representation and machine learning, we can find effective tools for feature extraction. We have found many types of research about techniques that could be used for talent recommendation.

N-grams for extracting context information of data
Comparing with early data mining technique, N-grams could extract the context information. The term N-grams has coined by (Brown et al.1992). Then many researchers have implemented the n-gram for talent recommendation. In an early research of (Su et al.2000), the researchers have developed a recommender system for predicting requests of the possible web page that users want to browse. The recommender system predict request by analyzing raw and filtered user log file with the n-gram model. They used unigram to 4-gram. As a result, the N-grams has shown great improvements of precision of prediction users viewing request.2.1 Other researches, like (Ashok et al.2009), the researchers developed Bug advisor which is a recommender system for debugging in the large software development projects. The researcher designed a baseline approach based on a full-text search i.e.
search full text of the error in the error report. The researcher analyzed the error report of the bug and extracted the context information using bag of words and call it the Bug feature search. The bug advisor improved 33% recall than the full-text search (e.g. find 33% more bugs compared with full-text set). In the research of (Stamatatos2011), the researcher implemented the n-gram model for the detection of plagiarisms based on the context information of the user profiles. and the results showed that using n-grams could provide high accuracy results (0.79) of plagiarisms of the document.

Word embedding for the expert recommendation
In the above section, we have considered extract context information in the user profile using N-grams. More techniques for feature representation need to be proposed. Word embedding could provide word vectors that keeping semantic information inside the document. Thus, we will consider using word vectors to extract more semantic information of user profiles.
The research of (Mikolov et al.2013)  There are some implementations of doc2vec (Vasile et al.2016) has developed a system for author profiling, they use doc2vec for feature representation and logistic regression classifier. The researcher selected data of blog post and social media, then the researcher set the baseline of the research of (Modaresi et al.2016), bag of words with logistic regression classifier, bag of c3grams with logistic regression classifier. The researcher then tested the doc2vec and collaborative filtering approach. Finally, the researcher calculated the accuracy of approaches and compare doc2vec with baselines. The result shown that doc2vec performs best with accuracy of 42.20 compared with baselines in the age data of the blog post.2.3

Dimension reduction techniques for feature representation
In order to lower the time costing and improve the usability of the recommender engine. We decided to implement dimension reduction techniques (e.g. Singular Value Decomposing) in our recommender system because the learning algorithms could be SVD, the singular-value-decomposition, as its proved in the previous researches. The SVD could greatly reduce the dimension they needed for the research, and it has shown the effect of many machine learning projects. The (Koren et al.2009) use SVD in the recommender systems, the author used the SVD to reduce the dimension of the vector space of movie features. The movie list is split to a train set and a test set, but the dimension of the movie matrix is too high and the sparse rate is 95.4% (4.6% non-zero data) is not acceptable for the recommendation engine, thus, in order to save time for the training, the researcher used the SVD to save most important information extracted from movie data After the implementation, the dimension of movie data is reduced to 700 and the F1 score dropped from 0.16 to 0.14, which is acceptable for the implementation. Now we have discussed techniques of the n-gram model and SVD. Now we will discuss the machine learning techniques used in our system for candidate recommendation for jobs.

Content-based recommendation
The research of (Paparrizos et al.2011) use the machine learning algorithms for job recommendation based on an employees job transaction. They first designed one baseline approach, which is built by predicting the most frequent job transaction in the user profile. Then they tested data using hybrid algorithm of nave bayes classifier. Finally,

Collaborative filtering recommendation
Collaborative filtering is one of the most widely-used approaches in the recommender system. The researchers in (Lee et al.2001) have listed approaches i.e. users based and item based collaborative filtering approaches, and evaluation metrics i.e. root mean square error, precision and recall Approaches like ( (Li et al.2005), use a collaborative filtering approach for talent recommendation in e-commerce, they use candidate profile, job description, and candidate relationships for talent recommendation of job candidates. They successfully implemented a collaborative filtering talent recommendation with MAE lower than 0.2.

What I learned
By reviewing the literature which describes existing research in the area of recommender systems and machine learning, I have learned the following: In order to build a successful recommender system, the first vital step is feature representation of the data. In the feature representation part, there are a range of techniques (i.e. N-grams, word embedding) that are capable of the feature representation. T h e findings of research into using different learning algorithms are also very interesting. We find that combining different machine learning algorithms and feature representation techniques could greatly increase the quality of recommendation achieved.
Another interesting result, based on the investigation of existing research, is that many researchers build recommender systems that follow a process of: (Data preprocessing ¿ Feature representation -¿ Training the model using learning algorithms-¿ recommend items) based on the research of (Paparrizos et al.2011) and (Ashok et

Introduction
In this chapter, we will present the design and technical details of our recommender system. Initially, different learning types will be presented and discussed and we will select the most suitable learning type for our research aim. Then, based on the selected learning type, approaches to our recommender system will be selected. After selecting suitable approaches to the recommender system, data preprocessing, learning algorithms and feature representation techniques will be selected. Finally, the design pattern of the system will be presented to maintain the robustness and scalability of our recommender system.

System
Machine Learning is a method of data analytics that aims to allow a computer to learn and make predictions regarding data based on an algorithm. Generally speaking, there are three types of machine learning: Unsupervised learning: The machine learning algorithm tries to create a model purely based on the input. It looks for the inner patterns and structures in the data.
Supervised learning: The algorithm is presented with both input and the desired output. The algorithm aims to create a model that will best match the given inputs to the outputs. The goal of supervised machine learning is to build a concise model of the distribution of class labels in terms of predictor features (Kotsiantis, Zaharakis, and Pintelas, 2007).
Reinforcement learning: The algorithm is presented with a dynamic environment and a desired goal. It must learn which are the optimal actions it must undertake in order to achieve the target goal based on whatever state it is in.
In our research, supervised learning algorithms will be selected. The reason we select supervised learning is that unsupervised learning is based on unlabelled input and attempts to learn from the raw data, however, we prefer supervised learning for classification of candidates has we have training data with labelled job positions. For reinforcement learning, the environment of recommendation in this research is not dynamic, we use static data for recommendation and we do not have any dynamic status for our data thus there is no need for implementing reinforcement learning in the recommender system. The recommender system recommends candidates for jobs and we can use LinkedIn profile data as the input and job position as the output in supervised learning. In the following section we will introduce the approaches to recommender system based on our selection of learning type.

Selecting Recommender System Approaches
We have selected the learning type for our recommender system. Then we will discuss approaches in the recommender systems. The recommender system has mainly three popular approaches: content-based, collaborative filtering and hybrid approach.
The content-based and collaborative filtering approach for building approach will be selected for recommendation. The reason is that the content-based approach could tell us what feature are used in the recommendation. Thus we could evaluate the importance of features based on content-based approach. Another advantage for contentbased approach is that it could solve the cold-start problem (i.e. the recommender system could not recommend similar items when adding a new item without any related information) in our recommender system because new items can be suggested without user ratings.
Collaborative filtering approach will also be selected because of following advantages like: Collaborative filtering approach work best when the user space is large (since that is their source of data). Content-based approach, however, needs technique for dimension reduction because content-based approach is more or less insensitive to user size. Meanwhile, adding collaborative filtering could fit our goal of the diversity of approaches for candidate/job recommendation. The hybrid recommendation approach will not be considered in this research because our literature review hasnt covered the related researches. We will implement it in the future works.
Thus, based on the techniques used for collaborative filtering, we will select contentbased and collaborative filtering approaches in our recommender system. We have developed recommender approaches from two aspects: content-based recommendation approaches using three different user feature representations, and collaborative filtering approaches. We will test the performance of different learning algorithms and find the best approach for the recommendation of the user profiles.

Design and Techniques of Content-based Recommender System
The high-level workflow of the recommender system is shown in Figure3.1 We will introduce important components of our design: Data preprocessing: In order to preprocess our data, we need to focus on two aspects: Data statistics and the design of a training/testing set. For data statistics, we will describe the details of the dataset and discuss what attributes of the user profile information could be used for different approaches in our recommender system. For generating training and testing set, we will discuss the feature engineering of the user data and then we will describe how training and testing data is generated for supervised learning.
Feature representation: We will implement 3 techniques to represent user feature. First is the N-grams: the language model of using N-grams to analyze the contextual information in user profiles. We will demonstrate how N-grams works in our dataset and how to construct the feature data with our dataset. Then we will bring out word embedding and document embedding: the technique of reflecting word or document as feature vectors and how this technique will be implemented in our data. Finally, we will apply dimension reduction technique: the approach for dimension reduction in matrices formed by n-gram approaches. We will show why we need dimension reduction of feature vectors generated by N-grams and how to implement them in our approach.
Learning algorithms for the recommendation are: Learning algorithms for contentbased recommendation: we will discuss different supervised learning algorithms in machine learning for user profile recommendations, describe the reason for applying algorithms and the parameter settings of different algorithms.

Data analyzing and Preprocessing for Content-based Approach
In this section, we will present attributes of the data in the beginning. Then we will do data statistics and analysis.
Data cleaning for user profile data: The mix of number and words could be disturbing the performance of the learning algorithm, thus for attributes like work duration of work experience, the work experience is described as string of x year y month(s) in the original dataset, thus we use the regular expression to match the work experience string and convert them to numbers. Finally, we will normalize them to decimals between 0 and 1 in our data. The dataset used in the research is collected from the popular professional social networking site LinkedIn, which consists of 158096 users. In our LinkedIn dataset, we have a CSV file with 158012 user profiles in the dataset containing background information of user talent. For every user, they have 67 attributes that can be categorized as following: 1. User id 2. Username 3. Connections number of user 4. Seven parts of work experience (current and past work experience, combined with job title, company name, company type, work duration, company location) 5. Four parts of educational backgrounds (university name, degree of education, major, end date of education, education details) 6. Skills 7. Languages Those attributes could describe the general information of the user profile with their talents.
Analyzing of attributes: we found some attributes with insignificant information, pure numbers or combination of number and words: user id (section1), username (section2), connection number of the user (section 3), work duration of work experience and graduation year of the education background (section 5). In this research we will remove username, con-nection number of user and graduation year of the school in the data we use because for username and connection number, we could neither dig more semantic nor contextual data those attributes. Remaining attributes will be analyzed through data statistic. Data cleaning for user profile data: Before the statistic of data, we need to implement the data cleaning to avoid the obstruct of messy data. The mix of number and words could be disturbing the performance of the learning algorithm, thus for attributes like work duration of work experience, the work experience is described as string of 'x year y month(s)' in the original dataset, thus we use the regular expression to match the work experience string and convert them to numbers. Finally we will normalize them to decimals between 0 and 1 in our data.
Data analysis for job titles: In our dataset, the top 15 job titles in the dataset are shown in Figure3.2 We need to analyze job title based on following reasons: For online recruitment websites, the foremost attribute in a job is the job title and most of users in job websites find jobs based on job titles. Thus we could set user data for the input of recommender system and set the prediction of job title as output of our system. Based on the statistic of job titles, we selected six different job titles for recommendation LinkedIn user to job (account manager, software engineer, research assistant, project manager, process engineer, consultant). The selected job titles are from different areas and some job titles have clear statement (software engineer), some are not (e.g., consultant). Thus, we could test the performance of different approaches thoroughly by using different job titles.
Work experience statistic: we want to learn the work experience in our LinkedIn users.
Thus we count the number of work experience per user. According to the statistical data of our user profiles. The number of effect value in seven work experience is shown in In this diagram, recent means the current job position of user, and past1-6 means the past work experiences of user. Thus for every user, the average number of effective work experience number per user is 2.19, thus for every user profile there is at least 2 work experience, we could use this feature in our recommendation system. Education background of the user: We want to investigate education in LinkedIn user profile. The Figure3.4showed the total number of education experience in four education experience. Thus based on the statistic data, the average education background per user is 1.76, which can be used for talent recommendation. This work also did the statistics on the averaged number of skills per user listed on his LinkedIn profile. Skills in the user profile are given by a long sentence separated

Building Training set and testing set and Setting label data for Content-based recommendation
Based on the analyzing and prepossessing of the data in the section3.5, we decided to use the section 4,5,6,7 (i.e. seven work experience, four education background, skills and languages attributes) of the user data in the section three, part of LinkedIn user profile data.
Supervised learning for job recommendation is chosen in this research and our research aim is to identify suitable approaches for candidate recommendation for jobs. Thus, for a specific job title, we could judge whether a user is suitable for given job title by setting the LinkedIn user profile data as input, and the output is the binary value of the prediction of user job title (i.e. If the job title is the chosen job title, the value will be 1, otherwise the value will be 0). Thus we selected the job title of the first work experience in the seven work experience as labels.
In our project, we will select 1000 users based on chosen job title. We will randomly select 500 users whose label is 1 (i.e. job title of first work experience is same with chosen job title) and 500 users whose label is 0(i.e. first work experience job title is not the chosen job title). The process of generating train set and test set is shown in Figure3.5

Recommender System
In this section we will contain 3 techniques of feature representation: N-grams model, word embedding and document embedding and 8 learning algorithms in the contentbased recommendation approach. We will present the details for those techniques and how we implement them in our recommender system.

Feature representation using N-grams
N-grams is widely used in probability, communication theory, computational linguistics (for instance, statistical natural language processing), computational biology (for instance, biological sequence analysis), and data compression. The advantage of using N-gram models (and algorithms that use them) are simplicity and scalability with if the sentence contains the word the value will be 1, otherwise 0). We can see N-grams could use context information in the user profile based on setting the N value (i.e. Length of tokens) in N-grams. To extract context information, we propose to use the ngram to extract the context information of users. Concretely, for a candidate, his profile will be tokenized (e.g. for trigram(length of token is 3), the text software engineer work in microsoft will be tokenized as list of software engineer work, engineer work in, work in microsoft) By doing so, the context information can be saved. In the experiments of this research project, following approaches are considered: We can see n-grams could use context information in the user profile based on using different n-grams. To identify experts we propose to use the n-gram to extract the context information of users. Concretely, for an expert candidate, his user profile will be split into a collection of n words combinations (e.g. for 2-gram split text senior software engineer as senior software,software engineer) thus every n-gram text will be related to its prior text. By doing so, the context information can be saved. We select a sentence 'consult three years deloitte' from the user profile and experiments from Ngrams are considered: 1. Unigram, i.e. bag-of-words the length of token is 1, the approach is illustrated in

Feature Representation Using Word/document embedding model
We already finished feature representation using the n-gram for the content-based recommendation and contained partially contextual and semantic information in every user profile. Now we need to investigate more approaches for extracting semantic in- Word2vec is a well-known tool that builds word embedding using the hidden neural network for prediction of words. Based on the research by Omer Levy (Goldberg & Levy2014), the advantage of using word2vec is that it works well on many semantic tasks like word similarity and word analogy, another advantage is that word2vec could use many pre-trained models such as the Glove and Google negative 300 and they could provide fairly good performance.
In order to extract the semantic information using word2vec, we need to tokenize each user profile into a list of tokens, then we could use pre-trained word embedding models (such as Glove, Google negative300) to convert words in the user profiles to word vectors, after that, we could calculate the vector of the user profile by calculating the average of the sum of word vectors. Finally, we could use user profile vectors in our learning algorithms. The vector of user profile contains the semantic information of words in the data. The process of forming user profile vector is shown in Figure3.9.
The doc2vec is the further development of the word2vec, in the input layer of doc2vec, the doc2vec adding a new paragraph vector on the basis of word2vec and it will produce word vector for each word in the document and document vector for each document by adding a list of documents as training data.The Figure3.10shown how to extract semantic information inside documents.
To implement document embedding with LinkedIn user profile data, we need to train the user profile dataset with doc2vec and get word and document vectors. The document vectors contain the context information of paragraphs in the user profiles.
Then we set every document as a input, the doc2vec will transfer the input document to a vector based on existing word and document vectors in the model.

Approaches for Dimension Reduction in Content-based Recommendation
The N-gram model could extract context information in the LinkedIn user profile data. However, feature vectors formed by N-grams are usually with high dimension shown in Many problems caused by sparse matrices: The sparse matrix will increase the training time of the user profile data, which can be seen in table3.2and a long training time is unacceptable for many recommender systems.  The SVD algorithms can be used in the dimension reduction of vector space, which can reshape the vectors and contains the most important features inside the user profile and give faster results.
The explanation of the singular value decomposition is shown as follows: In theory, the singular value decomposition is to decompose the matrix into the sum of several rank 1 matrix, then save the value with the high singular value in the formula inside the matrix and remove value with the low singular value inside the formula. Thus, the generated new matrix is smaller than the original matrix and it could reduce the training time of the model. In our example, we will use a picture with pixel 450*333. We form a matrix A with 450 row x 333 column based on the pixel of the picture.

Draft of 2:58 am, Thursday, August 30, 2018
26 Then we do the singular value decomposition to the matrix and get the result: The formula of the SVD is shown in Formula3.1: where σ is singular value, u and v are the column vectors.
If we only contain the first value of the formula, which contains a highest singular value, the formula is3.2 We can see the picture is still blurry in figure3.11because we captured few features of the picture.
If we contain the first 5 values of the formula, which is shown as following: We could see the outline of the figure in3.12 If we contain the top50 values of the formula, the picture is: We can see the new picture is very closed to original picture in figure3. 13 The new picture that contains the top 50 features has the pixel of 784*50 = 39200, which is much smaller than the original picture with the pixel of 450*333 = 149850. The reduced matrix using SVD could greatly reduce the training time of the model, the result will be shown in the next chapter. Thus, in our project, the feature vectors of LinkedIn users will be processed through singular value decomposition from high dimension to 50 dimension, preserve essential features while removing insignificant features.

Learning algorithms for Content-based Recommendation
Different algorithms will be tested on those vector spaces. We will present eight different learning algorithms and demonstrate how we implement them: The formula is shown as following: where .
Here w1, w2.....wn are the regression co-efficients of the model and are calculated by Maximum Likelihood Estimation and x1, x2.....xn are variables of formed feature vectors of LinkedIn user data. f(t) calculates the probability of the binary outcome and using the probabilities we classify the given LinkedIn user data into one of the two categories. The logistic regression CV is Logistic regression using k-fold cross validation.
2) SVM Support vector machine (SVM) is a supervised learning algorithm that could be used for regression and classifying. In this project we will use SVM for classification.
The reason why we implement SVM algorithm has several advantages like: SVMs can produce accurate and robust classification results on a sound theoretical basis, even when input data are non-monotone and non-linearly separable. SVM have the benefit of good performing in the high dimension spaces such as the feature data in our The SVM algorithm works by plotting data as points in the n-dimensional space (Where n is the feature number in the LinkedIn data profile). Then SVM will find a hyperplane which could best split two datasets. The figure is shown in figure3.15.
In our project, we use SVM SVC (C-Support Vector Classification), SVM NuSVC (Nu-Support Vector Classification), SVM Linear SVC (Linear support Vector Classification) to test our data.

3) Naive Bayes
We decide to implement the Naive Bayesian classifier based on following reasons: it is easy to build, with no complicated iterative parameter estimation which makes it particularly useful for very large datasets. Despite its simplicity, the Naive Bayes classifier often does surprisingly well and is widely used because it often outperforms more sophisticated classification methods.
In our research, we will set the user profile as the input and then the classifier will calculate the conditional probability of features for classification and then classify LinkedIn user data to suitable job titles.

4) Decision Tree
The decision tree is supervised learning algorithm which builds classification or regression models in the form of a tree structure. It breaks down a dataset into smaller and smaller subsets while at the same time an associated decision tree is incrementally developed. The final result is a tree with decision nodes and leaf nodes. A decision node (e.g., Outlook) has two or more branches (e.g., Sunny, Overcast and Rainy). Leaf node (e.g., Play) represents a classification or decision. The topmost decision node in a tree which corresponds to the best predictor called root node. The structure of model is shown in Figure3.16. The advantage of decision tree is: Easy to understand and can handle both categorical and numerical data.

5) Random Forest
Random forest is an ensemble machine learning technique used to perform classification. Ensembles combine multiple hypotheses to form a (hopefully) better hypothesis. The main principle behind ensemble methods is that a group of weak learners can come together to form a strong learner. Random forest uses a multitude of decision trees (weak learners) to perform its classification, which is shown in Figure3.17. The random forest could average or combine the results of different decision trees helps to overcome the problem of overfitting. We will put the user profile data as input and output the target value (i.e. prediction of the job title for LinkedIn users). In this re-search , the feature data will be the input of the learning algorithm and the prediction of job title will be the output.  The design of collaborative filtering is shown in figure3.18. We will introduce main components in the collaborative filtering approach: Data preprocessing: In the data preprocessing part we will discuss what attributes will be suitable for collaborative filtering approaches.

34
Forming matrix for learning algorithm: Process of forming user rating matrix will be presented and we will illustrate how we form the rating value for collaborative learning algorithm.

Data preprocessing for Collaborative Filtering Recommendation
For the collaborative filtering approach, we usually need to form a matrix based on our goal for the recommendation. For example, for a recommendation in e-commerce, we usually need to form user-item matrix to find similarity between customers. For a similar book recommendation, we usually need to build item-item matrix to show the inner connection between books.
Thus in our collaborative filtering approach, we decided to form the matrix of userworkexperience-rating matrix (i.e. user as row of matrix, work experience attribute except the first work experience as columns of matrix and value in cells is the rating of work experience, which will be discussed thoroughly in the feature representation section) for collaborative filtering learning algorithms. The reason why we implement the strategy is that the collaborative filtering approach could fit our research aim (i.e. suitable approach for candidate recommendation for jobs). Thus we consider using work experience in the collaborative filtering approach. In order to build data for supervised learning, the job title of first work experience will be the output of learning and user-workexperience-rating matrix will be the output.

Building Matrix for Collaborative Filtering Approach
We will build a user-workexperience-rating matrix based on the user profile data. The work experience (Columns of matrix) is six work experiences of a user mentioned in section 3.3 and we calculate the rating based on the following rules: For a specific given job title (i.e. software engineer), we will calculate the rating based on following rules: 1. If the job title in the work experience is same as the given job title and the work year is more or equal than 3 years, the rating value is 2.
2. If the job title in the work experience is the given job title and the work year is less than 3 years, the rating value is 1.
3. If the job title in the work experience is not the given job title, the rating value is -1.
Thus we could form a matrix of user-workexperience-rating in figure3.133.19

Learning Algorithms for Collaborative Filtering
We will test 2 different collaborative filtering algorithms in the research: SVDPlusPlus Recommender ASVDPlusPlus.

SVDplusplus Leanring Algorithm
In an easier way to explain terminology, SVD (i.e. singular value decomposition) finds the latent factors associated with some matrix. For example, in recommender systems, the user-rating matrix of movies after an SVD, will decompose into matrices that represents latent user-user features and item-item features e.g. same gender user, same age-group of users, action movies etc. and many other latent factors involved in the rating behavior that is not apparent from the user-rating matrix.3.19Based Figure 3.20: Theory of SVDplusplus on the research of (Koren2008), the SVDplusplus (i.e. singular value decomposition plus plus) incorporate the implicit feedback of the rating data (i.e. Possible preference for a specific job title such as data scientist for a user with work experience such as software engineer) by generating feature matrix by matrix factorization. In this research, the rating value in matrix will be selected as the input and the output of matrix factorization will be feature matrix with lower dimension. Thus, in order to find out the implicit information of user feature

ASVDplusplus Learning Algorithm
The ASVDplusplus (Asymmetric SVDplusplus) is the simplified version of the SVDplusplus learning algorithm because of reduced parameter in the feature extraction function. In this research, the rating value in matrix will be selected as the input and the output of matrix factorization will be feature matrix with lower dimension. Thus, in order to find out the implicit information of user feature, we will implement ASVDplusplus algorithm in the collaborative approach.

Introduction
Organized code style and structure is essential for software engineering. Wellorganized code will perform better in usability, scalability, and robustness of the software system.
We will develop the system based on the following concepts in order to make the recommender engine more robust and scalable.

Object oriented programming
In our system, we developed the object-oriented advantages of the object-oriented programming: Object-Oriented Programming has great advantages over other programming styles: Code Reuse and Recycling: Objects created for Object-Oriented Programs can easily be reused in other programs.
Encapsulation (part 1): Once an Object is created, knowledge of its implementation is not necessary for its use. In older programs, coders needed to understand the details of a piece of code before using it (in this or another program).
Encapsulation (part 2): Objects have the ability to hide certain parts of themselves from programmers. This prevents programmers from tampering with values they shouldnt. Additionally, the object controls how one interacts with it, preventing other kinds of errors. For example, a programmer (or another program) cannot set the width of a window to -400. Design Benefits: Large programs are very difficult to write. Object-Oriented Programs force designers to go through an extensive planning phase, which makes for better designs with fewer flaws. In addition, once a program reaches a certain size, Object Oriented Programs are actually easier to program than non-Object Oriented ones.
Software Maintenance: Programs are not disposable. Legacy code must be dealt with on a daily basis, either to be improved upon (for a new version of an existing piece of software) or made to work with newer computers and software. An Object-Oriented Program is much easier to modify and maintain than a non-Object Oriented Program. So although a lot of work is spent before the program is written, less work is needed to maintain it over time. In our system, we will form a class named LinkedIn data, the class will have 67 attributes corresponded to 67 columns in the dataset. The procedure of generating data is 1. Initiating the data: every row in our dataset is a user profile and every column is the attribute of the user, thus we generate the LinkedIn data class while reading every line of the CSV file. Then we generate the data into the list of the class.
Based on the analysis of our data, the organization of our data is shown in Figure  3.21: As a result, the attributes of the user profile class in Python is shown as follows: s e l f . l a n g u a g e s = l a n g u a g e s

Research Results
To evaluate the performance of various recommendation approaches on the constructed LinkedIn dataset, We will first introduce the metric for evaluation of our approaches. Then we look at different aspects of the performance of these approaches: training time, performance with different sizes of training data, different learning algorithms and different feature representation of data.

Evaluation metrics for recommender system Precision and Recall
For better evaluation of different approaches, we will calculate the precision, recall and accuracy value of different approaches. The precision formula is shown as follows: The recall formula is shown as follows: Considering our research aim, we will calculate average precision and recall of every approach by average the sum of precision value and recall value for all select job titles.

Experimental results and analysis
In this section, we will present experimental results and result analysis. We will split the evaluation in two main section: Evaluation for content-based approaches and evaluation for collaborative filtering approaches. In the evaluation of content-based approach, results from be presented based on different feature representation, then in every feature representation results, the performance of different learning algorithms for different job title will be presented. In the evaluation of collaborative filtering approach, the results of different learning algorithms for different job title will be discussed.

N-grams for Feature Representation in the Content-based Approach
We have results of 3 different N-grams (Unigram, Bigram, Trigram) for feature representation and 8 different learning algorithms (Logistic regression, Logistic regression CV, SVM Linear SVC, SVM NuSVC, SVM SVC, Nave Bayes, Decision Tree, Random forest, ) and 4 different parts of user data (All data, user work experience of the data, education background of user data and skills of user data). In total we have 96 results. In order to identify approaches that could support talent recommendation, we will evaluate them by calculating the average precision of all job titles for every   Table 4.8: Average recall for all job positions of using skills data Analysis of the n-gram feature representation: Based on above tables, we could find that 83.3% of highlighted approaches are using Trigram for feature representation. Meanwhile, we could find that all highlighted approaches using Random forest as the learning algorithm for recommendation and the combination of (trigram + random forest learning algorithm + user skills data) outperforms other combinations with precision value of 0.9015. Thus, based on our analysis, the best content-based approach using N-grams as feature representation is Trigram + Random forest learning algorithm + user skills data. The approach with highest recall is the combination of bigram, naive bayes learning algorithm for all user data and the value is 0.774.

Dimension reduction VS Non Dimension Reduction
In our research we implemented the singular value decomposition in the N-grams feature representation technique in order to reduce the dimension of feature vectors. The dimension reduction have greatly reduced the training time of data. The We can observed that The best result of dimension reduction(highlighted red value) is Nave Bayes, reduced 99.9016% of original Training time.

Document Embedding as Feature Representation in Content-based Approach
Highest precision for different job positions using document embedding and different learning algorithms (Logistic regression, Logistic regression CV, SVM Linear SVC, SVM NuSVC, SVM SVC, Nave Bayes, Decision Tree, Random forest, ) are shown in table4.10: Then the average precision of job titles for every approaches are shown in table4.11: Thus, based on our above tables, the approaches of document embedding + logistic regression algorithms have best performance with the highest average precision of all job titles.

Word Embedding as Feature Representation in Content-based Approach
The result of word embedding with different learning algorithms (Logistic regression, Logistic regression CV, SVM Linear SVC, SVM NuSVC, SVM SVC, Nave Bayes, Decision Tree, Random forest, ) is shown in table4.13. Then we calculate average precision of job positions for every approaches, the result is shown in table4. 15 We could see from table4.13that highest precision of recommendation for job title 'software engineer', 'research assistant' and 'process engineer' is much higher than pre-cision of job title 'consultant', 'project manager' and 'account manager'.

Evaluation for Collaborative Filtering Approaches
The result of the collaborative filtering approach with learning algorithms (SVDplus-

Discussion of Results
We have calculated the average precision of different job titles using a content-based approach with different feature representations, and using a collaborative filtering approach. We will discuss the results based on the metrics used.
The approach with highest precision is the content-based approach combining trigrams with the random forest learning algorithm. This approach outperforms other content-based and collaborative filtering approaches with a precision of 0.9015, which means 90% of the predicting positive is correct. Other approach's average precision are between 0.7 to 0.9. The approach with the highest recall is the content-based approach combining bigrams with naive bayes and all user data. The recall value is 0.774, which means 77.4% of positive samples are predicted positive in average for all job titles. We also find that this approach is still the best performing in terms of recall when we tried all different parts of the data.
Meanwhile, by implementing singular value decomposition for reducing the dimensionality of feature vectors, we have found that training time is greatly reduced, which is significant for online recommendation because users usually need real-time recommendation in an online environment (i.e. users on job websites will receive realtime recommendation of different jobs based on operations like updating their user profile and viewing j o b history).

Chapter 5 Conclusion and Future Work
In this chapter, we will conclude our findings of different approaches and the performance of our recommender engine. At the end of this chapter, we will discuss the futures works of our project and share some thoughts about that.

Conclusion of the Recommender System
The recommender engine developed by this research is designed to find the best approach for recommending the job title by combining machine learning and language models. This engine has been successfully implemented.

Conclusion of Results
The results presented in Chapter 4 illustrated that the content-based approach of combining trigrams with the random forest learning algorithm and skills has the highest average precision for all job titles in talent recommendation. The content-based approach of combining bigrams with the naive bayes learning algorithm and all user data has the highest average recall for all job titles. Thus from the perspective of accurate prediction, a combination of the trigrams, random forest learning algorithm and skills could support the most effective talent recommendation. From the perspective of high coverage of the correct prediction, the combination of bigrams, naive bayes learning algorithm and all user data is the most suitable approach for talent recommendation. The singular value decomposition technique for dimension reduction has shown great effectiveness in reducing the training time of the feature data by 99%, which would greatly improve the efficiency of talent recommendation.

Future Works
We have identified the approaches which have the best precision@n and recall@n by testing different machine learning algorithms and different language models. In the future, we will focus on the following aspects to improve our recommender engine: 1.Using practical ratios of data: the training set and testing set in current project is small, and the ratio of the data (50% positive 50% negative) is idealized, in the following steps of the optimization, we will test with real ratios (such as 60 relevant and 1520 non-relevant users) and change the ratio to more accurately mirror the real world situation faced by talent search.
2. Consider implementing more techniques based on state-of-the-art research. Other state-of-the-art research such as knowledge graph embedding could also be considered for talent recommendation (i.e. knowledge graph embedding). We could also consider building a hybrid recommender engine to maintain the precision and quality of recommendation (i.e. for top 200 recommendation, the top 30 result can be selected from approaches with high precision, while the rest of the recommendation can use the results from high recall value).