which had better results, the svm or the random forest model?
SVM
The text classification is one of the most critical areas in machine learning and artificial intelligence research. It has been actively adopted in many business applications such as conversational intelligence systems, news articles categorizations, sentiment analysis, emotion detection systems, and many other recommendation systems in our daily life. One of the problems in supervised text classification models is that the models performance depend heavily on the quality of data labeling that are typically done by humans. In this study, we propose a new network community detection-based approach to automatically label and classify text data into multiclass value spaces. Specifically, we build a network with sentences as the network nodes and pairwise cosine similarities between TFIDF vector representations of the sentences as the network link weights. We use the Louvain method to detect the communities in the sentence network. We train and test Support vector machine and Random forest models on both the human labeled data and network community detection labeled data. Results showed that models with the data labeled by network community detection outperformed the models with the human-labeled data by 2.68-3.75% of classification accuracy. Our method may help development of a more accurate conversational intelligence system and other text classification systems.
Text data is a great source of knowledge for building many useful recommendation systems, search engines as well as conversational intelligence systems. However, it is often found to be a difficult and time consuming task to structure the unstructured text data especially when it comes to labeling the text data for training text classification models. Data labeling, typically done by humans, is prone to make misslabeled data entries, and hard to track whether the data is correctly labeled or not. This human labeling practice indeed impacts on the quality of the trained models in solving classificastion problems. Some previous studies attempted to solve this problem by utilizing unsupervised BIBREF3, BIBREF4 and semisupervised BIBREF5 machine learning models. However, those studies used pre-defined keywords list for each category in the document, which provides the models with extra referencial materials to look at when making the classification predictions, or included already labeled data as a part of the entire data set from which the models learn. In case of using clustering algorithms such as K$-$means BIBREF4, since the features selected for each class depend on the frequency of the particular words in the sentences, when there are words that appear in multiple sentences frequently, it is very much possible that those words can be used as features for multiple classes leading the model to render more ambiguity, and to result a poor performance in classifying documents. Although there are many studies in text classification problems using machine learning techniques, there has been a limited number of studies conducted in text classifications utilizing networks science. Network science is actively being adopted in studying biological networks, social networks, financial market prediction BIBREF6 and more in many fields of study to mine insights from the collectively inter-connected components by analysing their relationships and structural characteristics. Only a few studies adopted network science theories to study text classifications, and showed preliminary results of the text clustering performed by network analysis specially with network community detection algorithms BIBREF7, BIBREF8. However, those studies did not clearly show the quality of community detection algorithms and other possible usefull features. Network community detection BIBREF9 is graph clustering methods actively used in complex networks analysis From large social networks analysis BIBREF10 to RNA-sequencing analysis BIBREF11 as a tool to partition a graph data into multiple parts based on the network's structural properties such as betweeness, modularity, etc. In this paper, we study further to show the usefulness of the network community detection on labeling unlabeled text data that will automate and improve human labeling tasks, and on training machine learning classification models for a particular text classification problem. We finally show that the machine learning models trained on the data labeled by the network community detection model outperform the models trained on the human labeled data.