Accuracy reduces drastically when using TruncatedSVD with hashingvector
I have around 0.8 million product description with categories. There are around 280 categories. I want to train a model with given dataset so that in future I can predict Category for the given product description. Since the dataset was large I was not able to able TF-IDF on that data it throws MemoryError.
I found that Hashingvector was desirable when dealing with large data. 650
But when Hashingvector was applied I found it produced data with 1048576 features. It took around 1 hour to train and SGD model and produced 78% accuracy.
Code:
import pandas as pd from sklearn.feature_extraction.text import HashingVectorizer from sklearn.model_selection import train_test_split from sklearn.metrics import accuracy_score from sklearn.decomposition import TruncatedSVD from sklearn.linear_model import SGDClassifier from sklearn.calibration import CalibratedClassifierCV
data_file = "Product_description.txt" #Reading the input/ dataset data = pd.read_csv(data_file, header = 0, delimiter= "\t", quoting = 3, encoding = "utf8") data = data.dropna()
train_data, test_data, train_label, test_label = train_test_split(data.Product_description, data.Category, test_size=0.3, random_state=100, stratify=data.Category)
sgd_model = SGDClassifier(loss='hinge', n_iter=20, class_weight="balanced", n_jobs=-1, random_state=42, alpha=1e-06, verbose=1) vectorizer = HashingVectorizer(ngram_range=(1,3)) data_features = vectorizer.fit_transform(train_data.Product_description) sgd_model.fit(data_features, train_label) test_data_feature = vectorizer.transform(test_data.Product_Combined_Cleansed) Output_predict = sgd_model.predict(test_data_feature) print(accuracy_score(test_label, Output_predict))
Output:
Accuracy 77.01%
Since the dimension was high I thought reducing the dimension would increase the accuracy and reduce the training time. I used TrancatedSVD to reduce the dimension but this drastically reduced the prediction accuracy but reduced the training time to 10 minutes.
Code2:
from sklearn.decomposition import TruncatedSVD
clf = TruncatedSVD(100)
clf.fit(data_features)
Output:
Accuracy 14%
Edit:
When I tried TruncatedSVD with 1000 as limit it throws memory error so only I opted to use 100 as the limit.
It is stated that reducing n_features on HashingVector will cause collisions there can be collisions: distinct tokens can be mapped to the same feature index. However, in practice, this is rarely an issue if n_features is large enough (e.g. 2 ** 18 for text classification problems) in scikit site.
I got optimum accuracy when I used ngram between 1 to 3 so only used that. 392