<p>First assess whether the accuracy is unreasonably high.  You as the domain expert are the best arbiter of that.  Keep in mind that a high accuracy is not always impossible.  For example, 99% accuracy can be seen on MNIST digits dataset.  It is also possible that you have <a href="https://machinelearningmastery.com/data-leakage-machine-learning/" rel="nofollow noreferrer">data leakage</a>, so assess whether this may be at play. The second thing you can do is perform an <a href="https://scikit-learn.org/stable/modules/cross_validation.html" rel="nofollow noreferrer">n-fold cross-validation</a>.  This will provide you with a better estimate (and error bounds) for your model's performance.  If your accuracy remains high on all folds, then you can exclude a spurious split on your holdout validation.</p>
<p>Another issue is that you claim that you have an imbalanced dataset.  However, your print out shows roughly equal support for 0 and 1 classes.</p>
<p>Finally, you are performing preprocessing of your features on the total dataset.  Instead, you should apply <code>fit_transform</code> on your training data, and the just apply the <code>transform</code> on your test data.</p>

CC BY-SA 4.0
2022-11-11T04:25:23.990Z
I have an imbalanced dataset but the data I show above is just a sample of what my data looks like. Also, why should I should apply fit_transform on my training data, and just apply the transform on my test data?
JayTicku
CC BY-SA 4.0
2022-11-11T04:33:32.083Z
Ok, makes sense.  In that case, I would say that it is customary to perform evaluation on the total test set, not just a subset that you selected, regardless of whether your dataset is balanced or not.  Regarding the second issue of fit / fit_transform, the suggestion has to do with data leakage.  see [article](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage)
Snehal Patel
CC BY-SA 4.0
2022-11-12T01:10:04.347Z
I ran a classification report and it returned a score of 0.99 as shown above, so does that say that the model was able to generalize well?
JayTicku
CC BY-SA 4.0
2022-11-10T03:23:59.163Z
2022-11-10T03:23:59.163Z
Snehal Patel
<p><a href="https://i.sstatic.net/ruz9r.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ruz9r.png" alt="This is how a sample of my data looks like" /></a>
I am building a model that predicts if a user will like a stock or not based on different features, such as Market Cap, Current Ratio, Sector, Trailing PE, etc. I am going to implement this model in a website, where the model is able to adapt over time to user preferences. I made an example dataset that generally represents how the data that gets passed to the model will look. I have build a KNN model based on that sample dataset, and I get 99% accuracy on the training split and test split. I also get 98%-99% on my classification report even though I have class imbalance, about 500 of one class, and 10500 of another class. My macro average and weighted average are both 99%. I was planning on optimizing this model by hyper tuning, but does anyone know why I got 99% accuracy immediately? This is how a sample of my data looks like above.
Here is some of my code:</p>
<pre><code>X = data.drop([&quot;Stocks&quot;, &quot;target&quot;], axis = 1)
y = data[&quot;target&quot;]
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer

categorical_features = [&quot;sector&quot;, &quot;quoteType&quot;]
one_hot = OneHotEncoder()
transformer = ColumnTransformer([(&quot;one_hot&quot;,
                                  one_hot,
                                  categorical_features)],
                               remainder = &quot;passthrough&quot;)

X = transformer.fit_transform(X)

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X = scaler.fit_transform(X) 
X = pd.DataFrame(X)
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier

np.random.seed(42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)
knn = KNeighborsClassifier(metric = &quot;minkowski&quot;)
knn.fit(X_train, y_train)
knn.score(X_test, y_test)

y_preds = knn.predict(X_test)

from sklearn.metrics import classification_report
print(classification_report(y_test, y_preds))

This produces:
precision    recall  f1-score   support

           0       0.98      1.00      0.99      1227
           1       1.00      0.98      0.99      1012

    accuracy                           0.99      2239
   macro avg       0.99      0.99      0.99      2239
weighted avg       0.99      0.99      0.99      2239
</code></pre>

CC BY-SA 4.0
2022-11-09T04:19:23.417Z
Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/140489/discussion-on-question-by-messi10-why-does-knn-model-return-99-accuracy-on-data).
Mithical
<classification><binary-classification><accuracy><scikit-learn><k-nearest-neighbors>
Why does KNN Model return 99% accuracy on dataset with default parameters?
JayTicku
# Why does KNN Model return 99% accuracy on dataset with default parameters?

(Asked by: username_0 on 2022-11-09)

I am building a model that predicts if a user will like a stock or not based on different features, such as Market Cap, Current Ratio, Sector, Trailing PE, etc. I am going to implement this model in a website, where the model is able to adapt over time to user preferences. I made an example dataset that generally represents how the data that gets passed to the model will look. I have build a KNN model based on that sample dataset, and I get 99% accuracy on the training split and test split. I also get 98%-99% on my classification report even though I have class imbalance, about 500 of one class, and 10500 of another class. My macro average and weighted average are both 99%. I was planning on optimizing this model by hyper tuning, but does anyone know why I got 99% accuracy immediately? This is how a sample of my data looks like above.
Here is some of my code:
X = data.drop(["Stocks", "target"], axis = 1)
y = data["target"]
from sklearn.preprocessing import OneHotEncoder
from sklearn.compose import ColumnTransformer

categorical_features = ["sector", "quoteType"]
one_hot = OneHotEncoder()
transformer = ColumnTransformer([("one_hot",
                                  one_hot,
                                  categorical_features)],
                               remainder = "passthrough")

X = transformer.fit_transform(X)

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
X = scaler.fit_transform(X) 
X = pd.DataFrame(X)
from sklearn.model_selection import train_test_split
from sklearn.neighbors import KNeighborsClassifier

np.random.seed(42)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2)
knn = KNeighborsClassifier(metric = "minkowski")
knn.fit(X_train, y_train)
knn.score(X_test, y_test)

y_preds = knn.predict(X_test)

from sklearn.metrics import classification_report
print(classification_report(y_test, y_preds))

This produces:
precision    recall  f1-score   support

           0       0.98      1.00      0.99      1227
           1       1.00      0.98      0.99      1012

    accuracy                           0.99      2239
   macro avg       0.99      0.99      0.99      2239
weighted avg       0.99      0.99      0.99      2239

**Tags:** classification, binary-classification, accuracy, scikit-learn, k-nearest-neighbors

> **Comments:**
> **username_1**: Comments are not for extended discussion; this conversation has been [moved to chat](https://chat.stackexchange.com/rooms/140489/discussion-on-question-by-messi10-why-does-knn-model-return-99-accuracy-on-data).  

---

## 1 answers

### Answer by username_2 on 2022-11-10. Score: 1 | Accepted answer

First assess whether the accuracy is unreasonably high. You as the domain expert are the best arbiter of that. Keep in mind that a high accuracy is not always impossible. For example, 99% accuracy can be seen on MNIST digits dataset. It is also possible that you have [data leakage](https://machinelearningmastery.com/data-leakage-machine-learning/), so assess whether this may be at play. The second thing you can do is perform an [n-fold cross-validation](https://scikit-learn.org/stable/modules/cross_validation.html). This will provide you with a better estimate (and error bounds) for your model's performance. If your accuracy remains high on all folds, then you can exclude a spurious split on your holdout validation.

Another issue is that you claim that you have an imbalanced dataset. However, your print out shows roughly equal support for 0 and 1 classes.

Finally, you are performing preprocessing of your features on the total dataset. Instead, you should apply `fit_transform` on your training data, and the just apply the `transform` on your test data.

> **Comments:**
> **username_0**: I have an imbalanced dataset but the data I show above is just a sample of what my data looks like. Also, why should I should apply fit_transform on my training data, and just apply the transform on my test data?  
> **username_2**: Ok, makes sense.  In that case, I would say that it is customary to perform evaluation on the total test set, not just a subset that you selected, regardless of whether your dataset is balanced or not.  Regarding the second issue of fit / fit_transform, the suggestion has to do with data leakage.  see [article](https://scikit-learn.org/stable/common_pitfalls.html#data-leakage)  
> **username_0**: I ran a classification report and it returned a score of 0.99 as shown above, so does that say that the model was able to generalize well?  

---

