Stack Exchange
Top 3 classes in predict_proba()
I am working on a multi-class text classification problem that requires the top 3 predicted labels with the corresponding probability. I am able to use sklearn predict_proba(), but is having a hard time formatting the outputs like in table A. My code is below:
cv = StratifiedKFold(n_splits = 10, random_state = 42, shuffle = None)

pipeline_sgd = Pipeline([
     ('vect', CountVectorizer()),
     ('tfdif', TfidfTransformer()),
     ('nb', CalibratedClassifierCV(base_estimator = SGDClassifier(), cv=cv)),
])
Model = pipeline_sgd.fit(X_train, y_train)

n_top_labels = 3
probas = model.predict_probas(test["text"])
top_n_lables_idx = probas.argsort()[::-1][:n_top_lables]
top_n_probs = probas[top_n_lables_idx]
top_n_labels = label_encoder.inverse_transform(top_n_lables_idx.ravel())

results = list(zip(top_n_labels, top_n_probas))

 

output:
[(A, .80),
 (B, .10),
 (C, .10)]

The challenge that I am having with the above output is that it does not give me the top 3 labels/probabilities for each of the rows of text. For example when I run inference on a new set of documents(text), I only get a single output versus an output for each document(row).
The second challenge that I'm having is when I insert this into a dataframe using pd.Dataframe(data = results), I get the following:
|   | 0 | 1               |
|---|---|-----------------|
| 0 | A | [[.80,.10,.10]] |
| 1 | B | [[.85,.10,.05]] |
| 2 | C | [[.70,.20,.10]] |

It should be:
|   | 0     | 1               |
|---|-------|-----------------|
| 0 | A,B,C | [[.80,.10,.10]] |
| 1 | B,C,A | [[.85,.10,.05]] |
| 2 | C,B,A | [[.70,.20,.10]] |

Table A
| Text                                       | Predicted labels | Probabilities  |
|--------------------------------------------|------------------|----------------|
| Hello  World!                              | A,B,C            | [.80,.10,10]   |
| Have a nice Day!                           | B,C,A            | [.90,.05,05]   |
| It's a wonderful day in the neighborhood.  | C,A,B            | [.80,.10,10]   |


When I ran your code, I have a very strange shape for top_n_probs and I find it hard to get the labels back. The argsort and the code for calling the sorted values seems a bit weird.
Below i wrote a quick implementation that should work.
Using an example dataset:
from sklearn.model_selection import StratifiedKFold
from sklearn.pipeline import Pipeline
from sklearn.feature_extraction.text import CountVectorizer,TfidfTransformer
from sklearn.calibration import CalibratedClassifierCV
from sklearn.linear_model import SGDClassifier

import pandas as pd
import numpy as np

df = pd.read_csv('./smsspamcollection//SMSSpamCollection', sep='\t', names=["label", "message"])
df['label'][df['label']=='ham'] = np.random.choice(['hamA','hamB'],np.sum(df['label']=='ham'))
X_train = df['message']
y_train = df['label']

My labels look like this:
df['label'].value_counts()

hamB    2425
hamA    2400
spam     747

And running your code for fitting:
cv = StratifiedKFold(n_splits = 10, random_state = 42, shuffle = True)

pipeline_sgd = Pipeline([
     ('vect', CountVectorizer()),
     ('tfdif', TfidfTransformer()),
     ('nb', CalibratedClassifierCV(base_estimator = SGDClassifier(), cv=cv)),
])

model = pipeline_sgd.fit(X_train, y_train)

This should work:
n_top_labels = 3
probas = model.predict_proba(X_train[:5])
top_n_lables_idx = np.argsort(-probas)
top_n_probs = np.round(-np.sort(-probas),3)
top_n_labels = [model.classes_[i] for i in top_n_lables_idx]

results = list(zip(top_n_labels, top_n_probs))

pd.DataFrame(results)

    0   1
0   [hamB, hamA, spam]  [0.608, 0.38, 0.012]
1   [hamA, hamB, spam]  [0.605, 0.391, 0.004]
2   [spam, hamB, hamA]  [0.603, 0.212, 0.185]
3   [hamB, hamA, spam]  [0.521, 0.478, 0.001]
4   [hamB, hamA, spam]  [0.645, 0.352, 0.003]


Update to the accepted answer.
n = 3

probas = model.predict_proba(X_train)
top_n_lables_idx = np.argsort(-probas, axis=1)[:, :n]
top_n_probs = np.round(-np.sort(-probas),3)[:, :n]
top_n_labels = [model.classes_[i] for i in top_n_lables_idx]
    
results = list(zip(top_n_labels, top_n_probs))

pd.DataFrame(results)

This ensures that I get top 3 in both columns.
