Predictions from BertForSequenceClassification model keep changing across runs
I used the code in run_classifier.py to train a model for intent detection which is a multi-class classification problem. After training the model, when I used it for prediction, I found the predictions to be changing from one run to another. I'm trying to understand the reason for the same and how I can avoid this behavior.

Let's say that model is the fine-tuned sequence classification model. I get different results (logits) every time I run the following: 
logits = model1(input_ids, segment_ids, input_mask, labels=None)

Why is that so?


You probably forgot to deactivate the DropOut modules with `model.eval()`<|||||>Oh ok, I get it. That's helpful. Thanks!<|||||>I tried that and now I seem to be getting the same predictions for any input. In other words, the logits don't change with change in input. Not sure why this is happening?<|||||>Maybe a bug in your code?
If you can share a simple self-contained example exhibiting the behavior we can have a look.<|||||>That's unlikely as I tried the same code with other problems and got reasonable results. The problems on which I tried this code include MRPC task, sentiment prediction on IMDB dataset and intent detection on smalltalk data. The results are logical and reasonable.
The dataset on which I get this behaviour reported above has about 20 examples for each of the 4 intents viz. restaurant_search(0), booking_table(1), greet(2) and thanks(3). So, I was thinking if this could be due to the data having less number of examples. The sequence classifier class is basically bert model + a single hidden layer neural network with output layer as the number of labels. I feel that there's not enough data to train the last (classifier) layer. Here's my test data:
test_df = pd.DataFrame({'text': ["hey", "indian hotels"],
                        'label': [2, 0]})
I print the logits for each of the 2 test samples which are as follows:
tensor([[ 28.9354,  28.3292,  20.1560, -20.7804]])
tensor([[ 28.9354,  28.3292,  20.1560, -20.7804]])

The logits tensor doesn't change for any input text.
<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Hi @varun-nathan, were you ever able to solve this?  I'm having a very similar issue with ReformerForSequenceClassification.<|||||>Hi @varun-nathan and @jstremme, were either of you able to find the issue? I'm experiencing the same thing with ElectraForSequenceClassification!<|||||>> Hi @varun-nathan and @jstremme, were either of you able to find the issue? I'm experiencing the same thing with ElectraForSequenceClassification!

Hi @mollha, I managed to solve this but can't remember exactly what I did.  Did you try `model.eval()`? Also, make sure you are training on more than just a few sample records and using a large enough model in terms of neurons per layer etc. <|||||>Thanks! I was already using model.eval(), but my dataset size was too small (around 1000). After increasing to 15000 I am getting much better results.<|||||>Excellent!