AttributeError: 'tuple' object has no attribute 'softmax'
I get the following error when I use the pytorch transformers, It used to work just fine in the previous pretrained-bert,
Original code: https://github.com/ceshine/pytorch-pretrained-BERT/blob/master/notebooks/Next%20Sentence%20Prediction.ipynb

Code which has error:
model.eval()

res = []

mb = progress_bar(eval_dataloader)
for input_ids, input_mask, segment_ids in mb:
    input_ids = input_ids.to(device)
    input_mask = input_mask.to(device)
    segment_ids = segment_ids.to(device)

    with torch.no_grad():
        res.append(nn.functional.softmax(
            model(input_ids, segment_ids, input_mask), dim=1
        )[:, 0].detach().cpu().numpy())
        
res = np.concatenate(res)


Error stacktrace:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-37-a816e24060d8> in <module>
     24     with torch.no_grad():
     25         res.append(nn.functional.softmax(
---> 26             model(input_ids, segment_ids, input_mask), dim=1
     27         )[:, 0].detach().cpu().numpy())
     28 

/common/users/rs1693/my_venv/venv_bert/lib64/python3.6/site-packages/torch/nn/functional.py in softmax(input, dim, _stacklevel, dtype)
   1261         dim = _get_softmax_dim('softmax', input.dim(), _stacklevel)
   1262     if dtype is None:
-> 1263         ret = input.softmax(dim)
   1264     else:
   1265         ret = input.softmax(dim, dtype=dtype)

AttributeError: 'tuple' object has no attribute 'softmax'

I read many posts where they say to do the following:(But not sure where in the code I have to make these changes)
1. disable aux_logits when the model is created here by also passing aux_logits=False to the inception_v3 function.

2. Edit your train function to accept and unpack the returned tuple here to be something like:

output, aux = model(input_var)

But where in the above function I have to do this?

HI,
I have the same problem!
What was the solution here?<|||||>Me too.<|||||>Need more information like version of python/pytorch/transformers (all the information requested in the issue templates actually)<|||||>> Need more information like version of python/pytorch/transformers (all the information requested in the issue templates actually)

I am experiencing this issue as well with the BertForNextSentencePrediction model and not having much luck with a solution. I'm using macOS Mojave 10.14.6, python 3.7, pytorch 1.3.1 and transformers 2.2.1. 

Please let me know if there is any more details I can provide.  Thanks!<|||||>You should open a new issue with a clean code example we can test and the associate full error message.