RuntimeError when running Reformer model
## Environment info
     
- `transformers` version: 2.10.0
- Platform: Linux-5.4.0-1034-aws-x86_64-with-debian-buster-sid
- Python version: 3.6.9
- PyTorch version (GPU?): 1.6.0+cu101 (True)
- Tensorflow version (GPU?): not installed (NA)
- Using GPU in script?: yes
- Using distributed or parallel set-up in script?: no

### Who can help
 --> @patrickvonplaten

## Information

Model I am using (Bert, XLNet ...): Reformer

The problem arises when using:
* [x] the official example scripts: (give details below)
* [ ] my own modified scripts: (give details below)

The tasks I am working on is:
* [ ] an official GLUE/SQUaD task: (give the name)
* [ ] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Run the example code from [here](https://huggingface.co/google/reformer-crime-and-punishment?text=My+name+is+Julien+and+I+like+to):
```
model = ReformerModelWithLMHead.from_pretrained("google/reformer-crime-and-punishment")
tok = ReformerTokenizer.from_pretrained("google/reformer-crime-and-punishment")
tok.decode(model.generate(tok.encode("A few months later", return_tensors="pt"), do_sample=True,temperature=0.7, max_length=100)[0])
```
<!-- If you have code snippets, error messages, stack traces please provide them here as well.
     Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
     Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.-->

## Expected behavior

RuntimeError: Overflow when unpacking long (more details below)

```
<ipython-input-33-0a824540b4e0> in <module>
      2 tok = ReformerTokenizer.from_pretrained("google/reformer-crime-and-punishment")
      3 tok.decode(model.generate(tok.encode("A few months later", return_tensors="pt"), 
----> 4                           do_sample=True,temperature=0.7)[0])

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/torch/autograd/grad_mode.py in decorate_context(*args, **kwargs)
     13         def decorate_context(*args, **kwargs):
     14             with self:
---> 15                 return func(*args, **kwargs)
     16         return decorate_context
     17 

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/transformers/modeling_utils.py in generate(self, input_ids, max_length, min_length, do_sample, early_stopping, num_beams, temperature, top_k, top_p, repetition_penalty, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, length_penalty, no_repeat_ngram_size, num_return_sequences, attention_mask, decoder_start_token_id, use_cache, **model_specific_kwargs)
   1179                 attention_mask=attention_mask,
   1180                 use_cache=use_cache,
-> 1181                 model_specific_kwargs=model_specific_kwargs,
   1182             )
   1183 

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/transformers/modeling_utils.py in _generate_no_beam_search(self, input_ids, cur_len, max_length, min_length, do_sample, temperature, top_k, top_p, repetition_penalty, no_repeat_ngram_size, bad_words_ids, bos_token_id, pad_token_id, eos_token_id, decoder_start_token_id, batch_size, encoder_outputs, attention_mask, use_cache, model_specific_kwargs)
   1221             )
   1222 
-> 1223             outputs = self(**model_inputs)
   1224             next_token_logits = outputs[0][:, -1, :]
   1225 

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/transformers/modeling_reformer.py in forward(self, input_ids, position_ids, attention_mask, head_mask, inputs_embeds, num_hashes, labels, do_output_hidden_states, do_output_attentions)
   1738             num_hashes=num_hashes,
   1739             do_output_hidden_states=do_output_hidden_states,
-> 1740             do_output_attentions=do_output_attentions,
   1741         )
   1742 

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/transformers/modeling_reformer.py in forward(self, input_ids, attention_mask, position_ids, head_mask, inputs_embeds, num_hashes, do_output_hidden_states, do_output_attentions)
   1588             num_hashes=num_hashes,
   1589             do_output_hidden_states=do_output_hidden_states,
-> 1590             do_output_attentions=do_output_attentions,
   1591         )
   1592         sequence_output = encoder_outputs.hidden_states

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/transformers/modeling_reformer.py in forward(self, hidden_states, attention_mask, head_mask, num_hashes, do_output_hidden_states, do_output_attentions)
   1324             all_attentions,
   1325             do_output_hidden_states,
-> 1326             do_output_attentions,
   1327         )
   1328 

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/transformers/modeling_reformer.py in forward(ctx, hidden_states, layers, attention_mask, head_mask, num_hashes, all_hidden_states, all_attentions, do_output_hidden_states, do_output_attentions)
   1220                 head_mask=layer_head_mask,
   1221                 num_hashes=num_hashes,
-> 1222                 do_output_attentions=do_output_attentions,
   1223             )
   1224             attn_output = layer_outputs.attn_output

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
    720             result = self._slow_forward(*input, **kwargs)
    721         else:
--> 722             result = self.forward(*input, **kwargs)
    723         for hook in itertools.chain(
    724                 _global_forward_hooks.values(),

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/transformers/modeling_reformer.py in forward(***failed resolving arguments***)
   1111             # for dropout and save seed for forward fn in backward
   1112             # to have correct dropout
-> 1113             self._init_feed_forward_seed()
   1114             # Y_2 = X_2 + g(Y_1)
   1115             hidden_states = hidden_states + self.feed_forward(attn_output)

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/transformers/modeling_reformer.py in _init_feed_forward_seed(self)
   1075         else:
   1076             # CPU
-> 1077             self.feed_forward_seed = int(torch.seed() % sys.maxsize)
   1078             torch.manual_seed(self.feed_forward_seed)
   1079 

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/torch/random.py in seed()
     43 
     44     if not torch.cuda._is_in_bad_fork():
---> 45         torch.cuda.manual_seed_all(seed)
     46 
     47     return seed

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/torch/cuda/random.py in manual_seed_all(seed)
    111             default_generator.manual_seed(seed)
    112 
--> 113     _lazy_call(cb)
    114 
    115 

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/torch/cuda/__init__.py in _lazy_call(callable)
    133 def _lazy_call(callable):
    134     if is_initialized():
--> 135         callable()
    136     else:
    137         # Don't store the actual traceback to avoid memory cycle

~/anaconda3/envs/Reformer/lib/python3.6/site-packages/torch/cuda/random.py in cb()
    109         for i in range(device_count()):
    110             default_generator = torch.cuda.default_generators[i]
--> 111             default_generator.manual_seed(seed)
    112 
    113     _lazy_call(cb)

RuntimeError: Overflow when unpacking long
```

Hey @albusdemens, could you maybe update your transformers version to 4.0.0?<|||||>Thanks @patrickvonplaten, that fixed the issue! 