Intel macOS system with AMD 6900XT GPU, using MPS: cannot get any usable result back from any model
### System Info

- `transformers` version: 4.28.0.dev0
- Platform: macOS-10.16-x86_64-i386-64bit
- Python version: 3.10.10
- Huggingface_hub version: 0.13.3
- Safetensors version: not installed
- PyTorch version (GPU?): 2.1.0.dev20230402 (False) says false but I am using PyTorch on GPU via `mps`
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using GPU in script?: Yes
- Using distributed or parallel set-up in script?: No

OS: macOS x64 Ventura 13.3
Hardware: Intel system with AMD 6900XT GPU

### Who can help?

@sgugger (possibly PyTorch related?)

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

My goal is to try models like Alpaca and GPT4ALL on my home Intel macOS system using my AMD 6900XT GPU.

I tried and failed to get tloen's Alpaca Lora UI and the LLaMa_MPS project running (details below).  In investigating this, I have run this simple test script I wrote to test using MPS to get a result from `t5-small`. As a base for the code I used https://github.com/huggingface/transformers/issues/22122#issuecomment-1475302212 as the discussions in that thread indicated it should work fine:

```
import torch
import transformers
from transformers import T5ForConditionalGeneration, AutoTokenizer

print("PyTorch version: ", torch.__version__)
print("transformers version: ", transformers.__version__)
print()

tokenizer = AutoTokenizer.from_pretrained('t5-small', model_max_length=512)
input_string = 'translate English to German: "The house is wonderful."'
print("Input string:", input_string)

## On CPU
print("Trying CPU")
model_cpu = T5ForConditionalGeneration.from_pretrained('t5-small', device_map='auto')
print("Running on: ", model_cpu.device)

inputs = tokenizer(input_string, return_tensors='pt').input_ids
outputs = model_cpu.generate(inputs, max_length=200)
print("Decoded Output: ", tokenizer.decode(outputs[0]))
print("raw output: ", outputs)

## On MPS
print()
print("Trying mps")
model_mps = T5ForConditionalGeneration.from_pretrained('t5-small')
model_mps = model_mps.to('mps')
print("Running on: ", model_mps.device)

inputs_mps = tokenizer(input_string, return_tensors='pt').input_ids
inputs_mps = inputs_mps.to('mps')
outputs = model_mps.generate(inputs_mps, max_length=200)
try:
    print("Decoded Output: ", tokenizer.decode(outputs[0]))
except Exception as e:
    print(e)
    print("raw output: ", outputs)
```

This produces the following result; CPU works fine, MPS produces a strange, repeating and very long result which throws an exception when being decoded:
```
PyTorch version:  2.1.0.dev20230402
transformers version:  4.28.0.dev0

Input string: translate English to German: "The house is wonderful."
Trying CPU
Running on:  cpu
Decoded Output:  <pad> "Das Haus ist wunderbar."</s>
raw output:  tensor([[    0,    96, 17266,  4598,   229, 19250,   535,     1]])

Trying mps
Running on:  mps:0
out of range integral type conversion attempted
raw output:  tensor([[                   0, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808, -9223372036854775808,
         -9223372036854775808, -9223372036854775808]], device='mps:0')
```

Before trying this script, I first tried tloen's alpaca-lora GUI which has MPS support (https://huggingface.co/spaces/tloen/alpaca-lora/blob/main/app.py), and LLaMA_MPS (https://github.com/jankais3r/LLaMA_MPS)

Both of these exhibit the same or a very similar problem: the code appears to run fine, it uses my AMD 6900XT GPU (as detected via macOS Activity Manager), but I either get no output at all (alpaca-lora), or the output is corrupted, showing each token as a `??` symbol (LLaMa_MPS).

I am an AI newbie so I'm unsure how to try and debug this but I am pretty sure that all three of these examples are exhibiting the same problem.

I don't know if it's an issue in transformers or in PyTorch which provides the MPS backend, so I thought I'd start here.

Note: I'm running PyTorch 2.1 dev version because trying 2.0.0 with alpaca-lora gave me the error `RuntimeError: MPS does not support cumsum op with int64 input` - this was fixed by updating to 2.1-dev and Ventura 13.3.  I have tested LLaMa_MPS with PyTorch 2.0.0 with the same result, so I don't believe it's specific to 2.1-dev.

Thanks in advance for any help.

### Expected behavior

The mps code shown above should output the same result as the CPU code.
I just did some mores searching and realised that -9223372036854775808 is "the smallest value that can be stored in a 64-bit signed integer" and that this issue looks the same as the Pytorch MPS issue reported here: https://github.com/pytorch/pytorch/issues/92311

In that thread, someone reported this workaround:
> Just replace argmax(...) with max(...).indices for instance replace output.argmax(dim=1) with output.max(dim=1).indices

I don't know if this helps me here as I'm not running any PyTorch code directly, but rather calling it through Transformers.

Anyway I guess this likely shows this isn't a transformers issue but is in PyTorch, and has already been reported. In which case, apologies for not noticing this before reporting this. <|||||>yea it looks like this is something that needs to be fixed on the PyTorch side. We can't just change the code of `generate` on our side to accommodate MPS devices and this is clearly a bug in PyTorch. So we just have to wait a bit for them to fix it.<|||||>Understood, thanks for the quick reply.